The Diva3d Media Player Display module for DIVA3D allows the user to view the selected movie as if it would have been opened in Windows Media Player. The player.dll exports it's function to any i/o module that resides in DIVA3D's directory so the viewing of the movie will be done through these modules.

The preview module allows the user to display the first frame contents of a video buffer.

The StreamIO module is the "heart" of all operations taking action inside DIVA3D. It is this module that manages all the video stream buffers and all the input/output modules. When the user loads a video using an I/O module, StreamIO module creates a video stream buffer and establishes an association between a video buffer and the video.

The FVAL IO module provides handling capabilities to the DIVA3D software package for video stream buffers which contain float values (fval video buffers). In order to support these video stream buffers the concept of an *.fval file, where DIVA 3D stores the data from fval video stream buffers, is assumed.

The IVAL IO module provides handling capabilities to the DIVA3D software package for video stream buffers which contain integer values (ival video buffers). In order to support these video buffers the concept of an *.ival file where DIVA 3D stores the data from ival video buffers is assumed.

This module includes video stream management operations which are described below. State of Video Streams: Displays the Video Streams dialog box where the user can get information about the imported video streams and their attributes. By double clicking on a video stream a DIVA3D Media Player window is created where the user can view the selected video stream.

This module includes 3D signal transform operations, like 3D Fast Fourier Transform (FFT), Inverse 3D Fast Fourier Transform, 3D Discrete Cosine Transform (DCT), Inverse 3D Discrete Cosine Transform, 3D convolution (convolution between a video stream and a 3D filter buffer) and correlation (3D correlation of two video stream buffers).

This module corresponds to various 3D processing operations of video streams. In most cases, the functions require an input video stream, selected through the Video Streams dialog box, and possibly some related input parameters, given through additional dialog boxes.

The Color Transformation Module is used to transform a color video stream to various color spaces. The following color transformations are supported.

Color Transformation functions available.

The output is unsigned character or floating point color video frame streams.

The Disparity Calculation Module calculates the disparity between the right and left video channel of a stereo video sequence. The user can specify the parameters of the block-matching algorithm used for the disparity calculation, such as the block size, the search region, and the search method.

The user can visualize the disparity vectors calculated with the disparity calculation.

This module includes several spatiotemporal video filtering operations. All the operations are supported by the CameraIO Module and applied either in one channel (in case of single channel (grayscale) video streams or on all three channels of a color video stream.

This module includes various 3D binary mathematical morphology operations, which require that a binary video stream buffer containing the 3D structuring element exists.

Motion compensated filtering performs filtering along moving object trajectories. Therefore, it produces superior filtering results, when compared to standard 3D or 2D video filtering that does not take into account motion information.

Motion estimation is performed by a three-step search block matching method on the noisy image sequence.

The operations of this module are supported by the CameraIO module.

Motion estimation is an important task in video analysis. It can be used to find the object or camera motion, to identify moving objects and to find their velocity.

The Motion Estimation module for DIVA3D allows the user to estimate the motion fields between two frames of a given video stream (color or grayscale) and to display the estimated motion field.

If the input video stream is a color one (24bit), it is automatically transformed to a grayscale video stream, before the block matching function is executed. The operation of this module is supported by the CameraIO module.

The Visible Video Watermarking module is used for embedding a visible logo in each frame of a video stream.

The user can determine the spatial coverage of each video frame by the logo, the watermark visibility and the position of the logo image in each video frame.

Visible watermarking modules stores its results in a video stream.

This module includes various analysis operations. The operations in this module, unless otherwise stated, are applied either in one channel (in case of single channel (grayscale) video streams) or on all three channels of a color video stream.

The Dialogue Detection module is used to detect dialogue, monologue or visual speech absence in a video stream based on alternating lip activity and the number of actors that produce visible speech alterations during a video scene.

The results of the Dialogue Detection module are saved in XML format. The dialogue information is saved in the following way : each actor appearance has value 'Dialogue' if dialogue is detected, 'Monologue', if an actor speaks, while other actors are silent in a 'temporal' window, and finally, 'No_Dialogue' is detected if nobody speaks, i.e. there is no lip activity.

The Face Clustering module clusters faces appearing in a video, i.e. it can count and label the different persons/actors appearing in a video file. It presents to the user the clusters produced by an unsupervised clustering algorithm and gives him/her the ability to correct false face clustering/classification results.

Face Clustering saves the face cluster results in an XML file.

Moreover, Face Clustering saves the cluster results in the hard disk in a special format to provide a training set for the Face Recognition module of DIVA3D.

The DIVA3D Video Tracking Module is designed to perform face detection and face/object tracking within DIVA3D. It integrates several face detectors and face/object tracking techniques. The tracking algorithms were designed to be robust when running on long video sequences. If the shot cuts are provided in an XML file, then they can be taken into account for face/object tracking, since each face/object appearance typically stops at each shot cut. This module can also perform facial detection (by choosing among three different face detector algorithms) to initialize facial region tracking. The detected or tracked facial or object regions of interest (ROI) information are stored in an XML file containing information for each video frame, called face/object instances that are grouped in face/object appearences lasting for a certain time period (number of video frames).

The Face Recognition module provides actor/person recognition in movies using facial features. It supports both system training procedure (using either know actor facial photos or the results of the face clustering module) and face recognition.

The system training option is used to train the Face Recognition system in order to recognise the specified actor in video streams. The training procedure has significant computational load, depending on the number of facial images used in training.

Once the system is trained, face recognition is performed for all facial regions that are detected in each video frame.

The Face Recognition module writes its results in an XML file.

The Facial Expressions Recognition module provides functions to identify the expressive state of a person at every video frame. The set of facial expressions that can be recognized includes anger, disgust, fear, happiness, sadness, surprise and the neutral state.

The output of the Facial Expression Recognition Module is an XML file that includes the facial expression characterization for every facial ROI on each video frame.

The Facial Expression Tracking module of DIVA3D allows the user to follow the motion of selected facial feature points (e.g lip corners) between two frames of a given video stream (color or grayscale) and to display a Candide following the facial deformations. These deformations are used subsequently to detect facial expressions. If the input video stream is color (24bit), it is automatically transformed to a greyscale one, before the facial feature tracking and facial expression recognition is executed.

The output of the Facial Expression Recognition Module is an XML file that includes the facial expression characterization for every facial ROI on each video frame and one VRML file per face per video frame describing the deformed Candide grid.

The Facial Feature Detection Module calculates the coordinates of eye and mouth regions and eye centres and mouth corners within each facial ROI in a video frame. For every detected face region, 16 points are extracted. For each eye ROI 2 points define the area corners and 1 point the eye centre. For each mouth 2 points define the area corners and 2 other points define the mouth corners. That is: for eye/mouth areas, the cooordinates are written out in the following way: the first two coordinates are the row and column coordinates of the lower-right point. The rest of the coordinates are a path to the rest of the points, with respect to the lower-right point. For eye centers/mouth corners, the first two numbers are the corresponding coordinates

The Facial Features Detection Module saves its results in an XML file.

The Facial Pose Estimation module allows the user to estimate the human facial pose for each face within a video frame vs the camera coordinates. The Facial Pose Estimation module for DIVA3D consists of two sub-modules: the Facial Pose Angle Estimation and the Facial Pose Detection. The first sub-module allows the user to estimate the facial pose angles of a person in every frame of a video sequence. The user has to pre-specify the initial face pose angles in the x-y-z axes in the first video frame. The results produced describe the angles of deflection from the absolute frontal position in the x-y axes. The second sub-module detects facial poses and finds pose label (e.g., frontal, profile) by using reference facial pose images. In this case, pose detection is performed on any video sequence containing moving human faces based on an image representing a reference facial pose. It is preferable that the input reference pose image belongs to the same person whose pose is to be detected. In its absence, any facial pose image representing a particular facial pose could be used. Alternatively, the user could use one or more of the five default pose images contained in the module (frontal, left, right, mid-right and mid-left) by clicking the respective box displayed within the user interface. The module can operate either on entire video frames or on facial ROIs produced by the DIVA3D face detection/tracking module.

The results are stored in an XML file.

The human body motion tracking module detects and tracks human bodies in an input video stream/file showing humans. It decides whether a video object is human person or not. It also finds the human body silhouette in a video frame and detects human body posture (pose).

The user can select one of the following options :

- Boundary Detection/Boundary Tracking: They are selected, when the user wants to detect/track an object (possibly human body) silhuette.

- Pose estimation: It produces a body posture (pose) label.

The results are stored in an XML file.

Digital movies can be split into video scenes that are subsequently split into video shots. The Shot Cut Detection module provides shot boundary detection and shot information management capabilities to DIVA3D.

To automatically detect the shot cuts, the module uses a combined method of the entropy search method and a new one which is based on the singular value decomposition. In this version, the user can even find shot transitions. The module also provides shot transition characterization, e.g. cut, or fade.

The Shot Type Detection Module extracts different grammar elements from a video sequence (shot size, shot type and actor position). The module takes as input an XML file with the face detection or tracking results. The main output of the module is an XML file with shot type characterization results (e.g. long shot, medium close-up, extreme close-up). The user can optionally generate a text file with frame-based results or an AVI file.

The Uncalibrated 3D Object/Face Reconstruction Module extracts a precise 3D model of an object (or a 3D face) from an input video file or multiple images depicting different poses of the given object from various viewpoints. The 3D face/object modelling uses reference points (landmarks) on the various images that are either derived automatically or are user defined. As a byproduct, it calibrated the camera, i.e. it finds the relative position of the camera to the 3D object coordinates. Furthermore, it can create more detailed 3D object/face models by employing generic 3D object/face models, if available (e.g. Candide model for faces) that are subsequently deformed to match the image landmarks. Its output is a VRML file containing the derived 3D face/object model.

The Visual Speech Detection module is used to detect visual speech (i.e. lip activity) in a video shot based on image intensity changes in the mouth/lip area.

There are two possible extracted lip activity labels, namely 'No Lip Activity Present' and 'Lip Activity Present' for facial ROI, according to the visual speech detection algorithm.

Visual Speech Detection module can store its results in an XML file and can also produce a video file, whose frames, where speech activity is detected, are marked.