Face-reenactment aims to generate a synthesized video that animates a target face based on the movements captured from a driving video, while preserving the identity conveyed by the source image.

The primary objective of the synthesized MetaHuman dataset is to facilitate the evaluation of generated images through cross-reenactment algorithms.

To evaluate cross-reenactment generated images use the proposed protocol outlined in [1], depicted below.

The protocol involves two video sequences, denoted as A and B, representing distinct identities. For each frame, the head pose and expression are identical in both sequences. The evaluation protocol can be summarized as follows:

1. Select a frame from video sequence A as the source image.

2. Select a driving video sequence, comprising video frames of identity B, to animate the source image. The head pose and expression in all frames of the driving video correspond to those of the source face.

3. Input the source image and driving video frames into a face-reenactment method to generate a new video sequence representing source identity A. This generated video sequence should accurately reflect the facial expressions and movements that match those of the driving video sequence.

4. Assess the accuracy of the generated frames by comparing them to the ground-truth video using metrics such as SSIM, CSIM, LPIPS, AKD, FID, and FVD.

Event cameras are a new family of vision sensors that differ completely from conventional cameras: instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes and output the event stream as a sequence of tuples [t, x, y, p] that encode the time t, pixel coordinates (x, y) and p the sign of the brightness changes.

The database is recorded using 2 cameras and a laptop. The DAVIS346 [1], [9], is used to record events. Its resolution is 346 × 260. For each event [t, x, y, p], x ∈ [0, 345] and y ∈ [0, 259] and the data is saved with the DV software. The RGB and thermal recordings were acquired with the dual sensor, visible and thermal, camera FLIR Duo R developed by FLIR Systems. The visible sensor is a CCD sensor with a pixel resolution of 1920 × 1080. The thermal sensor of this camera is an uncooled VOx microbolometer and has a pixel resolution of 640 × 512. Figure below shows the two cameras used in the database collection.

The data collection took place in a controlled environment. Videos were recorded in an empty room equipped with two lightings in addition to the room lights. The cameras were put about 1 meter from the ground and facing perpendicularly to the background (the wall of the room) at about 6 meters from it. Figure below shows an example of an individual from our database, recorded simultaneously with an event-based camera, RGB camera, and thermal camera.

A download link for the database compressed and a password for decrypting the compressed Gait3 ZIP files will be provided after receiving the duly signed license agreement. Please fill in the license agreement and send a scanned copy by e-mail at gait3@eurecom.fr