Epic Games launches iPhone app for 3D facial animation capture via Face ID

Epic Games today released a new iPhone app that leverages Face ID to make real-time facial animation capture accessible to all developers.

Introducing Live Link Face

Called Live Link Face, the free app makes it easy to capture 3D facial animation for virtual production, in real time, using an iPhone with Apple’s TrueDepth camera for Face ID. The data can be sent to Unreal Engine to map and visualize facial expressions to characters.

Live Link Face streams high-quality facial animation in real time from your iPhone directly onto characters in Unreal Engine. The app’s tracking leverages Apple’s ARKit and the iPhone’s TrueDepth front-facing camera to interactively track a performer’s face, transmitting this data directly to Unreal Engine via Live Link over a network.

Recorded data can be fine-tuned in animation tools.

The user can play back the reference video on their phone, delete takes or share them via AirDrop. Writing on its blog, Epic Games noted that the system can be incorporated into complex motion-capture stages because it uses timecode to sync with other capture devices.

When the app is used while sitting at your desk rather than wearing a head-mounted rig with a motion-capture suit, it captures additional head and neck rotation data that in turn provides more freedom of movement for your digital avatars using nothing more than an iPhone.

“Animators can also take advantage of the option to record both the raw blendshape data (CSV) and a front-facing video (MOV), each striped with timecode, to use as reference material for the performance if further adjustments need to be made in-engine,” said the company.

For further information, check out Epic’s support document.

Key features

The following features are supported:

Facial animation via front-camera and ARKit:

  • Stream out the data live to an Unreal Engine instance via Live Link over a network
  • Drive a 3D preview mesh, optionally overlaid over the video reference on the phone
  • Record the raw facial animation data and front-facing video reference footage

Timecode support for multi-device synchronization:

  • Select from the iPhone system clock, an NTP server, or use a Tentacle Sync to connect with a master clock on stage
  • Video reference is frame accurate with embedded timecode for editorial.

Control Live Link Face remotely with OSC:

  • Trigger recording externally so actors can focus on their performances
  • Capture slate names and take numbers consistently
  • Extract data automatically for archival

Browse and manage the captured library of takes within Live Link Face:

  • Delete takes, share via AirDrop
  • Play back the reference video on the phone

Epic Games is also working on Unreal Engine 5 (watch the impressive Unreal Engine 5 demo running on a PlayStation 5) that will be coming to Apple’s iOS and macOS devices too in 2021.

You can download Live Link Face from the App Store for free.