A new iOS app for Unreal Engine uses your iPhone to capture your facial expressions and animate an onscreen character in real time.
Epic Games, the company behind Unreal Engine, has released a new iOS app that lets developers record facial expressions from an iPhone and map them onto characters in real-time. The Live Link Face app is supposed to make it a lot easier to animate avatars in the engine that powers some of the most popular games out there, from Fortnite to Borderlands.
According to a new blog post, Live Link Face will let developers stream “high-quality facial animation in real-time from your iPhone directly onto characters in Unreal Engine.” It uses Apple’s ARKit and the iPhone’s TrueDepth front-facing camera to track a face, and transmits the data directly into Unreal Engine. Epic says it can be used by a professional on a stage with multiple actors, or just a single artist at a desk.
“The Live Link Face app harnesses the amazing facial capture quality of iPhone ARKit and turns it into a streamlined production tool,” noted Verizon Media Director of Animation Technology Addy Ghani. “At RYOT [a Verizon-owned studio], we believe in the democratization of capture technology and real-time content, and this solution is perfect for a creator at home or a professional studio team like ours.”
Live Link Face can stream to multiple machines at once, and “robust timecode support and precise frame accuracy enable seamless synchronization with other stage components like cameras and body motion capture,” according to Epic. Users get a CSV of raw blendshape data and an MOV from the phone’s front-facing video camera, with timecodes.
Epic likely sees the tool being used right at a developer’s desktop rather than having to perform in a mo-cap studio. Of course, this use case will only be ideal for animating the face — a studio will still be necessary for choreographed scenes involving the whole body.