In addition, in our pipeline the crops can also be generated based on the face landmarks identified in the previous frame, and only when the landmark model could no longer identify face presence is the face detector invoked to relocalize the face. Instead it allows the network to dedicate most of its capacity towards coordinate prediction accuracy. Having the face accurately cropped drastically reduces the need for common data augmentations like affine transformations consisting of rotations, translation and scale changes. Our ML pipeline consists of two real-time deep neural network models that work together: A detector that operates on the full image and computes face locations and a 3D face landmark model that operates on those locations and predicts the approximate 3D surface via regression. AR effects utilizing the 3D facial surface. The analysis runs on CPU and has a minimal speed/memory footprint on top of the ML model inference.įig 1.
Under the hood, a lightweight statistical analysis method called Procrustes Analysis is employed to drive a robust, performant and portable logic.
The face transform data consists of common 3D primitives, including a face pose transformation matrix and a triangular face mesh. It establishes a metric 3D space and uses the face landmark screen positions to estimate a face transform within that space. Utilizing lightweight model architectures together with GPU acceleration throughout the pipeline, the solution delivers real-time performance critical for live experiences.Īdditionally, the solution is bundled with the Face Transform module that bridges the gap between the face landmark estimation and useful real-time augmented reality (AR) applications. It employs machine learning (ML) to infer the 3D facial surface, requiring only a single camera input without the need for a dedicated depth sensor. MediaPipe Face Mesh is a solution that estimates 468 3D face landmarks in real-time even on mobile devices. This notice and web page will be removed on April 3, 2023. For more information, see the new MediaPipe Solutions site. As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe Solution. This site uses Just the Docs, a documentation theme for Jekyll.Īttention: Thank you for your interest in MediaPipe Solutions.
0 kommentar(er)
