hoshinolina/arkit-notes: Reverse engineering Apple's FaceKit and ARKit tracking

An author reverse-engineered Apple's internal FaceKit framework, revealing that ARKit face tracking relies on the CaraNet machine learning model using only RGB image input rather than TrueDepth or infrared camera data. The research demonstrates how FaceKit generates 51 blend shapes, handles user identity tracking through an ID fitting model, and computes synthetic facial landmarks and meshes directly from blend shape data. Open article →