GAN-SHI: DETECTING DEEPFAKES BY ANALYZING GAZE INCONSISTENCY

Abstract

Manipulative technologies such as DeepFakes, Face2Face, FaceSwap, and NeuralTextures endanger the integrity of digital media. The state of the art in DeepFake detectors perform at above average levels when it comes to detection of media manipulation, however, when a new manipulation tool is used, they commonly fail to generalize due to the overfit to manipulation-specific artifacts. In this research paper, we propose GAN-shi, a DeepFake detection technique that utilizes the observation of physical Consistency of Binocular Gaze Patterns. Our technique is founded on the notion that while generative models are capable of producing aesthetically pleasing faces, they are fundamentally unable to produce consistent Binocular Gaze -- in natural Human faces, both Eyes coalesce to a single focal point, while in generated faces, there is a statistically significant divergence of Gaze. GAN-shi is capable of extracting 468 Three-Dimensional facial landmarks, is capable of determining 3-D Gaze Vector for both Eyes, and is capable of assessing the consistency of the corneal specular highlight and the Gaze Shape. The results of these assessments are yielded as a 12-dimensional vector which is classified by a very small Multi-Layer Perceptron. In terms of performance, GAN-shi was assessed against the FaceForensics++ framework and achieved the highest accuracy, being 97.5%, in the assessment of 4 distinct media manipulation methods, in comparison to the other evaluated frameworks, XceptionNet (88.6%) and EfficientNet-B4 (93.2%). In addition, during a cross-dataset assessment on Celeb-DF (v2), GAN-shi attained a 94.5% Area-Under-The-Curve (AUC) while XceptionNet attained a 65.3% AUC, further demonstrating its generalization capability. Additionally, GAN-shi is of extremely low computational cost, requiring only 5 milliseconds per frame on a CPU. Thus, the framework was designed with the capability of real-time use in mind.

Keywords

Deepfake detection, gaze estimation, eye tracking, MediaPipe, face manipulation, FaceForensics++, digital forensics.