FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals

Background: 

In recent years, technology development of deep learning has advanced to the level of high realism in creating synthetic videos. Such deep fake videos pose a big threat to video authentication for security, safety, privacy protection, as well as health-care safety. 

Technology Overview: 

We invented a novel approach to detect synthetic content in portrait videos, as a preventive solution for the emerging threat of deep fakes. The core of this technique is to detect biological signals (e.g., heart rate) from face regions of image videos, and apply the learning model to distinguish the real and fake, based on the heart-rate consistency (real) and inconsistency (fake). Our approach achieved over 90% accuracy. Biological signals are a unique signature for distinguishing face videos and real face videos. It offers a powerful tool for detecting fake subjects from private media as well as social media. This new technique exploits spatial coherence and temporal consistency of biological signals, for both pairwise and general authenticity classification, which has never been done before.

Advantages: 

• It offers a powerful tool for detecting fake subjects from private media as well as social media.
• This new technique exploits spatial coherence and temporal consistency of biological signals, for both pairwise and general authenticity classification, which has never been done before.
• The biological signal detection is unique and the biological signal map is constructed to train a network for authenticity classification. The generalized and interpretable deep fake detector can work in-the-wild.

 

Intellectual Property Summary: 

Pending patent  application US 2021-0209388 A1 

Additional Information: 

 FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals, IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2020.3009287