Artificial Intelligence Researchers From China Propose Head Swap (HeSer) For A Few Stroke Head Swap In The Wild

This Article Is Based On The Research Paper 'Few-Shot Head Swapping in the Wild'. All Credit For This Research Goes To The Researchers 👏👏👏

Please Don't Forget To Join Our ML Subreddit

Deepfakes are fake media in which a person’s likeness is replaced with someone else’s in an existing photograph or video. Although false information is not new, deepfakes use advanced machine learning and artificial intelligence techniques to modify or synthesize visual and auditory content with high potential for deception. Not only facial features, but also head shapes and hairstyles have a significant impact on the perception of human identity.

The head-swapping task attempts to perfectly place a source head on a target body, which is essential in various entertainment contexts.

Although the face change has received a lot of attention, few have studied the head change so far, especially in the environment a few shots. Many papers have attempted to apply face-swapping techniques to the head-swapping task. But head swapping is an inherently difficult task due to its special requirements:

  1. Capturing the structural information of an entire head and non-stiff hair is necessary not only for accurate face identification and expression modeling, but also for head swapping. Therefore, previous face-swapping identity extraction algorithms cannot be directly applied to head-swapping.
  2. Changing head shapes and hairstyles would result in a significant mismatch of the region between the leading edges and switched backgrounds.
  3. It is important to manage color disparity between source and target skins.

Researchers from Baidu and South China University of Technology are forging ahead by developing the Head Swapper (HeSer), which can move the entire head, not just the face. The main goal was to align the source head with the destination in a single mixer that could cleanly handle color and background offsets.

https://arxiv.org/pdf/2204.13100.pdf

The Head Swapper (HeSer) uses two delicately constructed modules to perform a head swap in a few taps. The team first created a Head2Head Aligner to holistically move position and expression information from the target to the source head by looking at multi-scale data. It fully aligns the source head to the same position and expression as the target image. By incorporating multi-scale local and global information from both photos, a style-based generator significantly balances identity, expression, and pose information. Additionally, subject-specific adjustment could further increase identity preservation and posture consistency.

Next, they introduce a Head2Scene Blender to solve the problems of skin color fluctuation and head background mismatch in the swap method by simultaneously changing face skin color and filling in incompatible spaces on the background around the head.

The researchers used a semantic-guided color reference creation method and a Blending UNet in particular to achieve a seamless blend. They tested their model and found that it performed better in changing heads in a variety of settings.

Recent generative models have had a significant impact on identity security, image authenticity, and other areas. Therefore, the team plans to share HeSer’s findings with the fake face/head detection community to promote the growth of AI technology.

Paper: https://arxiv.org/pdf/2204.13100.pdf

GitHub: https://github.com/jmliu88/heser

Project: https://jmliu88.github.io/HeSer/

Reference: https://80.lv/articles/deepfake-can-now-have-different-heads/

James G. Williams