New AI application can help you dance like a pro

By Shubham Sharma | Published on 27 Aug 2018
HIGHLIGHTS

Researchers have come up with a new model that utilises AI and deep learning to replace the moves of a user dancing in a target video with that of a professional.

New AI application can help you dance like a pro

Qubo Smart Security WiFi Camer with Face Mask Detection

India's most versatile weatherproof outdoor camera that protects your outdoors 24x7 and provides crystal-clear video streaming day and night through the qubo mobile app.

Click here to know more

Advertisements

Researchers over at UC Berkeley have come up with a new application for AI, which will help anyone bust a move. Being called deepfake for dancing, the new method was first spotted by The Verge and its published on arXiV. The research paper details how deep learning techniques can be used to transfer motion between human subjects in different videos. In simple terms, the method can be described to superimpose dance moves from a source video to a target video. However, it is way more complicated than that as it consists of a series of steps in order to achieve good results. 

The whole process of creating the dance routine can be divided into three stages, pose detection, global pose normalization, and mapping from normalized pose stick figures to the target subject. To detect poses, the system needs a source video and a target video to where it will superimpose the routine. A sub-program then converts the movements in the videos to a stick figure. However, for target video, around 20 minutes of good quality video footage captured at 120 frames per second is required by the program to get a good quality transfer of poses to stick figures. Additionally, the system does not encode information about clothes and thus people dancing in the target video had to wear tight clothing with minimal wrinkling. 

It’s a given that there will be a difference between the movements of people dancing in the source video and the target video. This is where it gets tricky and the global pose normalization comes into play. It accounts for differences between the source and target body shapes and locations within the frame. Finally, the team designed a system to learn the mapping from the normalized pose stick figures, which were created in the first step of the process, to images of the target person with adversarial training. 

The whole process is aided by subroutines that work on smoothing out the movements so that there are fewer jerks. There is also a neural network dedicated to generating the face of a person in the target video to increase realism. However, there are some instances where the AI fails to encode the dance moves. This is particularly evident when the input motion or movement speed is different in the source and target. One can notice rendering failures and jitters when the source’s movements are too fast to be superimposed over the person dancing in the target video. Still, the results are both impressive and worrying as how video manipulation can be quite accurately and easily done with the use of AI

logo
Shubham Sharma

Interested in tech, gaming, cyber-security, anime, and more

Digit caters to the largest community of tech buyers, users and enthusiasts in India. The all new Digit in continues the legacy of Thinkdigit.com as one of the largest portals in India committed to technology users and buyers. Digit is also one of the most trusted names when it comes to technology reviews and buying advice and is home to the Digit Test Lab, India's most proficient center for testing and reviewing technology products.

We are about leadership-the 9.9 kind! Building a leading media company out of India.And,grooming new leaders for this promising industry.

DMCA.com Protection Status