The multimodal model, called OmniHuman-1, can create immersive videos of characters speaking, singing, and moving with quality “superior to current video generation methods,” according to ByteDance’s development team.

AI technology that creates images, videos and sounds that resemble real people is also known as “deepfake” - a technology that is increasingly used in fraud as well as entertainment.

ByteDance is one of the hottest AI companies in China. Its Doubao app is the most popular among mainland users.

Although OmniHuman-1 has not been released to the public yet, sample videos have quickly gone viral.

One standout demo is a 23-second video of Albert Einstein giving a speech. TechCrunch describes the app’s work as “shockingly awesome” and “the most realistic deepfake videos to date.”

OmniHuman-1 only needs a single image as reference data along with audio data such as speech or vocals to create a video of any length, the developer said.

The output video frame rate can be adjusted, as well as the “body proportions” of the character in it.

d6a434e5a4dc974582b09c05b3646092afcf9490.jpeg
ByteDance is one of the most prominent AI companies in China. Photo: TechCrunch

Not only that, the AI ​​model, which was trained on 19,000 hours of video content from non-public sources, is also capable of editing existing videos, even changing human hand and foot gestures with a convincing level.

However, ByteDance also admitted that OmniHuman-1 is not perfect, as it still has difficulty with certain poses, and that “low-quality reference images” will not produce the best videos.

ByteDance's new AI model shows China's progress despite Washington's efforts to limit tech exports.

Concerns

Last year, political deepfakes went viral. In Moldova, deepfake videos simulated the country's president Maia Sandu giving a resignation speech.

And in South Africa, a deepfake of rapper Eminem supporting a South African opposition party went viral ahead of the country's elections.

Deepfakes are also increasingly being used to commit financial crimes. Consumers have been duped by deepfakes of celebrities recommending investments, offering fake investment opportunities, while companies have lost millions of dollars to impersonators impersonating senior executives.

According to Deloitte, AI-generated content contributed to more than $12 billion in fraud losses in 2023 and could reach $40 billion in the US by 2027.

Last February, hundreds of people in the AI ​​community signed a letter calling for strict regulation of deepfakes. In the absence of federal deepfake criminalization in the US, more than a dozen states have enacted laws against AI forgery.

However, detecting deepfakes is not easy. Although some social networks and search engines have taken measures to limit their spread, the amount of deepfake content online is still growing at an alarming rate.

In a May 2024 survey from identity verification company Jumio, 60% of respondents said they had encountered a deepfake in the past year; 72% of respondents said they were worried about being fooled by deepfakes on a daily basis, while a majority supported passing laws to address the proliferation of AI-generated fake videos.

Google 'greenlights' use of AI to develop weapons and surveillance technology Google has abandoned its "promise" not to design and deploy AI tools for use in weapons and surveillance technology.