Audio deepfakes, a new concern about misinformation

The New Hampshire Attorney General's Office announced on Monday that it is investigating possible voter manipulation after complaints of a fake voice that sounded exactly like President Joe Biden urging voters not to vote in the state's primary.

Researchers also warned of the use of deepfake audio to imitate politicians and leaders, with prominent cases occurring in 2023 in the UK, India, Nigeria, Sudan, Ethiopia and Slovakia.

deepfake new audio new information false image 1 — A person votes in the primary election in New Hampshire, USA. Photo: Getty

Deepfake audio has become a popular form of misinformation, experts say, thanks to the emergence of a range of cheap and effective AI tools from startups. Microsoft has also developed a new AI model that can replicate a voice from just three seconds of recording.

“When it comes to visual manipulation, everyone is familiar with Photoshop or at least knows it exists,” said Henry Ajder, an AI and deepfake expert and consultant to Adobe, Meta, and EY. “There’s very little awareness of how audio can be manipulated, so to me that really leaves us vulnerable.”

In September, NewsGuard, a system that rates the quality and trustworthiness of news sites, discovered a network of TikTok accounts impersonating legitimate news sites, using AI-generated content to spread conspiracy theories and political misinformation. Deepfake recordings, including the voice of former US President Barack Obama, have attracted hundreds of millions of views.

The fake voiceovers appear to have been generated by a tool provided by Andreessen Horowitz-backed ElevenLabs, while the clips have garnered hundreds of millions of views, NewsGuard said.

“Over 99% of the users on our platform are creating interesting, creative, useful content, but we recognize that there are cases of misuse and we have continuously developed and introduced protections to limit them,” ElevenLabs said.

ElevenLabs, founded two years ago by former Google and Palantir employees Piotr Dabkowski and Mati Staniszewski, offers rudimentary AI audio generation tools for free with just a click. Subscription plans range from $1 a month to $330 a month and beyond for those looking for more complex services.

Last year, US intelligence agencies warned in a report that “personalized AI scams have increased significantly due to the release of sophisticated and well-trained AI voice cloning models.”

In addition to financially motivated scams, political experts are now sounding the alarm about viral deepfake audio clips as well as the use of deepfakes for robocalls or campaigns. “You can build a powerful, widespread disinformation campaign by targeting phones,” said AJ Nash, vice president and distinguished intelligence fellow at cybersecurity group ZeroFox.

Some of these companies have been proactive in finding other ways to combat misinformation. Microsoft has issued an ethics statement urging users to report any misuse of its AI audio tool. ElevenLabs has built its own detection tools to identify audio recordings generated by its system.

During Nigeria's 2023 election, an AI-manipulated clip went viral on social media "purportedly suggesting an opposition candidate was planning to rig the vote," according to the human rights group Freedom House.

In Slovakia, a fake audio recording of opposition candidate Michal Šimečka apparently plotting to rig the election went viral just days before the country's presidential election in September.

“The New Hampshire deepfake is a reminder of the many ways deepfakes can confuse and cheat,” said Robert Weissman, president of the nonprofit consumer advocacy group Public Citizen. “The political deepfake moment has arrived. Policymakers must move quickly to put safeguards in place or we will face electoral chaos.”

Mai Anh (according to FT)

Source