In a slickly produced TikTok video, former President Barack Obama, or a voice eerily like his, can be heard fending off an explosive new conspiracy theory about the sudden death of his former chef.
“While I cannot understand the basis of the accusations leveled against me,” the voice says, “I urge everyone to remember the importance of unity, understanding, and not rushing to judgment.”
In fact, the voice did not belong to the former president. It was a convincing fake, generated by artificial intelligence using sophisticated new tools that can clone real voices to create artificial intelligence puppets with a few mouse clicks.
The technology used to create ai voices has gained traction and wide recognition since companies like oncelabs launched a series of new tools late last year. Since then, audio deepfakes have quickly become a new weapon in the online misinformation battlefield, threatening to accelerate political misinformation ahead of the 2024 election by giving creators a way to put their theories to rest. conspiracy in the mouths of celebrities, news anchors and politicians.
The fake audio adds to ai-generated threats from deepfake videos, human-like writing from ChatGPT, and images from services like Midjourney.
Disinformation advocates have noted that the number of videos containing ai voices has increased as content producers and misinformation peddlers adopt these novel tools. Social platforms like TikTok are struggling to flag and label such content.
The video that sounded like Mr. Obama was discovered by NewsGuard, a company that monitors online misinformation. The video was posted by one of 17 TikTok accounts making unfounded claims with fake audio that NewsGuard identified, according to ai-voice-technology-creates-conspiracy-videos-on-tiktok/” title=”” rel=”noopener noreferrer” target=”_blank”>a report the group published in September. The accounts primarily posted videos about celebrity rumors using narration from an ai voice, but also promoted the baseless claim that Obama is gay and the conspiracy theory that Oprah Winfrey is involved in the slave trade. The channels had collectively received hundreds of millions of views and comments suggesting some viewers believed the claims.
While the channels had no obvious political agenda, NewsGuard said, the use of ai voices to share mostly salacious gossip and rumors offered a roadmap for bad actors who wanted to manipulate public opinion and share falsehoods with massive online audiences.
“It’s a way for these accounts to gain a foothold, gain followers that can attract engagement from a broad audience,” said Jack Brewster, business editor at NewsGuard. “Once they have the credibility of having a large following, they can dive into more conspiratorial content.”
TikTok requires that labels revealing realistic ai-generated content be fake, but they did not appear in the videos flagged by NewsGuard. TikTok said it had removed or de-recommended several of the accounts and videos for violating policies on impersonating news organizations and spreading harmful misinformation. It also removed the video using the ai-generated voice that mimicked Obama’s for violating TikTok’s synthetic media policy as it contained highly realistic content not labeled as doctored or fake.
“TikTok is the first platform to provide a tool for creators to tag ai-generated content and an inaugural member of a new industry best practice code that promotes the responsible use of synthetic media,” said Jamie Favazza, spokesperson for TikTok, referring to a framework recently introduced by the nonprofit Association on ai
Although the NewsGuard report focused on TikTok, which has more and more become a news source, similar content was found spreading on YouTube, Instagram and Facebook.
Platforms like TikTok allow ai-generated content from public figures, including news anchors, as long as they do not spread misinformation. Parody videos featuring ai-generated conversations between politicians, celebrities or business leaders (some dead) have spread widely since the tools became popular. Manipulated audio adds a new layer to misleading videos on platforms that have already featured fake versions of Tom Cruise, Elon Musk and tech/tech-news/deepfake-scams-arrived-fake-videos-spread-facebook-tiktok-youtube-rcna101415″ title=”” rel=”noopener noreferrer” target=”_blank”>news anchors like Gayle King and Norah O’Donnell. TikTok and other platforms have been dealing lately with a deluge of misleading ads featuring fakes of celebrities like Mr. Cruise and YouTube star Mr. Cruise. tech/mrbeast-ai-tiktok-ad-deepfake-rcna118596″ title=”” rel=”noopener noreferrer” target=”_blank”>Mrbeast.
The power of these technologies could profoundly influence viewers. “We know that audio and video are perhaps stickier in our memory than text,” said Claire Leibowicz, director of ai and media integrity at the Partnership on ai, which has worked with technology and media companies on a set of Recommendations for creating, sharing and distributing ai-generated content.
TikTok said last month it was introducing a label that users could select to show whether their videos used ai. In April, the application began require users to disclose manipulated media showing realistic scenes and ban deepfakes of young people and private figures. David G. Rand, a professor of management sciences at the Massachusetts Institute of technology who TikTok consulted for advice on how to word the new labels, said the labels were of limited usefulness when it came to misinformation because “people “Those who are trying to be deceptive are not going to put the label on their stuff.”
TikTok also said last month that it was testing automated tools to detect and label ai-generated media, which Rand said would be more useful, at least in the short term.
YouTube bans the use of ai in political ads and requires other advertisers to label their ads when ai is used. Meta, which owns Facebook, added a label to its suite of fact-checking tools in 2020 that describes whether a video is “altered.” YX, formerly known as Twitter, requires misleading content be “significantly and deceptively altered, manipulated, or fabricated” to violate their policies. The company did not respond to requests for comment.
Obama’s ai voice was created using tools from oncelabs, a company that burst onto the international stage late last year with its free-to-use ai text-to-speech tool capable of producing realistic audio in seconds. The tool also allowed users to upload recordings of someone’s voice and produce a digital copy.
After the tool’s launch, users of 4chan, the right-wing message board, ai-voice-firm-4chan-celebrity-voices-emma-watson-joe-rogan-elevenlabs?utm_source=reddit.com” title=”” rel=”noopener noreferrer” target=”_blank”>organized to create a fake version of actress Emma Watson reading an anti-Semitic speech.
ElevenLabs, a company with 27 employees based in New York City, responded to the misuse by limiting the voice cloning feature to paid users. The company also launched ai-speech-classifier” title=”” rel=”noopener noreferrer” target=”_blank”>an ai detection tool that is capable of identifying the ai content produced by its services.
“More than 99 percent of the users on our platform are creating interesting, innovative and useful content,” an ElevenLabs representative said in an emailed statement, “but we recognize that there are cases of misuse and we have been developing and releasing safeguards continually. to stop them.”
In tests conducted by The New York Times, ElevenLabs’ detector successfully identified audio from TikTok accounts as ai-generated. But the tool crashed when music was added to the clip or when the audio became distorted, suggesting that misinformation peddlers could easily evade detection.
ai companies and academics have explored other methods for identifying fake audio, with mixed results. Some companies explored the possibility of adding an invisible watermark to ai audio by incorporating signals that it was generated by ai. Others have pressured ai companies to limit the voices that can be cloned, potentially banning replicas of politicians like Obama, a practice that already exists with some imaging tools like Dall-E, which refuses to generate some political images.
Ms Leibowicz, of the Partnership on ai, said synthetic audio was especially difficult for listeners to detect compared to visual disturbances.
“If we were a podcast, would we need a tag every five seconds?” Mrs. Leibowicz said. “How can you have a signal in a long piece of audio that is consistent?”
Even if platforms adopt ai detectors, the technology must constantly improve to keep up with advances in ai generation.
TikTok said it was developing new detection methods internally and exploring options for external partnerships.
“Big tech companies, multi-billion dollar or even trillion dollar companies, can’t they do it? That surprises me a little bit,” said Hafiz Malik, a professor at the University of Michigan-Dearborn who is developing ai audio detectors. “If they intentionally don’t want to do it? That’s understandable. But can’t they do it? I do not accept it”.
Audio produced by Adriana Hurst.
