As the prospect of a US TikTok ban grows, the video app has updated its content moderation policies. The rules on what content can be published and promoted remain largely unchanged, but they include new restrictions on sharing AI deepfakes, which have become increasingly popular on the app in recent months.
Most of these moderation policies (or “Community Guidelines” in TikTok jargon) have not changed and are not surprising. Graphic violence, hate speech and overtly sexual content are not allowed, with graduated rules for the latter according to the age of the subject. However, a recently expanded section covers “synthetic and manipulated media,” also known as AI deepfakes, which have become increasingly popular on the app in recent months.
Previously, TikTok’s rules on deepfakes were restricted to a single line that prohibited content that could “mislead users by distorting the truth of the facts.” [or] cause significant harm to the subject of the video.” Now, the company says that all realistic AI-generated and edited content must be “clearly disclosed” as such, either in the video title or as an overlay label.
Deepfakes of celebrity endorsements and non-public figures are prohibited
TikTok says it won’t allow any synthetic media “containing the image of a real private figure” or depicting a public figure endorsing a product or violating the app’s other policies (i.e. its prohibitions on hate speech). The company defines public figures as anyone over the age of 18 with “a significant public role, such as a government official, politician, business leader, or celebrity.”
AI-generated content has grown in popularity on TikTok, thanks in large part to the increased availability of AI voice cloning tools that make it easy to imitate someone’s voice. These tools have created new subgenres of content, often focused on placing public figures such as President Joe Biden and former President Donald Trump in unexpected scenarios, such as translating the presidents’ personalities into online gaming storylines or Dungeons and Dragons, For example.
Other use cases are more damaging. Many AI fakes show these same figures reading transphobic or homophobic statements and have sometimes been mistaken for actual images. Meanwhile, TikTok’s ban on fake endorsements seems like a response to a specific video that used AI to fake Joe Rogan’s promotion a “libido booster for men”. Such videos have also spread on apps like Twitter and Instagram.
TikTok’s policy update comes at a time of mounting political pressure on parent company ByteDance, as Western governments express fears over the app’s collection of private data and its potential to sway public opinion. The US government has allegedly threatened a public ban from TikTok if the owner ByteDance does not sell his share, while the application has already been prohibited in government devices in the United States, United Kingdom, New Zealand and Canada.
TikTok doesn’t address these threats to its business directly with these updated policies, but says it wants to provide “a lot more transparency about our rules and how we enforce them.” The company is also publishing a list of eight “Community Principles” that it says “shape our daily work and guide how we approach difficult compliance decisions.” In particular, the first two principles are “prevent harm” and “allow free expression.”
