Auditing content moderation: Extreme memetic videos on TikTok and a test of their automated review

The presentation reports upon a test case of content moderation by the social media platform, TikTok, of hateful conspiracy videos targeting Muslims. The videos in question depict a European future when in 2050 cities have been ‘Islamised’. Made by a diverse set of creators, the videos are portrayals of one European city after another – Amsterdam, 2050, Barcelona, 2050, Milan, 2050 and so forth – devastated after the religious and cultural shift. The paper examines how creators position these videos as content-as-usual, with platform hashtag and descriptor norms referencing memetic user culture and virality efforts. Subsequently, in the test case, the videos are reported to the platform through the content moderation reporting function as violating its hateful conspiracy policy. Only when reviewed a second time by the human moderators, after appeals are issued, are certain videos removed or demoted, demonstrating an unevenness in moderation between automated and manual review. We discuss the implications of these findings in terms of automated slippage and human-machine misalignment as well as the user reporting burden assumed in an algorithmic environment promoting personalised content. It concludes with a call for moderation auditing to test platform claims about the efficacy of especially automated moderation.