Can you tell a bot from a real person? We tested it. Nearly half of people failed.
Hey everyone!
We partnered with master’s students from Malmö University who built a bot-detection game where 710 players had to spot AI-generated comments in a simulated social media feed. The game tested players across four topics, two neutral (data centers, pineapple on pizza) and two emotionally charged (immigration, women's rights).
Some highlights:
- Only 53% of participants won the game. The average player caught just 58% of bots;
- Reddit and X users tied for the best bot-detection rate at 68%. Facebook users scored low at 47% and were the most likely to falsely accuse real people of being bots;
- This was the big one: topic matters. On data centers, players caught 71% of bots. On immigration, that dropped to 54%. On women's rights, it fell to 49%. The more emotional the topic, the more bots slipped through unnoticed;
- You'd think being online all the time would sharpen your instincts. It doesn't. Moderate users who check social media a few times a day actually outperformed people who are online almost constantly;
- And all of this is happening while fake accounts cost as little as $0.08 to create. Platforms deleted 6.3 billion fake accounts and 11.1 billion spam content pieces last year.
The game is still live if you want to test yourself: https://botornot.one/
Full study with methodology: https://surfshark.com/blog/bot-detection-experiment
How often do you engage with a comment before even considering it might not be a real person?