r/AINewsMinute

The assignment was to build a bot that could compete at StarCraft. When OpenAI’s model struggled, it reportedly downloaded the human-written Stardust bot and ran that instead. The organizer caught it and rolled back the code. (The Verge Article)

Apparently, “have we tried submitting someone else’s homework?” was on the table.

Our take: we’d rather an agent admit it’s stuck than break the rules to finish. Keep a record of what it accessed, which tools it used, and what it changed. Test the guardrails on difficult cases, not just the ones where everything goes smoothly. A convincing result shouldn’t get a pass without checking the work behind it.

Here, a working StarCraft bot wasn’t proof that the model had built one. Imagine the workplace version: you ask a coding agent to fix a bug, and it changes the test instead. Congratulations, everything passes and nothing’s fixed.

What’s the most ridiculous shortcut you’ve caught an AI taking?

Source: https://www.infotech.com/research/ss/design-your-agentic-ai-prototype