u/reasonosaur

GPT-5.5 plays Pokémon FireRed . Watch the stream here!

• This run use "almost" screenshot (vision) only harness, and it's using the Codex harness instead of regular API call. Vision only, no RAM extraction.

FAQ:

• How are we doing compared to the previous run?
  • Check out 5.5 playing Emerald here.
  • Check out 5.4 playing FireRed here.
• What is the Agent Harness? Watch the live feed, explore the harness, and browse all of the AI’s data: https://gpt-plays-pokemon.clad3815.dev

According to Anthropic: "Fable 5 is the new state-of-the-art model for tasks involving vision. It can extract precise numbers from detailed scientific figures and can perform complex vision-based tasks like rebuilding a web app’s source code from screenshots alone. It also needs less scaffolding: for example, previous Claude models struggled to play Pokémon FireRed even with harnesses that gave them additional helpful tools, but Fable 5 beat FireRed with a minimal, vision-only harness.

https://www.youtube.com/watch?v=Ty_50J84fMY

A timelapse of Claude playing Pokémon FireRed from start to finish using only raw game screenshots — with no maps, navigation aids, or extra game-state information. Earlier Claude models needed a complex helper harness to play Pokémon; Claude Fable 5 completed the game with vision alone."

If Anthropic is to be believed, Claude Fable 5 with *no harness* beat GPT-5.5 with a *strong harness* by 1 hour and 11 minutes.