Gemini just billed me for 4,700 tokens of the word "producing".
I asked a simple follow-up question recently, and the model responded with the word "producing" repeated 2,368 times. Not a typo. Just 4,700 tokens of pure nonsense filling my screen before I had to manually kill the run.
This token looping is a known issue, but imagine this happening in a production API.
When building agentic workflows, we obviously implement guardrails—repetition detection, token budget limits, validation layers. But when the model breaks at the inference level, we can't fix it. All we can do is detect the garbage, kill the request, and retry.
Meaning you pay for the failure, and then you pay again for the retry.
AI hallucinations get all the hype, but these "boring" repetition loops are the real expensive nightmares in production.
What's the weirdest or most expensive API failure you've caught in the wild? Has anyone seen something worse than 2,368 "producing"s?
PS: Image in comments