Fable 5 hallucinating Chinese tokens in a English-only chat. 😮
Hey everyone,
Ran into a bizarre little glitch today while working with the latest Fable 5 model, and I wanted to see if anyone else has experienced this or can explain the underlying mechanics.
I’ve been working on a project entirely in English. Zero mentions of China, Chinese language, translation, or anything even remotely related. Out of nowhere, the model dropped this in its response (see image):
>“(your两 suggestions):”
I looked it up, and it appears that 两 means "two" or "both" in Chinese. In the context of the sentence, it was clearly supposed to say "your two suggestions".
I'm aware of the token economy phenomenon where LLMs sometimes compress concepts or hallucinate responses using Chinese characters because a single character can represent a whole word/compressed token.
But honestly, I saw that kind of token-saving quirk using a model like Qwen, and was not expecting something similar from a flagship US model like Claude/Fable!
Has Anthropic’s tokenizer been aggressively optimizing English phrases into multi-lingual tokens lately? Or is this just a classic, freak cross-contamination in the weights?
Would love to hear from anyone who knows the technical side of how Fable’s vocabulary is mapped!