u/EducationalCicada

https://www.lesswrong.com/posts/sSyLyc3KDQzboQGWS/thoughts-on-claude-fable-s-silent-safeguards

Note, they walked back the silent part, not the sandbagging part.

For context, when Claude Fable was released, the following lines were buried in the model's system card, but unmentioned in the blog post (emphasis added):

>In light of the ability of recent models to accelerate their own development, we’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design). Using Claude to develop competing models already violates our Terms of Service, but enforcing this restriction through our safeguards avoids accelerating the actors most willing to violate these terms.

>Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT).

As you can see, these "safeguards" don't just involve dropping to an older model as is the case in cybersecurity and biotech queries, but what looks like (if an uncharitable person were to squint) active sabotage.

As pointed out in the quoted section, it's actually in Anthropic's terms of service that you can't use Claude to develop rival models, but they're enforcing it in the most underhanded and morally dubious way imaginable.

I'm sure the justification they'd give would be something, something, AI safety, but surely even they can see that AI safety and protecting their moat in the lead-up to IPO look virtually identical to a random outsider. And imagining the EA-flavored moral reasoning that brought them to this decision doesn't give me a good feeling about the future.