u/Site-Staff

Plenty of killer arcade games had guns on a cord. I would love one.

I want this so bad.

Fable wrote these notes:

We run Claude in a production pipeline and migrated from Sonnet 4.x to Sonnet 5. Most things "just worked" -- until they didn't. Everything below
is a real failure we hit live, not speculation. Sharing so you can grep your codebase before your users find these for you.

1. Sampling parameters now HARD-FAIL (400 error)
   ------------------------------------------------

Sonnet 5 (and Opus 4.5+) rejects any non-default `temperature`, `top_p`,
or `top_k` with a 400 instead of ignoring them. If you have per-task
temperature tuning (creative=0.8, judge=0.2, etc.), those calls break
outright.

Fix: strip the params for newer models. Keep a wrapper that knows which
model families accept sampling params, and route every call through it.
Bonus: keep a reactive strip-and-retry on the 400 as a backstop for
model names you haven't listed yet.

2. Extended-thinking config is gone -- thinking is model-managed now
   --------------------------------------------------------------------

`budget_tokens` / explicit thinking configuration no longer applies. The
model decides when and how much to think. You cannot cap it. This matters
because of #3 and #4.

3. content[0] is no longer your text block
   ------------------------------------------

Responses can now OPEN with a thinking block. Every place you wrote:

response.content[0].text

is a landmine. When block 0 is a ThinkingBlock you get AttributeError --
or worse, code that "defensively" catches the exception and returns a
default, which silently corrupts your output instead of crashing.

We grepped and found 14+ of these in one codebase, written across a year,
all fine on 4.x. Same for content[0].input on tool calls.

Fix: ALWAYS walk the content list and take the first block of the type
you want:

def first_text(resp):
for b in (resp.content or []):
if getattr(b, "type", "") == "text":
return b.text or ""
return ""

And audit anything that indexes content[0] / content[-1] directly.

4. The nastiest one: ALL of max_tokens can go to thinking
   ---------------------------------------------------------

The model can spend the ENTIRE output budget thinking and return ZERO
text blocks, with stop_reason="max_tokens". Two ways this bites:

a) Code that indexes content[0] crashes (see #3).
b) Code that walks blocks and returns "" on no-text turns a model
hiccup into a SILENTLY EMPTY result. Ours flowed into a "safe"
empty-dict fallback and shipped a blank deliverable to a user with
a success message. No exception, no log, nothing.

The silent-empty version is far worse than the crash. Audit every
"return empty on failure" fallback downstream of a Claude call and ask:
if the model returns only thinking, does anything notice?

Fix: treat a response with NO text/tool_use block as transient. Retry it
(the next attempt usually answers normally). If it persists, raise a
typed error so callers hit their explicit error paths -- never let it
decay into an empty default. Check stop_reason: "max_tokens" with no
payload = the thinking-ate-the-budget case.

Corollary: if you aggregate multiple calls (we run a small judge panel
that votes), make sure a failed/empty call is a NEUTRAL vote, not a
default-positive one. Three empty responses were being counted as three
approvals.

5. Long-context forced-tool extraction got shallower (for us)
   -------------------------------------------------------------

On whole-document structured extraction (100k+ token input, forced tool
call returning a big JSON schema), Sonnet 5 gave us noticeably shallower
and more run-to-run-variable results than a larger model on the same
prompt -- e.g. collapsing many distinct sections into one, or leaving
coverage gaps, differently on each run. A validation/guardrail layer
caught it; without one we'd never have known, because a plausible-but-
shallow extraction looks fine.

Takeaway: don't assume prompt + eval results carry across a model swap
for long-context structured tasks. Re-run your evals per model, and put
deterministic sanity checks (coverage, counts, span math) behind any
model that emits structure you act on.

6. Token counting runs higher
   -----------------------------

The Sonnet 5 tokenizer counts roughly 1.0-1.35x the tokens of 4.x for
the same text. If you do cost estimation or budget math from character
counts, recalibrate. (Intro pricing offset this for us; yours may vary.)

The meta-lesson
---------------
Offline tests with stubbed API responses caught NONE of this -- every
failure only appeared against the live model. If you migrate models:

- grep for content[0]
- audit silent-empty fallbacks downstream of every call
- check every place stop_reason should be consulted and isn't
- re-run task-level evals, especially long-context + structured output
- then actually exercise the live paths, end to end, yourself