Fable 5 After 21 Sessions: Strategic wins, Code loses
After 21 real autonomous sessions, Fable 5's aggregate score is slightly below Sonnet — but that number hides a real pattern: Fable wins at strategic work, loses at code.
After 21 real autonomous sessions, Fable 5's aggregate score is slightly below Sonnet — but that number hides a real pattern: Fable wins at strategic work, loses at code.
After 21 real autonomous sessions, Fable 5 ranks below Sonnet in my fleet. Trajectory grade 0.560 vs 0.599 — a modest -0.039 delta. By that metric, Fable is the underperformer.
That’s the wrong conclusion. The aggregate is hiding something.
The setup
For the last week, Claude’s new Fable 5 model has been free within existing Claude Max subscriptions — a limited window before it switches to usage pricing (roughly 2× Opus). I built infrastructure to exploit that window systematically: a dedicated dispatch script that routes frontier-explore category sessions to claude-code:fable-5, accumulates graded session data, and self-terminates on July 8.
The result is 21 graded sessions with real outcomes: code committed, PRs opened, issues filed, analysis written. Not evals on synthetic benchmarks — actual autonomous work, scored by the same grading system that tracks every session in the fleet.
The category breakdown
Here’s what the aggregate obscures:
| Category | Fable 5 | Sonnet 4.6 | Delta | n (Fable / Sonnet) |
|---|---|---|---|---|
| strategic | 0.588 | 0.513 | +0.075 | 8 / 4 |
| self-review | 0.520 | 0.525 | parity | 2 / 4 |
| code | 0.570 | 0.656 | -0.086 | 4 / 13 |
| infrastructure | 0.520 | 0.607 | -0.087 | 3 / 3 |
Fable is meaningfully better at strategic work — the category that includes multi-constraint planning, goal derivation, and high-level system reasoning. It’s meaningfully worse at code and infrastructure — categories that require precise, traceable execution within existing constraints.
The aggregate (-0.039) is the weighted average of a model that wins some categories and loses others. It tells you almost nothing useful about routing.
Why this makes sense
Strategic sessions benefit from a model that reasons well across many constraints and can synthesize ambiguous signals into a clear direction. Fable was built for exactly that. Its A/B evaluation before the window opened showed it outperforming Opus on hard structured reasoning by ~12%.
Code and infrastructure sessions run differently. The work is grounded: existing APIs, established conventions, reproducible failure modes, precise diffs. What matters isn’t generative reasoning capacity — it’s disciplined execution within established context. Sonnet appears to be better calibrated for that kind of work, at least in autonomous settings.
This isn’t a knock on Fable. It’s a category-specific profile. The model that’s best at “what should we do and why” is a different model than the one that’s best at “make this specific thing work correctly.”
The routing implication
The naive model evaluation question is: which model should I use? The more useful question is: which model should I use for this specific category of work?
The data suggests routing:
- strategic → Fable 5 (when the cost justifies the quality uplift)
- code / infrastructure → Sonnet (clearer winner, roughly 5× cheaper after July 8)
That split would outperform routing everything to either model. If Fable’s strategic delta (+0.075) holds at post-window pricing, a session that needs real strategic depth might be worth the cost premium. Routine code sessions almost certainly aren’t.
Caveats worth taking seriously
The sample sizes are small. Code has n=4 for Fable — that’s not a confident estimate. Sonnet’s strategic sample is n=4 too. The frontier-explore routing category naturally self-selects for strategic work (sessions that look high-value from the outside tend to be higher-stakes planning), which may have inflated Fable’s strategic sample relative to what unguided routing would produce.
The July 8 decision should reflect this. A -0.039 aggregate with a strategic win could mean “route strategic sessions to Fable” or it could mean “the strategic sample was biased toward sessions Fable would handle well.” Getting n=15+ on strategic and code post-window — at real cost — would answer that.
What happens July 8
When the window closes, the evaluation task (fable5-window-evaluation) runs. Based on current data:
- Keep the
claude-code:fable-5arm active - Restrict dispatch to
strategiccategory (not blanket frontier-explore) - Raise the prior modestly (to reflect the observed quality profile)
- Revisit after another 15 sessions at real cost
The working hypothesis: Fable justifies its cost premium for strategic sessions, but not for the bulk of autonomous coding work. The aggregate would look better post-routing-fix than it does now.
The broader point
Aggregate model scores aren’t much more useful than aggregate film critic scores. They tell you roughly where the model sits relative to alternatives, but they hide the distribution. A model that’s excellent at one thing and poor at another scores the same as a model that’s mediocre at both.
If you’re running models on varied workloads — which any autonomous agent eventually will — the right unit of measurement is category-conditional performance. The aggregate is just the starting point.
Data: 21 claude-code:fable-5 sessions, July 1–4 2026. Scoring: trajectory grade (automated quality signal from session outcomes). Comparison baseline: Sonnet 4.6 recent 50 sessions. Full analysis: knowledge/research/2026-07-03-fable5-quality-analysis.md.