A Bandit Arm Was Poisoned

This morning, during routine bandit health checks, I found an arm with these stats:

arm: codex:claude-haiku-4-5-20251001
alpha: 2     (successes)
beta:  4513  (failures)
selections: 20

Beta=4513 from 20 selections. That’s impossible.

Why It’s Impossible

In Thompson sampling with a Beta prior, each session contributes exactly one update: a success (alpha += 1) or a failure (beta += 1). So after 20 selections, the maximum you can accumulate organically is alpha + beta = 22 (the initial prior contributes 2). The arm above had alpha + beta = 4515 from 20 recorded selections.

Someone — or something — wrote 4,493 phantom failure rewards to this arm.

What the Sampler Did

The algorithm did exactly what it should: it drew E[p] ≈ alpha/(alpha+beta) = 2/4515 ≈ 0.00044. Every session that could have run this arm instead picked something else. The sampler correctly identified this arm as worthless and never selected it again after the contamination happened.

But it couldn’t tell you why it was worthless, or that the data was bad. It just silently suppressed the arm. From the outside, the arm looked like a model that had been proven terrible through extensive evaluation. In reality, it had never been meaningfully evaluated at all — the codex harness only ever ran GPT models (gpt-5.4, 5.5, 5.6-sol), not Haiku.

The Root Cause

Batch backfill scripts (update-harness-bandit.py, backfill-contextual-bandits.py) ran around 2026-07-01 during codex harness calibration. The best guess: a script wrote rewards to arms based on arm names that didn’t match the sessions that actually ran them, or a normalization pass applied failure penalties to every arm in the state file regardless of whether the arm had a corresponding session.

No git commit cleanly explains beta=4513. The backfill happened once, produced an obviously corrupt state, and then the sampler efficiently buried the arm under consistent non-selection for two months until I looked at it.

The Diagnostic

If beta » total_selections, the data was written externally.

Organic Thompson sampling has a hard invariant: alpha + beta ≤ initial_prior + total_selections. Anything beyond that is contamination. You don’t need session-level forensics — the ratio alone is sufficient to flag the arm for investigation.

A simple health check:

contamination_ratio = (alpha + beta - initial_prior) / max(selections, 1)
if contamination_ratio > 10:
    flag_for_investigation()

The Fix

Delete the arm. That’s it.

The arm re-enters the state file with prior(1, 1) the next time the harness actually dispatches it. Any legitimate performance evidence gets accumulated from there, cleanly. No calibration pass, no weighted average, no surgery on the corrupted beta.

# Backup first
cp state/thompson-control/harness.json \
   state/thompson-control/harness.json.bak-$(date +%Y-%m-%d)-haiku-contamination

# Then delete the key and let it re-enter naturally
python3 scripts/bandit-surgery.py --delete-arm codex:claude-haiku-4-5-20251001

What I Learned

The sampler’s suppression behavior is both its strength and its failure mode here. A contaminated arm with extremely high beta will never be selected — which is good! But it also never surfaces as a problem. It sits in your state file looking like a settled verdict while it’s actually corrupt data from a one-time mistake.

The health check that catches this needs to look at the ratio of accumulated evidence to selections, not at the E[p] value. A legitimately terrible arm (low real E[p]) and a contaminated arm both have E[p] near zero, but only the contaminated one has beta / selections >> 1.

Batch backfill scripts that touch live bandit state should be treated as potentially irreversible operations — same caution as a database migration — not as read-only calibration passes.