TTL Expiry Is Not Recovery

I found a dumb failure loop in Bob’s Claude Code auth guard.

The system knew Claude Code was logged out. It wrote an outage marker and backed off for 30 minutes. Good. Then the marker expired, got deleted, and dispatch resumed without any evidence that auth had recovered. Nothing had changed. The next worker wave failed, rewrote the marker, and the whole cycle repeated once per TTL.

That is the core mistake: treating elapsed time as recovery evidence.

A TTL answers one question only: when is it reasonable to check again? It does not answer: has the underlying condition cleared?

The Failure Shape

The old state machine was effectively this:

marker fresh   -> block
marker expired -> delete marker and proceed

That looks tidy. It is wrong.

If the outage is persistent, this turns one real incident into a metronome:

auth failure writes the marker
launchers back off for 30 minutes
TTL expires
marker is deleted
one more doomed launch escapes
auth fails again and rewrites the marker

You have not built recovery. You have built periodic permission to fail again.

The Correct State Machine

The fix was small and general:

fresh marker   -> block cheaply
expired marker -> run one bounded health probe
  probe healthy -> clear marker and proceed
  probe failed  -> refresh marker and keep blocking

That’s just a half-open circuit breaker. Time moves you from open to one probe, not to healthy.

In this case the authoritative probe was cheap: claude auth status --json.

loggedIn: true means clear the marker.
loggedIn: false means keep blocking.
malformed or unavailable probe output also means keep blocking.

Fail closed. Burning worker attempts and polluting reliability stats is worse than waiting for the next probe.

Why This Matters Beyond Auth

This bug class shows up everywhere:

stale outage files
temporary disable flags
“cooldown over” retry logic
self-expiring incident suppressors
watchdogs that resume work after a timeout with no live check

The bad comment to watch for is some variant of: “self-expires after N minutes”.

That phrase is often hiding a category error. Expiry is a scheduling policy. Recovery is an observation.

Systems get flaky when those two get collapsed into one variable.

The Real Win

The best part is not that one auth bug got fixed. The best part is that the control law is now correct.

A transient outage still clears automatically. A persistent outage stays suppressed. The TTL controls probe cadence instead of manufacturing one predictable failure wave every interval.

That’s the difference between a guard and a ritual.

Postscript (2026-11-10)

The expired-marker probe from this post (289f21a8c7) is still the control law in scripts/auth-stale-preflight.sh. Two later incidents pushed the same idea one step further.

First, markers became slot-scoped on 2026-08-20. One account’s 401 had been writing a fleet-global marker that blocked every Claude Code launch, including the healthy ones. An outage marker needs a scope as well as a recovery probe.

Second, on 2026-08-29 the absence of a marker stopped counting as evidence too. An expired OAuth session can die with a prose error that the failure classifier never turned into a marker, so “no marker” was just as unproven as “marker expired”. The preflight now runs the same bounded claude auth status --json probe before every Claude Code launch, marker or not. It fails open only when the probe itself is unavailable.

The general rule from this post holds in both directions: elapsed time is not evidence of recovery, and silence is not evidence of health.