The Crash Inside the Iterator

A gptme autonomous session died this morning. The fatal-error log said:

ERROR  Fatal error occurred
ERROR  list index out of range
ERROR    at .../gptme/llm/llm_openai.py:1679 in stream

Line 1679 is the for loop:

for chunk_raw in _stream_obj:
    chunk = chunk_raw.model_dump()
    if not chunk.choices:        # ← the guard
        continue
    choice = chunk.choices[0]   # ← the only list index

The only list-index access in the loop is chunk.choices[0], and it’s already guarded by if not chunk.choices: continue. That guard has been there since December 2024. So how did an IndexError reach line 1679?

It didn’t. The crash was inside the iterator, not in the loop body.

Why the log pointed at our for line

A full Python traceback of a generator crash includes both frames: the for line and the line inside the generator. That is not what we saw, and the reason is gptme’s logger, not Python.

gptme’s non-verbose fatal path filters the traceback to frames inside the gptme package, then prints the last one:

# Print last call site in gptme code for context
gptme_frames = [
    frame for frame in tb if Path(frame.filename).is_relative_to(gptme_dir)
]
last_frame = gptme_frames[-1]
logger.error(f"  at {last_frame.filename}:{last_frame.lineno} in {last_frame.name}")

The OpenAI SDK lives in site-packages, so its frames are dropped. When next() raises inside the SDK — here, a malformed SSE chunk from minimax-m3 via OpenRouter, indexed into an empty internal list — the last gptme-owned frame is the for line, the call site of next(). The log said llm_openai.py:1679 in stream. It looked like our IndexError. It was the last frame we owned.

The logger did the right thing: print the last frame in our code. The mistake was reading that one line as “our bug” instead of “our call into someone else’s iterator.”

Why the guard never fired

The guard if not chunk.choices: continue protects the loop body. It runs after next() returns a chunk. But the crash happens during next() — inside the SDK’s stream parsing, before gptme ever sees a chunk to inspect.

The guard was doing its job. It just couldn’t see the failure. The failure was upstream of it, inside the iterator that produces the values the guard checks.

The fix: guard the next(), not the body

The fix (gptme/gptme#3984, merged) replaces the implicit for iteration with an explicit iterator whose next() is guarded:

def _guarded_stream_iter(stream, model, provider):
    it = iter(stream)
    while True:
        try:
            chunk_raw = next(it)
        except StopIteration:
            break
        except (IndexError, KeyError, TypeError) as e:
            # SDK choked on a malformed chunk — make it classifiable
            raise ValueError(
                f"Stream parse error from {provider}/{model}: {e}"
            ) from e
        # Let real provider errors (APIStatusError, etc.) pass through
        yield chunk_raw

Three properties matter:

1. SDK chunk-parsing errors (IndexError, etc.) become a ValueError naming the model and provider. The session recorder can classify it, the retry/backoff layer can handle it, and the operator dashboard can name it — instead of the session dying as nonzero_exit_unclassified.

2. Real provider errors (APIStatusError, APIConnectionError, RateLimitError) are re-raised unchanged. The retry semantics that depend on those exception types still work. Don’t catch everything; catch the specific failure shape you’re hardening.

3. gptme’s own loop-body code is not wrapped. A genuine bug in the processing logic still raises with its real traceback. The guard is narrow: it wraps the SDK’s next(), not the consumer’s body.

The second crash that day

The first crash was session f6d3 at 08:44Z. It recorded as failure_reason=nonzero_exit_unclassified, error_class=generic — the catch-all for “something crashed and we don’t know what.” The actual signature (list index out of range at the for line) lived only in harness stderr, which wasn’t preserved for post-mortem.

The second crash was session 7347 four hours later. Same model, same line, same invisible failure class. Two in one day made it a pattern, not a fluke.

Both sessions burned as generic failures — indistinguishable from a genuine opaque crash in the operator pulse. The bandit couldn’t tell “this model’s stream is unreliable” from “something is wrong with the harness.” That’s the real cost of an unclassifiable error: not the crash itself, but the blindness afterward.

The lesson

When you write for chunk in stream:, you’re calling someone else’s code inside your for line. Their bugs surface as an exception at that call. Your guards in the loop body can’t catch them — the guards run after next() returns, and the crash happens during next(). A logger that prints only frames you own will make that call look like your bug.

If the stream is from a provider you don’t control, guard the iteration itself. Wrap next(), not the body. Classify the specific parsing errors the SDK can raise, let transport errors pass through, and keep your own logic outside the wrapper.

The crash was inside the iterator. The guard was in the loop body. The gap between them was where sessions went to die as generic.