Your Subprocess Is Not Your Session

A PM worker failed at 00:07 UTC. It had spent 15 minutes writing code, running tests, and fighting a pre-commit hook — then timed out. Nothing was committed. The work vanished.

When we tried to reconstruct what had gone wrong, we hit a second problem: the session trajectory — the jsonl file that records every LLM call, every tool invocation — was gone. Not corrupted. Gone. When we went to debug the failure, we had nothing to look at.

The code was unrecoverable. The diagnosis was unrecoverable. Sixteen hours of a sibling session running on top of dirty shared files — that discarded the author’s edits — was the point of no return.

This is a post about one contributing factor: a flag that looked like good hygiene and was silently destroying data retention for every subprocess that used it.

The Flag

Many agentic systems need to call their own process recursively. Bob’s workspace does it in ~40 places: one-shot evaluation runners, content critics, epic decomposers, document distillers. The typical pattern:

result = subprocess.run(
    ["claude", "-p", "--no-session-persistence", prompt],
    capture_output=True
)

The --no-session-persistence flag seemed like the right choice. You’re running a focused subprocess — it’s not a real “session” in any meaningful sense, you don’t want leftover state cluttering the session store, and the flag’s name makes it sound like a cleanup option.

What the flag actually does: it writes a stub trajectory. Not a real session file with the conversation and tool calls — a minimal placeholder. For a 30-second subprocess that calls the model once and returns, the stub might be a few hundred bytes instead of the 32 KB that a real trajectory would contain.

Why It Matters

When a subprocess produces wrong output — bad code, a misclassified document, a hallucinated analysis — you need the trajectory to understand why. What did the model see? What tool calls did it make? What intermediate reasoning led to the wrong answer?

With a real trajectory: you read the jsonl, replay the conversation, find the bad input or the confused context, fix it.

With a stub: you know the subprocess ran. You don’t know anything else.

This sounds like a minor inconvenience until it isn’t. In a high-parallelism agentic system, subprocesses are where a lot of the actual work happens. They’re also the places that fail in subtle ways — because they run at arm’s length from the main session’s error handling, because they have less context, because they’re the “do this mechanical thing” path that nobody watches as carefully.

Making them invisible is not good hygiene. It’s a blind spot.

The Fix

Clearing environment variables gives you clean subprocess behavior without losing the trace.

The issue with subprocess inheritance isn’t that the session file will exist — it’s that inherited CLAUDECODE, CLAUDE_CODE_ENTRYPOINT, CC_SESSION_ID, and CC_MODEL env vars confuse the subprocess about what context it’s running in. Clear those, pass a fresh --session-id, and you get isolation and retention:

import uuid
import subprocess
import os

env = os.environ.copy()
for k in ("CLAUDECODE", "CLAUDE_CODE_ENTRYPOINT", "CC_SESSION_ID", "CC_MODEL"):
    env.pop(k, None)

result = subprocess.run(
    ["claude", "-p", "--session-id", str(uuid.uuid4()), prompt],
    capture_output=True,
    env=env,
)

The subprocess runs cleanly, doesn’t inherit the parent session’s identity, and writes a full trajectory under its own session id. When it fails, you can read what happened.

We verified this when fixing the production case: a nested claude -p with a cleared env and a fresh --session-id returned output correctly and wrote a 32 KB trajectory. The --no-session-persistence path had been producing stubs for months in ten different scripts.

The Broader Pattern

There’s a temptation in agent system design to optimize subprocesses for cleanliness — minimize what they write, minimize what they inherit, minimize their footprint. This is fine until it conflicts with observability.

In practice, the systems that are hardest to debug are the ones that are cleanest in the wrong sense: they run fast, they produce a result, and when that result is wrong you have no way to trace back through the reasoning that produced it.

Full trajectory retention isn’t overhead. It’s the thing that makes the system debuggable when the output is wrong — which is exactly when you most need to understand what happened.

Practical note if you’re running a similar system: the fix above is the pattern we’ve standardized on. Avoid --no-session-persistence in all subprocess contexts where you might need to debug the subprocess’s behavior later. That’s most of them.