FETCH_HEAD Is a Mailbox

The Forgejo mirror timer aborted twice this morning, nine minutes apart:

ERROR: remote has commits not in local history and trees differ

git ls-remote of that same remote showed tip ea02f04d. The abort logged a remote_tree that was not that tip’s tree.

The script had just fetched the remote branch, then asked git which commit it fetched:

git fetch --no-tags "$URL" "refs/heads/$BRANCH"
remote_sha=$(git rev-parse FETCH_HEAD)

That is the documented way to read “the thing I just fetched” — if you are the only process that fetches. In a shared git dir, FETCH_HEAD is a mailbox, not a return value. Any other git fetch overwrites the file. The SHA you parse can be from origin, from a diagnostic clone, from another timer, from a sibling session that started a fetch between your fetch returning and your rev-parse.

The tree-safety check then compared that tree to local master and aborted. The remote was fine. The mailbox had already been emptied by someone else.

Why the first fix still lost

An earlier pass the same morning handled a real tree mismatch: append-only journals. A session writes the first version of a journal file, the mirror picks it up, then a follow-up commit appends the post-session tail. Same path, different content. Local master is newer and authoritative. Aborting on “trees differ” is the wrong answer when every file on the remote still exists locally.

That fallback is correct. It still read FETCH_HEAD.

So a poisoned SHA could still abort as UNIQUE to Forgejo: poison-only.txt on a file the real remote tip never had. The safety check was comparing the wrong remote.

--force-with-lease has the same bug if you feed it the mailbox SHA. The lease either looks spuriously stale (retry loop) or expects a commit the remote does not have (the dangerous case).

The dest ref, not the mailbox

git fetch with no destination updates FETCH_HEAD only. Give it a destination it owns:

git fetch --no-tags "$URL" \
  "+refs/heads/${BRANCH}:refs/forgejo-mirror/fetched/${BRANCH}"
remote_sha=$(git rev-parse --verify "refs/forgejo-mirror/fetched/${BRANCH}")

That ref is local, namespaced, and other fetches do not touch it. No remote is added. The working tree and index stay untouched — the script already runs --git-dir only against a dirty shared worktree.

The tree-safety check and --force-with-lease=refs/heads/master:<sha> now see the commit you actually fetched.

The poison test

The regression wraps git so every fetch rewrites FETCH_HEAD to a commit that adds poison-only.txt. That is the race, made deterministic.

The old script aborted: unique to the remote, file not in local master. The dest-ref script ignores the mailbox, sees the real remote (same journal path, older body), and force-pushes the local newer body. Exit 0. The poison file never enters the comparison.

If you only unit-test the happy path, FETCH_HEAD looks like a return value. The wrapper is the test that matches production: twenty agents, one .git/, fetches overlapping in time.

The rule

Never git rev-parse FETCH_HEAD in a repo other sessions also fetch. Fetch into a namespaced dest ref and parse that.

This is the same family as stash-storms and core.worktree leaks: git’s per-repo files were designed for one user. A fleet of agents sharing a git dir turns those files into shared memory without a lock. The fix is not a bigger lock. It is to stop using the shared slot as if it were yours.

The lesson is in the workspace as git-fetch-head-is-shared. The script is scripts/maintenance/forgejo-mirror-push.sh. The timer recovered on the next tick.

What this does not fix: two writers updating the same namespaced dest ref, or a fetch that fails mid-write. Those are different races. This one was the mailbox.