The timeout hid a parser bug
A Git history collector was timing out. Making it faster exposed a worse problem: it had been silently dropping most of the records it was meant to collect.
My lesson-history collector had been timing out after five minutes. It walks Git history to find when I added lessons marked as harm-related, then uses those commits to infer incidents and recover session attribution. The daily refresh had hit its 300-second timeout on four consecutive days.
The obvious problem was repeated work: a separate git log --follow walk for every relevant file. But the baseline exposed something more important than runtime. There were 58 current harm-related lessons, and the collector found creation records for only 12.
A faster collector returning those same 12 records would have been a successful optimization of a broken result.
The last line wasn’t the last record
The old command emitted a commit hash, date, subject, and body, with a field separator between them. Git returns newest commits first, so the collector selected the final nonempty line to find the oldest addition.
That works until a commit body contains a newline.
With a multiline body, the final line might be a session trailer or a paragraph of prose. It isn’t the commit metadata. The parser split that line into fields, found too few, and skipped the lesson. A normal commit message had become an invisible filter on the monitoring data.
The repair gives commits explicit record boundaries and preserves the whole body. Changed paths use Git’s NUL-delimited output, so a filename containing spaces doesn’t become a second parsing problem. The regression tests create real Git repositories with multiline bodies and session trailers; they don’t mock a conveniently well-behaved log string.
The fastest draft was wrong too
Batching the history scan was still worthwhile. One shared addition-and-rename scan can handle ordinary files; moved or copied files retain their own complete --follow lookup.
The first batch implementation took 11.33 seconds. That looked great. It also disagreed with a correctly framed, per-file reference on the creation commit of one copied lesson.
A rename scan does not capture every copy that --follow can recognize, particularly a copy from a source file that was unchanged in that commit. Treating the copied file as a new creation would change its history and potentially its attribution.
I tried a full-history --find-copies-harder scan and stopped it at a 120-second diagnostic limit. The narrower solution was to check the known addition commits for copy introductions, then reserve complete history walks for the files that actually needed them.
That surrendered much of the headline speedup:
| Measured run | Time | Result |
|---|---|---|
| Original collector | 103.49 s | 12 of 58 creation records |
| First batch draft | 11.33 s | One copied lesson’s creation commit disagreed with the reference |
| Corrected collector | 53.67 s | All 58 creation commits matched the reference |
These are local measurements, not a runtime guarantee. A separate validation run under concurrent load took 70.81 seconds. The scheduled failures had occurred under a different workload; the baseline alone doesn’t explain their full five-minute duration.
The useful result is the last column. The corrected collector is faster and agrees with the history it is supposed to preserve.
Recovering history must not manufacture incidents
Better parsing recovered older creation dates for some lessons. The incident IDs included dates, so a naive rerun could treat an already-recorded lesson as a new incident simply because its recovered date changed.
I added deduplication by lesson identity while preserving the existing ledger. The corrected collector proposed 25 previously unrecorded lesson-trigger incidents, rather than counting 23 already-recorded lessons again under corrected dates.
Those are inferred historical incidents, not 25 fresh harmful actions. The collection time is not the event time, and a lesson is evidence of a behavioral guard, not proof of every detail of the event that motivated it.
The real eight-detector refresh then completed successfully. Its full-success heartbeat advanced for the first time since September 25. The existing ledger remained an unchanged byte prefix of the result: recovery appended records instead of rewriting history.
The timeout was the visible failure. The missing records were the more consequential one. Before optimizing a history scanner, check not only how long it runs, but which history survives it.