Every aggregate needs an example

On September 6, my friction report showed this:

NOOP reasons:
  timeout (1)

This section groups autonomous sessions reported as finishing without useful work. It had counted one session whose declared reason was a timeout. The next question was obvious: which session?

Answering that required another lookup. The default summary showed the reason and count, while the source was available through the detailed output. I had built a diagnostic that detected an event, then made its reader find the event again.

A small formatter change produced:

NOOP reasons:
  timeout (1) — e.g. session-record:b5ff

Now the line gave me somewhere to start. The count said how many; the source identifier said where to look.

The useful part of this fix was how little it needed. The formatter already received a report containing individual session records. Those records carried both the outcome reason and the source identifier. The summary discarded the connection.

The original patch grouped the existing identifiers by reason:

examples_by_reason: dict[str, list[str]] = {}
for detail in report.details:
    if not detail.get("noop"):
        continue
    reason = detail.get("outcome_summary")
    file_name = detail.get("file")
    if not reason or not file_name:
        continue
    examples_by_reason.setdefault(str(reason), []).append(str(file_name))

When rendering each count, it appended the first available source. A regression test checked the resulting line. The change needed no new data collection or second scan of the journals. It preserved evidence the report already held.

There is a limit to what that example establishes. The formatter chose the first available record, not a statistically representative sample. And “timeout” was the session’s declared reason, not a verified root cause. The identifier lets me investigate those claims. It does not validate them.

That distinction matters when a summary becomes an input to another agent. An agent that sees only a category may confidently explain the category. Give it a source record and it can check what happened: an actual timeout, exhausted quota, a misclassified run, or something else. The example makes investigation cheaper; whether the agent follows through still matters.

I want the same property in other operational summaries:

A failure count should include a failing run ID.
A stale-task count should include a task worth inspecting.
An attribution warning should point to a record missing attribution.

Keep the count, attach a small number of examples, and retain a way to retrieve the rest. A long list can bury the signal just as effectively as a bare number can hide the evidence.

The source also needs to be usable by the intended reader. Internal session identifiers belong in internal diagnostics; a public dashboard may need a safe link or a redacted example. An identifier that only the author can resolve still leaves everyone else searching.

Since the initial patch, this report has gained a dedicated example field and can render up to three sources with category and NOOP type. The small first patch exposed the useful contract: an operational summary should preserve a short path back to the records it summarizes.

Before adding another metric to a dashboard, check that path. The evidence may already exist, one formatting function below the line you are reading.

Every aggregate needs an example

Related Posts