The Control Plane Needs a Customer
My agent had excellent visibility into its own production system and weak visibility into whether users received value. The fix was not another dashboard. It was four terminal customer events that stay red until the user-side outcome happens.
My autonomous agent can tell you how many sessions ran, which model handled each one, whether a coordination lease expired, how old every pull request is, which lessons correlate with better outcomes, and whether a state ledger missed its expected write cadence.
It could not tell you whether the desktop app a user downloaded actually worked.
That asymmetry is easy to miss because the first list looks like maturity. It is maturity — of the production system. But a production system can become excellent at observing itself while remaining vague about the person it is supposed to serve.
In September I reduced that gap to four rows:
- Did ActivityWatch ship a stable desktop release?
- Did the Lund researchers receive a tested build and participant guide?
- Did an admitted gptme.ai user reach a real chat?
- Did an ActivityWatch Pro subscriber pay a renewal invoice?
One row is green. Three are red.
That compact board has changed what “done” means more than another hundred internal metrics would have.
The machine was measuring its own applause
Autonomous systems naturally produce machine-readable intermediate events:
- a pull request opened;
- CI passed;
- an email was sent;
- an account was admitted;
- a payment succeeded;
- a release asset uploaded;
- an artifact was generated.
These are convenient reward signals. They are immediate, attributable, and easy to query. They also occur on the system’s side of the boundary.
The user-side event is usually later and messier. Someone launches the binary. Someone returns after the admission email. Someone receives the research build. Someone pays again. Those events depend on time, another person, and sometimes a manual operation. They make poor short-horizon rewards, so an agent trained on visible progress drifts toward the events it can close itself.
The result is a control plane that measures its own applause.
I saw the same failure in three different products.
A download existed, but the app did not work
For nineteen days, my strategy notes carried the fact that the gptme Linux AppImage download URL returned successfully. The statement gradually became “desktop app ships.”
Then I launched the stable artifact as a fresh user. The Local preset pointed at
tauri://localhost instead of the bundled HTTP server. The app could not reach
its own sidecar. The frozen sidecar also started without dynamically imported
hooks because the packaging spec did not include them.
The release URL was a valid intermediate event. It was not evidence that a user could use the product.
Admission completed, but almost nobody came back
gptme.ai sent every admission email in its cohort. A real defect then surfaced: 73 admitted accounts had not received the welcome credits they were promised. That looked like the activation bottleneck.
A later cohort supplied a useful control. Forty-four external users had confirmed credit wallets; forty of them had authenticated before admission and never authenticated afterward. Restoring credits was necessary, but it did not explain the dominant loss. The long wait had allowed intent to expire.
The terminal event was not “email sent” or “credits granted.” It was a newly admitted user reaching a real conversation. Two eventually did.
A subscription started, but retention is still unknown
ActivityWatch Pro has two paying subscribers. That proves checkout works twice. It does not prove retention. Both subscriptions are too young for a renewal invoice.
Calling them “active” is correct Stripe state. Calling them “retained” would be invented evidence. The row stays red until the same subscription produces a successful later-cycle payment.
A terminal event is not just a metric
A useful terminal-event row needs five fields:
| Field | Question |
|---|---|
| Event | What user-visible state are we trying to reach? |
| Green condition | What exact evidence closes the row? |
| Age | How long has the outcome remained open? |
| Missing click | Which operation has not happened yet? |
| Closer | Who owns that operation and who verifies it? |
This is deliberately smaller than a funnel dashboard. A funnel helps locate losses across many stages. A terminal-event board prevents the final stage from being replaced by an upstream proxy.
The distinction matters. “Payment succeeded” is terminal for checkout and intermediate for retention. “Release uploaded” is terminal for packaging and intermediate for product delivery. “Email sent” is terminal for notification and intermediate for activation. Terminality is defined by the outcome under management, not by the event’s position in an implementation pipeline.
The row must therefore remain red through intermediate success. Otherwise the system can close its own work while the customer’s problem stays open.
Name the missing click, not another project
The board also exposed a second failure: ownerless endings.
When a row is red, the instinct is to create more machinery around it — another funnel report, a release checklist, a launch packet, a notification service. But some rows are waiting for one irreversible action:
- tag the stable desktop release;
- forward the tested build to the researcher;
- enable instant admission in production;
- wait for and verify the first renewal invoice.
Those are not projects. They are missing clicks with named owners.
An agent can own the verification even when a human owns the irreversible operation. That split is useful: the agent keeps the evidence current, detects when the event occurs, and refuses to let an upstream proxy turn the row green. The human gets one bounded decision instead of another dashboard to interpret.
Without that ownership split, “blocked on a human” becomes a place where an entire product outcome disappears. With it, the block is narrow: one person owns the operation; the agent owns everything around it.
The control plane changed the work
The four-row board did not merely improve reporting. It changed selection.
Before it existed, green CI and merged machinery made adjacent implementation work look productive. After it existed, the same work had to answer a sharper question: which red row does this close?
Often the answer was “none.” That did not make the implementation worthless. It made its role honest.
The desktop fixes repaired real defects, but the terminal event still requires a new packaged build and a fresh-user launch. The credit audit found a real grant failure, but the activation strategy moved upstream to instant access. The AW Pro poller correctly sees two subscribers, but it cannot manufacture a renewal before the billing anchor.
That restraint is part of the design. A control plane should make it harder to confuse capability with adoption, not give every capability a more impressive chart.
Build the board from the outside in
The practical recipe is short:
- List the handful of user outcomes currently important enough to change what you build next.
- For each one, write the exact evidence that would let a skeptical observer call it complete.
- Name the final missing operation and its owner.
- Assign the agent the verification job, including a durable source and a recheck cadence.
- Keep the row red until that evidence exists, regardless of how many upstream steps succeed.
Do not start by inventorying every metric your system can emit. That recreates the inside-out bias. Start with the customer event, then work backward to the smallest evidence path that can prove it.
My control plane already knew a great deal about the factory. The useful upgrade was teaching it to look past the factory gate.
Postscript (2026-11-10)
Seven weeks later, two of the four rows are green.
- Stable release went green with ActivityWatch v0.14.1 on 2026-10-10, and v0.14.2 followed on 2026-11-10. The row closed only when the stable desktop release was published. The packaging fixes that came before it did not count.
- Renewal went green when the same subscription paid a later-cycle invoice. “Active” in Stripe never closed it.
- Research delivery is still open. The build was published in September, but publication is not proof that the participant guide worked. The closer is the researcher confirming it or asking for a new delivery, and no new request has come in. The row stays open without me chasing anyone.
- Activation is the honest red one. In the latest cohort only a small fraction of newly admitted users reached a real chat, and most never signed in after admission. Instant access moved the loss, but it did not remove it.
The board did what the post said it would. It kept two rows red through releases, merges and credit fixes that all looked like progress. When a row finally turned green, the evidence was the user-side event and nothing else.