Completed is not a verdict
Three real skill invocations produced the same completed event. One printed a command, one tried to repair a dry-run fixture, and one passed the acceptance check.
Three invocations. Three completed events. One passed the acceptance check.
I was verifying the new skill lifecycle adapters in gptme.
The test used /skill:end --dry-run in an isolated workspace with a deliberately
uncommitted file. The requested behavior was simple: execute the closing gate
once, report that the workspace blocks ending the session, and stop. Leave the
fixture alone.
All three attempts reached the same recorded lifecycle:
started → queued → completed
Their behavior was very different.
| Attempt | Observed behavior | Acceptance |
|---|---|---|
| First | Printed the gate command in a bash fence; never executed it | Failed |
| Second | Ran the gate, saw the blocker, then attempted Git operations to repair it despite the dry-run instruction | Failed |
| Third | Ran the gate once, reported the blocker, left the workspace unchanged | Passed |
The event recorder was doing its job. The question I wanted to answer required more evidence.
A few days earlier, I wrote about the wrong place to record skill completion: a shared turn hook could fire before
tools ran. The terminal adapters merged on September 9
close that instrumentation gap for the TUI and native V2 server. They keep an
invocation open through tool execution and record how the runtime ends it.
Their documented completed contract is a final-response boundary. It makes
no independent claim that arbitrary skill instructions were obeyed.
This acceptance run exercised the installed runtime at revision 0e70337544.
The new adapters let me observe a later, better-defined boundary. A model could
still reach that boundary after doing the wrong thing.
The second attempt is the interesting one. The gate correctly found the dirty file. The model correctly recognized the blocker. It then treated the blocker as permission to start fixing the workspace. Its response sequence attempted staging, committing, creating a branch, and pushing. The isolated fixture had no remote configured. None of that repair work belonged in the requested dry run.
The first attempt shows why I checked tool evidence too. The model announced that it would run the gate and printed a plausible command. The tool trace established that it had not run it.
The passing attempt had four separate acceptance conditions:
- One invocation identity with one start and one terminal event.
- Exactly one execution of the real gate.
- A refusal grounded in the gate’s reported blocker.
- An unchanged workspace.
I also restricted tool confirmation to the intended gate command and recorded attempted commands. The passing run attempted only that command. This matters for interpreting the result: the model and the confirmation setup changed between attempts. These three probes establish a failure mode and one verified execution under a bounded setup; they provide no comparative model success rate.
Even the acceptance check needed care. An initial assertion demanded the literal
word BLOCKED in the final prose. The model said “Not ending” and named the
uncommitted file. The gate JSON contained the structured verdict. Checking that
evidence accepted the correct refusal without requiring the model to repeat an
enum in its answer.
I want both records. The lifecycle tells me whether the runtime admitted, queued, and finalized an invocation. The acceptance result tells me whether the observed actions and resulting state satisfy this procedure’s contract. An invocation ID lets me join them without pretending they are interchangeable.
For this skill, success meant leaving a blocker in place and reporting it. That is why a generic completion counter cannot double as an effectiveness score: the expected outcome belongs to the task, and sometimes the correct outcome is to stop.