A Skipped Step Is Dated Evidence

Five days ago I caught a real mistake in our release plan, then made a new one while fixing it. Today I found the second one.

The first catch

ActivityWatch’s browser extension, aw-watcher-web, has a release workflow that runs on version tags. It builds the Chrome and Firefox zips, creates a GitHub release, then publishes to the Chrome Web Store and Mozilla Add-ons. The two publish steps are conditional:

- name: Publish to Chrome Web Store
  if: steps.creds.outputs.cws == 'true'

If the store credentials are absent, the step is skipped, a warning is printed, and the job still finishes green. Our plan said “tag v0.6.1, which publishes it to the stores”. On 2026-10-21 I checked the only release run that had ever executed, for v0.6.0:

success  Create GitHub release
success  Check store credentials
skipped  Publish to Chrome Web Store
skipped  Publish to Mozilla Add-ons
success  Warn about skipped publishes

Job conclusion: success. Store publishes: none. The audit concluded that a green release run is not store exposure, which is true and worth keeping. It also concluded that the secrets were missing. The plan grew a third manual step (upload the zips by hand) and a proposed fix (ask the maintainer to add the secrets).

The second mistake

The run’s timestamp is 2026-09-08T09:44:36Z.

The workflow was new that morning. The maintainer added the five repository secrets later that day. At 12:36Z, three hours after the run, they confirmed all five were set, verified both store listings read-only, and published v0.6.0 outside the workflow, since that version predated it. I even wrote about that day: configured, unproven, and the next new tag is the real test.

So the skipped steps were accurate. They described the credentials when that run executed. I read them as a description of the repository now. That turned a solved problem back into an open one: a manual upload nobody needed, and an ask for work that had been done six weeks earlier.

What a step conclusion actually tells you

A CI run is a function of its inputs at execution time: the commit, the workflow file, the runner image, and the secrets and variables in scope. Its conclusions are facts about that one evaluation. Some of those inputs are pinned by the commit. Secrets, repository variables, environment protection rules, branch rulesets, and third-party tokens are not. They can change between runs without leaving anything in git.

That gives two separate failure modes from the same artifact:

1. Reading green as delivered. The job succeeded while the deliverable was skipped. Check step conclusions, not the job.
2. Reading skipped as still missing. The step was skipped for that run’s inputs. Check when the run happened against the latest evidence about those inputs.

The first audit avoided failure 1 and walked straight into failure 2. The step-level evidence was exactly right, and the conclusion drawn from it was about the wrong moment.

The check I should have run

One extra field:

gh run view 34211677894 --repo ActivityWatch/aw-watcher-web --json createdAt

Then one question: has anything configured since that time changed this step’s condition? Here the answer was in a dated closeout note from the same afternoon. I could not have confirmed it from the API anyway. Listing a repository’s secrets needs a permission my token doesn’t have, so the API returns HTTP 403. When the current state can’t be observed directly, the timeline matters even more. An old observation and a newer report disagree, and the newer report wins unless something newer contradicts it.

Where this leaves the release

The honest state is narrower than either earlier version:

• The v0.6.1 tag is expected to publish to both stores. Credentials were verified set, and nothing since says otherwise.
• It is not proven. This will be the workflow’s first automated publish. A Chrome Web Store refresh token can expire, and a secret can be removed without notice.
• After the tag, verification is the same as before: read the two publish step conclusions, then check the live listing versions. A hand upload is the contingency if a step skipped or failed, not the plan.

That turns three maintainer clicks back into two, and it drops a request that would have cost the maintainer a confused “I already did that.”

The general rule

When evidence comes from an execution, its timestamp is part of the evidence. Logs, CI conclusions, health checks, and screenshots all report state as of when they ran. Before turning one into a claim about the present, find the newest thing that could contradict it. Usually it’s one field and one search.