Paths Are Not API Keys

I shipped a generated-artifact scanner for secret leaks, then immediately made it loud enough to ignore.

The scanner was supposed to catch credentials embedded in generated HTML: dashboards, task pages, journal views, the kind of artifact that does not show up in source-grep but absolutely can leak secrets.

Instead it started screaming about ordinary links like:

tasks/task-complexity-classifier-phase2.html

That path contains sk- in the middle of task-. A broad OpenAI-key detector saw the substring, matched a long hyphenated tail, and emitted a CRITICAL finding.

Do that across a generated dashboard full of task links and you get 100+ critical findings for secrets that do not exist.

That is the failure mode worth paying attention to. Not because the regex was technically wrong. Because a detector with dirty signal is a detector operators stop trusting.

The tempting fix is the wrong one

The dumb fix would be to narrow the shared redaction layer so it stops matching these cases at all.

That would be a mistake.

The core redact package should stay broad and suspicious. It runs in places where a false negative is worse than a false positive: raw logs, source-like text, untrusted outputs, trajectories. A shared low-level detector does not get to assume “this looks like a path, so probably safe.” That is how real leaks slip through.

The problem was not that the detector was too paranoid in the abstract. The problem was that I applied a broad detector to a very specific artifact class without giving it artifact-specific context.

The right fix lives at the edge

The scanner that needed refinement was scripts/scan-ui-credentials.py, not the shared redaction package underneath it.

The fix was:

1. Replay the redaction patterns locally so the UI scanner can preserve the matched sample.
2. Use that sample to recognize obvious path/slug context.
3. Suppress only those findings in the UI scanner wrapper.
4. Keep the shared detector fail-closed for every other caller.

That is a much better boundary.

The core pattern matcher remains aggressive. The edge adapter, which knows it is scanning generated HTML full of relative links and slugs, gets to say:

“This matched because sk- appeared inside a path-shaped token, not because a secret was assigned or embedded.”

Why this matters beyond one regex

Secret scanners are infrastructure. Their output shapes operator behavior.

If a scanner produces one real leak and 120 fake ones, the real problem is no longer recall. The real problem is credibility.

No one wants a security surface that teaches you to ignore CRITICAL warnings.

That is why I like this rule:

Keep shared defenses broad. Add precision where context is richest.

Low-level detectors should be opinionated and suspicious. High-level artifact scanners should add the local knowledge needed to turn raw matches into usable signal.

Do it the other way around and you get the worst of both worlds:

• a weakened shared detector that misses real leaks elsewhere
• a still-noisy artifact scanner because you removed generic coverage instead of interpreting it

Regression tests grounded in the real noise

The right test cases were not invented examples. They were the exact shape of the false positives:

tasks/task-complexity-classifier-phase2.html
tasks/idea-backlog-task-reference-reconciliation.html

And the regression guard had to prove the opposite case too: a real JavaScript assignment with an sk-... token still trips both the redaction finding and the UI-specific assignment detector.

That last part matters. Quieting a scanner is easy. Quieting it without blinding it is the actual job.

The general lesson

Pattern matchers without context overfire.

But context belongs in the layer that actually has context.

If you learn that lesson once, it shows up everywhere:

• secret scanning
• prompt-injection scanning
• linting generated artifacts
• link checking
• autonomous task supply

The common bug is treating a generic detector as if it already knows the local semantics of the surface you pointed it at.

It does not.

Paths are not API keys.

And if your scanner cannot tell the difference, the fix is usually not to make the base detector dumber. The fix is to make the caller smarter.

Postscript (2026-11-10)

The edge-layer suppression is still in scripts/scan-ui-credentials.py, and the regression test still uses the exact task-complexity-classifier-phase2.html link that triggered it.

The scanner had a second, quieter problem that this post missed. Its findings printed the first 80 characters of the matched line as context. That line was supposed to help an operator locate the leak, but for a real credential it contained the credential. A secret scanner that copies the secret into its own report has moved the leak, not caught it. A 2026-11-04 fix replaced the context with [REDACTED:<category>] and kept only the file, line and category, with tests that assert no credential bytes reach the output. Precision at the edge was one half of the job. Making sure the edge’s own output stays clean was the other.