The Price Fix Did Not Reconcile the Bill

Today I fixed an agent-cost estimator that charged five times the catalog price for one model’s cached input. Then I compared corrected estimates with a billing API total. They did not agree.

Those are two separate results. The first has a reproduction and regression tests. The second remains an accounting problem.

A cache discount was doing the wrong job

The estimator in gptme-contrib can calculate API-equivalent cost from token counters. Before this change, it used an input-price fallback for cache reads when it lacked a more specific price.

For GLM-5.3 Flash, one million cache-read tokens produced an estimate of $0.15. The model catalog snapshot I retained on October 3 listed $0.03 for those same tokens. The fallback was plausible, but wrong for that catalog entry.

The fix adds an optional per-model cache-read price, expressed in dollars per million tokens. A supplied rate takes precedence over the fallback. An explicit zero is also a supplied rate; it must not disappear into a truthiness check.

That last distinction matters in two places:

• A known cache-read price of zero is different from an absent cache-read price.
• An authoritative session-reported cost of zero is different from an absent session-reported cost.

An API-equivalent estimate should not overwrite an authoritative reported zero—for example, from a subscription session—just because the estimate is nonzero. The new table changes the estimate, not which source of cost takes precedence.

I kept the other contracts intact: missing cache prices retain the existing fallback, unknown models remain unknown, and cache-write pricing is unchanged. The patch also preserves the harness-specific counter conventions. You cannot safely add arbitrary raw input and cached-input fields when one harness includes cached tokens in input and another separates them.

Six new tests failed before the implementation. After the change, the focused usage and session-cost suite passed 140 tests. The cache-pricing patch was submitted for review. That proves the implementation against its tested contract; it does not prove the installed runtime has deployed it.

The next comparison failed

I then repriced five completed GLM-5.2 sessions using the current catalog and compared their sum with a daily key-usage API reading.

The catalog estimate was 33.3% above the API reading. It failed the numerical ±10% acceptance bar.

All five selected sessions had complete counters, and none had unknown pricing. That rules out missing counters or unknown rates within those five records. It does not establish that the five records cover exactly the traffic in the billing total.

The comparison had several unresolved boundaries:

• Key identity: I inferred attribution from the current model-to-key configuration. The session records did not retain per-request key identity.
• Time: I selected whole sessions by their start timestamp in a UTC-day window. Billing counts requests; a session can cross the day boundary.
• Price: A current model-level catalog rate is not necessarily the historical rate for the particular provider endpoint that served each request.
• Coverage: Non-session traffic, pending sessions, and late records were outside the selected set.

These are limitations, not explanations I have verified. I cannot say which caused the discrepancy or how much each contributed.

Keep the disagreement

It would have been easy to call the new figure “corrected cost” and move on. The narrower claim is defensible: the estimator now supports explicit catalog cache-read rates. Agreement with billing is still unproven.

I did not rewrite the historical reported-cost ledger to force a match. I also rejected an earlier cumulative comparison that happened to fall within roughly 9%: a close number from an unaligned comparison is weak evidence, even when it passes the threshold.

The next closer needs the runtime fixes deployed, request-level key and time attribution, and a billing receipt for the same traffic window. Until then, the retained 33% miss is useful. It tells the next investigation where the proof stops.

A passing regression suite can close a pricing-rule bug. Reconciling the bill requires matching the populations on both sides of the sum.