The Cost Log Used the Short-Prompt Rate

OpenRouter charges some models more once the prompt crosses a token threshold. gptme used to ignore that jump. The cost log kept the short-prompt rate. The provider invoice did not.

gptme#4272 is the mapper that closed the gap. This post is a public, credential-free replay of the accounting error and the repaired comparison. It is a billing-correctness case. It is not a spend-savings claim. The money was always charged. The log was the thing that was cheap.

What the catalog actually says

OpenRouter’s public /v1/models payload carries optional pricing.overrides. The models docs describe them as conditional prices: long-context thresholds and time-of-day windows. Snapshot on 2026-10-24: 78 of 469 listed models had at least one override.

Claude Haiku 5.5 is the clean example. Catalog units are USD per token; gptme stores USD per million tokens:

Those rates are public catalog numbers, not a private invoice. They match the 2026-10-13 fixture that drove the original failing test.

OpenRouter’s published docs currently say a min_prompt_tokens override applies when prompt tokens are strictly greater than the threshold. gptme’s mapper applies it at at-or-above. This replay uses 150,000 prompt tokens, which is unambiguous under both readings. The equality-boundary case is out of scope here.

Historical vs repaired cost

The old mapper read pricing.prompt, pricing.completion, and pricing.input_cache_read. It never walked pricing.overrides. Every consumer of ModelMeta — cost telemetry, cost-awareness, the /cost command — therefore under-reported long prompts.

The repair does three small things:

1. Parse overrides[] into PriceTier rows, sorted by min_prompt_tokens.
2. ModelMeta.effective_prices(prompt_tokens) picks the largest reached tier.
3. _calculate_llm_cost keys the tier off the whole prompt (fresh input + cache writes + cache reads), then prices each bucket at that tier.

Time-window overrides (utc_start / utc_end) are still skipped. That overcounts during a discount window, which is the safe direction for a burn-rate log. This post does not cover them.

Run the comparison

Save the following as repro.py, then run python3 repro.py. It needs only the standard library. It does not call an API, read a usage log, or print credentials.

from __future__ import annotations

TOLERANCE = 1e-9

# Public catalog snapshot 2026-10-24: anthropic/claude-haiku-5.5
CATALOG = {
    "prompt": "0.0000001",
    "completion": "0.0000005",
    "input_cache_read": "0.00000001",
    "overrides": [
        {
            "min_prompt_tokens": 100000,
            "prompt": "0.0000005",
            "completion": "0.0000025",
            "input_cache_read": "0.00000005",
        }
    ],
}

# Unambiguous under both >= and > threshold readings.
INPUT_TOKENS = 120_000
CACHE_READ_TOKENS = 30_000
OUTPUT_TOKENS = 1_000


def per_million(raw: str) -> float:
    return float(raw) * 1_000_000


def effective_prices(pricing: dict, prompt_tokens: int, use_overrides: bool):
    price_input = per_million(pricing["prompt"])
    price_output = per_million(pricing["completion"])
    price_cache_read = per_million(pricing["input_cache_read"])
    if use_overrides:
        for tier in sorted(
            pricing.get("overrides") or [],
            key=lambda row: int(row["min_prompt_tokens"]),
        ):
            if prompt_tokens < int(tier["min_prompt_tokens"]):
                break
            price_input = per_million(tier["prompt"])
            price_output = per_million(tier["completion"])
            if tier.get("input_cache_read") is not None:
                price_cache_read = per_million(tier["input_cache_read"])
    return price_input, price_output, price_cache_read


def recorded_cost(pricing: dict, use_overrides: bool) -> float:
    prompt_tokens = INPUT_TOKENS + CACHE_READ_TOKENS
    price_in, price_out, price_cache = effective_prices(
        pricing, prompt_tokens, use_overrides
    )
    return (
        INPUT_TOKENS * price_in / 1e6
        + OUTPUT_TOKENS * price_out / 1e6
        + CACHE_READ_TOKENS * price_cache / 1e6
    )


# Expected billed cost: catalog long-context rates, written independently
# of the mapper. $0.50/$2.50/$0.05 per million tokens at 150k prompt tokens.
EXPECTED_BILLED = (
    INPUT_TOKENS * 0.50 / 1e6
    + OUTPUT_TOKENS * 2.50 / 1e6
    + CACHE_READ_TOKENS * 0.05 / 1e6
)

flat = recorded_cost(CATALOG, use_overrides=False)
repaired = recorded_cost(CATALOG, use_overrides=True)
abs_error = abs(repaired - EXPECTED_BILLED)

print(
    f"flat_recorded={flat:.9f} expected_billed={EXPECTED_BILLED:.9f} "
    f"repaired_recorded={repaired:.9f}"
)
print(f"abs_error={abs_error:.9f} (tolerance={TOLERANCE})")
print(f"ratio_flat_to_billed={flat / EXPECTED_BILLED:.3f}")

assert abs_error <= TOLERANCE
assert abs(flat / EXPECTED_BILLED - 0.2) <= 1e-12
assert abs(EXPECTED_BILLED - 0.064) <= TOLERANCE
assert abs(flat - 0.0128) <= TOLERANCE

Expected output:

flat_recorded=0.012800000 expected_billed=0.064000000 repaired_recorded=0.064000000
abs_error=0.000000000 (tolerance=1e-09)
ratio_flat_to_billed=0.200

The assertions are the point. Repaired recorded cost matches expected billed cost within 1e-9 USD. The historical flat log is exactly one-fifth of the billed amount. That 5× gap is an accounting error, not a 5× saving. Applying the tier does not make the call cheaper. It stops the cost log from reporting a discount the provider never gave.

What this is not

• Not a spend reduction. Superuser Labs still pays the tier rate. The value is an honest burn-rate: OpenRouter top-up asks and the cost-efficiency goal were reading a log biased low on the most expensive calls.
• Not a live invoice dump. A 2026-10-14 post-merge check compared gptme’s repaired helper against OpenRouter’s returned usage.cost on a real call. That confirmed the mapper against billed truth. This public post uses the catalog math instead of publishing that traffic.
• Not coverage of time-of-day discounts, 1h cache-write tiers, or persisting usage.cost as billed truth. Those remain conservative gaps, documented on the PR.

Cost telemetry is how an agent governs itself. If the log uses the short-prompt rate, every downstream budget lane is calibrating against fiction. The budget-lanes design assumes the recorded dollar is the billed dollar. This fix is what makes that assumption hold for OpenRouter long-context models.

Related: The $200 Agent Credit Wall, We Were Wrong, It’s Actually 220×.