Spend · Billing Guard
Billing Guard
See it in motion
Where to find it
- Dashboard: the Spend > Telemetry > Billing Guard subtab of the local dashboard.
- API:
GET /api/billing-guard?repo=<id>&window=<days>&tolerance=<pct> - CLI: repoops-billing-audit, the same engine and evidence bundle without the dashboard. It runs seven of the nine signatures: the two request-level ones (duplicate-request, ghost-prompt) read the local request store, which only the dashboard and the API feed in.
- Claude Desktop: the .mcpb extension for managed laptops.
The three-ledger model
Every reconciliation joins three independent records of the same spend:
- Your ground truth. Claude Code session telemetry captured locally: per-request token counts by type (input, output, cache read, cache write), model, and timing. It never leaves your machine.
- The provider's claim. Anthropic's Admin Usage & Cost API: token usage by model and service tier, plus the daily cost report. Anthropic's own documentation lists cost reconciliation as a primary use case for this API.
- The independent recomputation. A versioned per-model rate card prices your token counts, with cache reads and cache writes at their own multipliers. Token counts are the stable primitive; dollars are derived.
The join runs day by model, the granularity all three sides share. A day where all three agree inside the tolerance band reconciles. A day where they disagree becomes a finding.
The nine detector signatures
- variance-band: the billed day total exceeds the telemetry-recomputed expected cost beyond the tolerance band.
- ghost-usage: the provider reports usage for a day and model on which local telemetry (which was capturing that day) logged zero tokens.
- rate-check: the billed amount implies a per-token price matching no known rate-card row.
- duplicate-burst: provider-reported usage is at least 2x the locally logged usage for the same day. Runs only on days with no request-level coverage.
- cache-share-divergence: the cache-read share of input-side tokens diverges between local telemetry and the bill for a day and model by more than the materiality gap (15 share points), even when total cost stays within band.
- delta-alarm: day-over-day billed total grew 3x or more.
- contradiction: the provider's own surfaces disagree; the usage report shows zero tokens while the cost report bills money.
- duplicate-request: the same request id appears more than once in the local request store for a day. A definite double-log rather than a statistical guess, so it supersedes duplicate-burst on any day the request-level store covers.
- ghost-prompt: a local api_error record still carries a nonzero cost. Needs no Admin API report; the local request log alone is enough.
Every finding carries the verdict verified-mismatch: both sides of the comparison were present and disagreed. Anything the engine could not check lands in coverage, never in findings.
Reading the dashboard section
- Per-day ledger. One chip per billed day: green check for clean inside the band, red cross for a day with findings, amber question mark for cannot-verify. Hover a chip for billed vs expected.
- Findings table. One row per finding: day, model (or day total), expected, billed, delta, and the signature badge.
- Coverage summary. Telemetry-covered days vs billed days, verified mismatches vs cannot-verify days, and which of the three sources responded. Structural blind spots appear as cannot-cross-verify annotations.
- Tolerance band control. The band the comparisons ran inside, read back from the API response. Change it and the section re-runs.
- Evidence bundle export. Downloads a JSON bundle (the same format the CLI writes) with your token counts, the recomputed expected cost, the provider's figure, and the delta per finding, ready to attach to a support ticket.
Without an Admin key the section runs in telemetry-only mode: the local expected-cost ledger still renders, with an honest prompt to set ANTHROPIC_ADMIN_KEY before anything can be verified against the invoice.
The tolerance band
Anthropic explicitly documents client-side cost figures as approximations, so Billing Guard never compares dollars for cent-equality. Expected cost is recomputed from token counts times the published per-model rates, then compared inside the band (default 2%). A band tighter than the real-world approximation noise produces noise, not findings; if your baseline drift runs hotter than 2%, widen the band before trusting the alerts.
Measuring your baseline
Before trusting any band, measure what your normal drift actually is. Billing Guard computes the per-day drift between expected and billed, over fully-verified days only (days your telemetry captured and every logged model priced by the rate card), and reports the distribution: median and 90th-percentile absolute drift, the maximum, and a recommended band (the 90th percentile with a 25% margin, rounded up to the nearest 0.5 point, never below 1%). The recommendation is never applied for you; read it, then set the band yourself.
- Dashboard: the measured-baseline line next to the tolerance-band control in the Billing Guard section. With too few verified days it says so instead of guessing.
- CLI:
node bin/billing-guard.mjs --measure-baselineprints the per-day drift table plus the recommendation;--jsonbundles carry abaselinefield. - Persisting your choice: set
REPOOPS_BILLING_GUARD_TOLERANCE_PCTin the aggregator's environment and it becomes the default band for the API, the CLI, and the daily alert sweep. An explicit?tolerance=or--tolerancestill wins.
Alerts
With ANTHROPIC_ADMIN_KEY set, Billing Guard also checks the last seven days once a night and pushes a notification for the two signatures that compare the provider's own surfaces against each other: runaway day-over-day invoice growth (delta-alarm) and the usage-says-zero-but-cost-bills-money contradiction. The other signatures (variance-band, ghost-usage, duplicate-burst, cache-share-divergence, rate-check) stay dashboard and CLI visible only, so a coverage gap never pages you. Alerts flow through the same notifications spine as the rest of RepoOps, so they appear on the Notifications tab and in any Slack or email channels you already connected. Each finding alerts at most once per seven days per day, model, and signature; the copy carries the day, the model, billed vs expected, and the signature name.
- No key, no noise. Without an Admin key the sweep does nothing: missing coverage is never an alert.
- Off-switch: set
REPOOPS_BILLING_GUARD_ALERTS_OFF=1to disable the sweep even with the key present.
Honest limits
- Telemetry is opt-in and local. A day with no local capture reads cannot-verify, never a finding. Missing coverage is not evidence of overbilling.
- Subscription plans cannot verify. The Usage & Cost reports need an organization Admin key; Claude Pro/Max plans do not expose invoice-side data this way.
- Known Admin API blind spots. Priority Tier costs are excluded from the cost report and code execution from the usage report; both surface as annotations rather than silently mismatching.
- The rate card is versioned, not omniscient. A model missing from the card makes its day cannot-verify rather than producing a made-up expected cost.
- Single-machine view, with a per-repo rollup above it. The dashboard section reconciles the telemetry this install captured. The hosted rollup at /team/billing-guard aggregates reconciliation history per repository, not per developer, and only once a connected desktop syncs its sweep.
- Scope gap. Local telemetry sees a single working directory, while an organization Admin key bills the whole org. Detectors that compare against local capture are marked scope-sensitive for exactly this reason; the ones that compare the provider's surfaces against each other or the rate card are not.
- Detection only, never enforcement. Billing Guard has no budget caps, no kill switch, and no throttling. It tells you what to dispute; it cannot stop a charge.
Read more
- Billing Guard overview: why AI invoices deserve an audit trail.
- Billing Guard CLI: the same audit from a terminal or a scheduled task.
- Billing Guard for Claude Desktop: the .mcpb extension.
- Who audits your AI bill?: the research behind the detectors.
Last updated