v1.5 — Credit safety: reserve + settle
Every LLM request now reserves credits before the call, settles against the real cost after, and releases the hold on every failure path. Negative balances are blocked at the database — not by application logic.
Credits are the product. If we lose track of them, we don't have a business. v1.5 closes the last gap in the credit ledger — the gap between "we accepted the request" and "the provider billed us".
The old way (broken)
Old code path: deduct an estimated cost before the call. If the call failed mid-flight, the deduction stayed and the user got a free refund only if our cleanup script noticed. If the call succeeded but the real cost was higher, we under-charged. If the user fired two parallel requests with $0.001 balance each costing $0.0008, both passed the pre-flight check, both billed, balance went negative. None of this was acceptable.
The new way (reserve + settle)
Three operations, all atomic under Postgres advisory locks:
reserve_credits(org_id, max_estimated_cost)— debit the reservation ledger, return a reservation ID. If the resulting balance would go negative, this fails immediately with HTTP 402.- Make the LLM call.
settle_credits(reservation_id, real_cost_from_x_litellm_header)— debit the real cost from the reservation, return any unused estimate to the balance.
Every failure path (timeout, provider error, guardrail block, client disconnect) MUST call release_reservation(reservation_id). This is enforced via try/finally in route_llm_request.py and verified by the credit-integrity gap hunter (hunt-check SCANNER=credit_signal).
Database invariants
credit_balances.amount_cents BIGINT CHECK (amount_cents >= 0)— negative balances rejected at the column level. No application bug can produce one.credit_transactionsis append-only. Reservations, settlements, releases, top-ups, refunds — every state change is an immutable row.- Reconciliation: total of
credit_transactionsper org ==credit_balances.amount_cents. Drift = bug, surfaced by the ledger-parity scanner.
What you see
A 402 with error.code = insufficient_credits and your current balance in the response body. No partial bills, no overdrafts, no after-the-fact "we forgot to charge you" emails. Top up at /{org}/billing or set auto-topup to keep the balance above your floor.