ADR-122: LiteLLM usage for an unknown customer is held and released by its key, never re-timed

Date: 2026-10-17 Status: Accepted (amended 2026-10-27: release counts follow the plan stretches and count no_price)

Summary

When the LiteLLM callback gets a token row whose customer or meter does not exist, Hardcount keeps the row in usage_held_events, counts it as held in the 200 response, and shows it on the Usage page. An operator releases the rows once the customer exists. A release writes each row to usage_events under its original idempotency key and time, so a row is recorded at most once, whatever order a release, a LiteLLM resend and a second release arrive in. A released row bills only if its time falls in a billing period whose invoice is not built yet. Hardcount never moves it onto a later invoice (ADR-120). There is no automatic release when a customer is created.

Context

Issue #1044. The callback answers 200 for a row whose customer or meter is unknown, because a non-2xx makes LiteLLM resend the whole batch, so one bad call would block the rest. The row was then dropped: LiteLLM reads any 2xx as delivered and does not resend. The only traces were errors[] in a response nobody reads and a count in the server log. Usage sent before the customer was created, or with a user value that does not match an external ID, was lost. usage_events.customer_id must name a real customer, so the row cannot be stored there.

What other usage-billing engines do with an event for an unknown customer (read 2026-10-17):

• Orb stores it: “If the external_customer_id is specified, the customer in Orb does not need to exist.” “Events will be attributed to any future customers with the external_customer_id on subscription creation.” (ingest events)
• Metronome stores it and attributes it later by ingest alias: “if you first send usage keyed on IngestAlias1 and later add this ingest alias to an existing customer, Metronome retroactively associates that usage to the correct customer.” (provision a customer)
• Lago saves the event without checking the subscription (app/services/events/create_service.rb), then matches subscriptions by external id and time: started_at <= event.timestamp (app/services/events/post_process_service.rb). An event from before the subscription started matches nothing.
• OpenMeter / Kong store events by subject and match customers when usage is queried. Kong: “Metering & Billing only invoices and meters events that are sent after the subscription is created.” (Kong customers)
• Stripe meter events report meter_event_customer_not_found after acceptance and do not keep the event: “Correct and resend invalid events for re-processing.” (recording usage)
• Flexprice answers 202 and the event does not show up; its fix is “Create the customer first” (troubleshooting). This was Hardcount’s behaviour too.

None of them silently drops the event while telling the sender it was delivered. None moves late usage onto a later invoice; Lago and Kong attribute only inside the subscription’s own time window.

Correction (2026-10-23): two facts are missing above. Flexprice can also create the customer, and then a subscription and a wallet, from the usage event itself, through a Temporal workflow, if the tenant turns on its customer-onboarding setting. And Kill Bill, which is not in the list above, does move late usage onto a later invoice (see ADR-120). Checked in Flexprice 1c23b81 (2026-10-22): internal/ee/service/meter_usage_tracking_post_insert.go:200-244, internal/temporal/models/workflow.go:28-32.

One fact shapes the value of a release. In Hardcount a subscription cannot start in the past (subscription.CreateInput has no start date: a subscription starts now or is a draft), and an invoice adds up only the usage inside its own period. So rows that arrived before the customer existed usually fall before any billing period, and a release stores them without billing them. The value is: never lose the row, show it, and release it once with an honest statement of what it will bill.

Decision

1. Hold. Migration 0179 adds usage_held_events (row-level security, livemode from the session, UNIQUE (tenant_id, livemode, idempotency_key)). The LiteLLM handler calls usage.Service.HoldCall (through a consumer-defined Holder) when the customer or meter lookup returns not-found. It holds the call’s rows in one transaction. When a row already stored or held under the call’s input key has the quantity the mapper wrote before the 2026-10-22 cache-write split, it holds the rows in that older form, with no cache-write rows (item 6). Each insert is ON CONFLICT DO NOTHING; a resend of a held row is counted as deduplicated. A hold that cannot be written is a 503, so LiteLLM resends. A payload with no id (no key) or no user stays an error and is not held.
2. Show. GET /v1/usage-events/held groups the held rows by external id, event name and reason, with whether a matching customer and meter exist now. It lists the 200 external ids held most recently, each with all its groups, and returns total_customers and total_rows over every held row so the notice can say when the list is cut. The Usage page shows them in a notice with Release and Discard.
3. Release once. POST /v1/usage-events/held/release runs one transaction:
   • claim the rows with UPDATE … SET status='released' WHERE status='held' … RETURNING, so concurrent releases get disjoint rows;
   • insert each through ingestOneTx, the one INSERT every ingest path uses, with its original key and time and origin backfill;
   • record the event id, write one audit row, and commit. If usage_events already has the key (LiteLLM resent after the customer existed), the held row is marked released and nothing is written. Rows whose meter is still missing, or whose time is after the customer’s current time (a test clock in the past), stay held (still_held). One release claims at most 5,000 rows; the rest it could have taken are counted apart (more_to_release), and another release takes them.
4. Never re-time. The release response counts released rows inside an active subscription’s current period on a meter its plan prices (in_open_period), inside that period on a meter no plan on it prices (not_on_plan, never invoiced), and outside it (outside_open_period). The plan check read each item’s current plan, so a row from before a mid-period plan change was judged by the new plan. Since 2026-10-27 (ADR-120 amendment of that date) it reads the plan stretches the close bills by, so such a row is judged by the plan it was on, and a row that plan prices but no price of it takes counts as no_price. The counts are what Hardcount found when the release wrote the rows, as for ingest’s billing_status (ADR-120 amendment of 2026-10-27). An outside_open_period row was then outside every subscription’s billed time, in a period already invoiced (whatever plan the subscription is on now), or before a paid threshold invoice’s end. The operator bills it with a one-off invoice, as for any late usage (ADR-120). Since 2026-10-22 the counts follow ADR-120’s after-close rule, spend-threshold invoices included: a sum or count row before the end of a paid one counts as outside_open_period, and before the end of an unpaid one as pending_threshold_invoice (billed by the period’s invoice only if that threshold invoice is voided first).
5. Discard. POST /v1/usage-events/held/discard marks rows discarded. They are kept as a record and never billed. Audited.
6. A resend after the customer exists. The LiteLLM door writes each call through IngestCall, in one transaction per call. It first locks the held rows with the call’s keys (SELECT … FOR UPDATE, the lock release and discard take). A discarded row means nothing is written and the resend counts as deduplicated, so discard stays final. A held row is written and marked released, so a later discard cannot mark usage that is already billed. Two cases mark a held row released and write nothing, so its released_event_id stays empty: usage_events already has its key (the usage is billed), or it is a cache-write row of a call whose input row has the quantity from before the 2026-10-22 cache-write split, which counted the writes inside the input.

Why this design

The idempotency key is the one fact that makes “release later” safe. The held row carries the exact key the mapper builds (<id>:input, <id>:cache_read, <id>:output), and the release goes through the same INSERT and the same unique constraint as live ingest. So the row lands at most once, and no new dedup mechanism is needed.

Holding is scoped to the LiteLLM door because it is the only door that must answer 200 for a row it cannot ingest. POST /v1/usage-events answers 422 synchronously, so its caller already knows.

Alternatives considered

• A. Only report (a counter and a log line). Stripe’s shape. Rejected: Stripe’s sender sees the error and resends; LiteLLM never sees errors[] and never resends, so a report alone still loses the usage.
• B. Release automatically when the customer is created. Rejected for now: it makes the customer domain call usage (a new cross-domain edge), needs a fallback for a crash between create and release, and mostly adds rows that do not bill (see Context). It can be added later through a consumer-defined interface on customer.Service.
• C. Bill released rows on the next invoice. Rejected: it re-times usage into a period it did not happen in. No engine checked does this, and ADR-120 already decided against it for late usage.
• D. Hold rows with no user. Deferred: they need an “assign to a customer” step. They stay errors.

Consequences

• An operator sees held usage on the Usage page without reading logs, and the LiteLLM response gains held.
• usage_held_events grows only with misrouted traffic; there is no retention job. Released and discarded rows stay as a record.
• A row with no payload time is stamped with the time Hardcount received it. Released to a customer on a test clock frozen before that time, it stays held until the clock passes it.
• Metric hardcount_usage_held_events_total{reason} counts new held rows.