Handling Late-Arriving Usage Events After Cycle Close

A metering pipeline that assumes events arrive in order and on time works until the first network partition. Then a batch of usage timestamped for the 30th arrives on the 3rd, the invoice for that period has already been issued, and someone has to decide whether to reopen a closed document, absorb the cost, or bill it later. This guide sits under Usage-Based Billing Implementation and covers designing that decision before it arrives as an incident.

The core insight is that “when did this happen” and “when did we learn about it” are different facts, and a schema that stores only one of them cannot express lateness at all. Once both are recorded, every other rule — the grace window, the watermark, the adjustment — becomes expressible.

The second insight is that closed invoices should stay closed. Reopening a finalised document breaks sequential numbering, invalidates tax filings that referenced it, and confuses every downstream system that already consumed it. Adjustments are almost always the better answer.

Trade-offs

Strategy for late events Invoice integrity Customer experience Revenue accuracy Complexity
Reopen and reissue the invoice Broken Confusing correction Exact High, plus tax rework
Next-period adjustment line Preserved One extra line, explainable Exact, delayed Low
Discard events past the window Preserved Invisible Under-bills Trivial
Extend close until all events arrive Preserved Late invoices Exact Unbounded delay
Grace window and adjustment path Events arriving within the grace window join the period's aggregate, while anything later is posted as an adjustment on the following period. Period open events accumulate Grace window stragglers still count Frozen and invoiced later → adjustment the freeze is a decision, not an accident of job timing bounded delay, bounded correction, no reopened documents
A bounded grace window converts an open-ended correctness problem into a scheduling one.

Discarding late events is more common than teams admit, usually by accident: a pipeline with no lateness handling silently drops events whose period is closed. If that is the behaviour, it should at least be a deliberate policy with a measured cost, because under-billing is a revenue decision.

Step-by-Step Implementation

1. Store both timestamps

CREATE TABLE usage_events (
  event_id     TEXT PRIMARY KEY,              -- client-supplied, deduplicated
  customer_id  UUID NOT NULL,
  meter_key    TEXT NOT NULL,
  quantity     NUMERIC NOT NULL,
  occurred_at  TIMESTAMPTZ NOT NULL,          -- event time: when usage happened
  ingested_at  TIMESTAMPTZ NOT NULL DEFAULT now(),  -- when we learned about it
  billing_period_start DATE,                  -- assigned at rating time
  adjustment_for_period DATE                  -- set when it lands late
);

CREATE INDEX ON usage_events (customer_id, meter_key, occurred_at);
CREATE INDEX ON usage_events (ingested_at) WHERE billing_period_start IS NULL;

Lateness is ingested_at - occurred_at, and it is a distribution worth charting from the first day. Most systems find that the ninety-ninth percentile is far larger than anyone assumed.

2. Close on a watermark

The period should not close because a cron job fired; it should close because the pipeline asserts it has processed everything up to a point in event time. That assertion is the watermark.

def watermark(consumers) -> datetime:
    """The earliest event-time position across all partitions — nothing older is outstanding."""
    return min(c.position_event_time for c in consumers)

def can_close(period_end: datetime, consumers, grace: timedelta) -> bool:
    return watermark(consumers) >= period_end + grace

A single lagging partition holds the watermark back, which is correct: closing while one customer’s events are still draining is precisely how an under-billed invoice is produced.

3. Choose the grace window deliberately

Sizing the grace window A grace window shorter than typical lateness produces frequent adjustments, while one far longer delays every invoice for a rare straggler. Too short below p95 lateness adjustments every cycle noisy invoices p99 plus margin a few hours typically rare adjustments the target Too long days of waiting invoices land late cash delayed
Size the window from the measured lateness distribution, not from a round number that feels safe.

4. Post later events as adjustments

def assign_period(event, closed_periods) -> tuple[date, date | None]:
    """Returns (billing_period, adjustment_for_period)."""
    period = period_containing(event.occurred_at)
    if period in closed_periods:
        return next_period(period), period      # bill next cycle, labelled for the old one
    return period, None

Carrying adjustment_for_period onto the invoice line is what makes the extra charge explainable: “usage for March, billed in April” is a line a customer accepts, while an unexplained April overage is a support ticket.

5. Alert on the distribution, not the event

A single late event is normal. A shift in the distribution — the ninety-fifth percentile doubling, or a customer whose events are consistently hours late — is a pipeline problem. Alert on the aggregate and let individual lateness be handled by the rules above without human involvement.

Bounding the Correction You Owe

Adjustments are the right mechanism and they still need a bound, because an unbounded correction window means an invoice from six months ago can be added to today’s bill.

Set a maximum adjustment age — commonly one or two billing periods — beyond which late events are recorded but not billed. The cost of that decision is measurable: sum the quantity of discarded events and price it. In practice the number is small, because genuinely ancient events are almost always a replay or a backfill rather than real usage, and billing a customer for a six-month-old event they cannot verify damages trust more than the revenue is worth.

Treat backfills as a separate class entirely. When a customer or an internal job replays historical data, those events carry old occurred_at values and current ingested_at values, and running them through the normal lateness path can generate an enormous adjustment. Mark backfilled batches explicitly at ingestion, exclude them from billing by default, and require an explicit decision to bill any of them.

Cap the adjustment as a proportion of the original period too. An adjustment larger than, say, a fifth of the period it corrects is more likely to be a pipeline defect than genuine late usage, and it should be held for review rather than invoiced automatically. The customer-facing cost of a held adjustment is a day of delay; the cost of an incorrect one is a refund and a credibility loss.

Finally, publish the rules. A customer whose contract or documentation states “usage may be adjusted on the following invoice for up to one period” has agreed to the mechanism in advance, and the first adjustment is then an expected event rather than a dispute. The same clarity helps internally: support can explain an adjustment line without escalating when the policy is written down.

Bounding late adjustments Adjustments are limited by age, by proportion of the original period, and by whether the batch was marked as a backfill. Age bound one or two periods older: record only measure what is dropped Size bound share of the period over: hold for review likely a pipeline defect Backfill flag marked at ingestion excluded by default billed only on a decision
Three independent bounds — each one has, somewhere, prevented an accidental invoice nobody could explain.

Measuring Lateness Before It Costs You

The lateness distribution is the single most useful metric in a metering pipeline, and it is almost never instrumented until after the first incident.

Chart the difference between ingestion time and event time as a histogram, segmented by customer and by ingestion path. Segmentation matters: a mobile SDK that batches uploads behaves nothing like a server-side integration, and a single blended distribution hides a customer whose events consistently arrive a day late because their agent buffers.

Track the tail rather than the average. The average lateness is dominated by the overwhelming majority of events that arrive within seconds and tells you nothing about the grace window you need. The ninety-ninth percentile is the number the window should be sized from, and the maximum is the number that tells you whether your adjustment bound is realistic.

Alert on shifts, not on values. A pipeline whose ninety-ninth percentile moves from two minutes to two hours has developed a problem worth investigating even though both numbers are inside a generous grace window. Absolute thresholds catch only the incidents severe enough to be obvious.

Publish the adjustment volume alongside it — how many events were billed late, and how much they were worth, per period. That figure is what justifies engineering effort on the pipeline: if adjustments are consistently a trivial fraction of revenue, the current window is fine, and if they are growing, the case for work is quantified rather than argued.

Finally, keep a per-customer lateness view available to support. When a customer questions an adjustment, the answer is frequently that their own integration delivered late, and being able to show when their events were sent versus when they occurred turns a dispute into a shared debugging session.

Verification & Testing

Test the watermark logic with a lagging partition: advance three consumers past the period end and hold the fourth behind it, and assert the period cannot close. Then advance the fourth and assert it can. This is the property that prevents under-billing and it is easy to break during a refactor of the consumer group.

Test period assignment across the boundary with a fixed clock: an event whose occurred_at is inside the period and arrives during the grace window joins the period; the same event arriving one second after the freeze becomes an adjustment carrying the original period label. Assert both the amount and the label.

Test the bounds: an event older than the maximum adjustment age is recorded and not billed, an adjustment exceeding the proportional cap is held, and a batch marked as backfill is excluded regardless of age. Each is a policy that will otherwise be discovered missing during an incident.

Gotchas & Production Pitfalls

  • Closing on wall-clock time. Guarantees under-billing whenever the pipeline lags, and the shortfall is invisible because the invoice looks complete.
  • Only storing one timestamp. Without both event time and ingestion time, lateness cannot be measured, bounded, or explained.
  • Reopening finalised invoices. Breaks sequential numbering, tax filings, and every downstream consumer. Adjust forward instead.
  • Backfills billed as late usage. A replay of a month of data becomes a single enormous adjustment. Mark and exclude backfills at ingestion.
  • Adjustment lines with no explanation. An unlabelled extra charge reads as a billing error even when it is correct.

Frequently Asked Questions

How long should the grace window be? Long enough to cover the ninety-ninth percentile of measured lateness with margin, which is usually hours rather than days. Measure before choosing; assumptions here are consistently wrong in the optimistic direction.

Should late events ever reopen an invoice? Almost never. The exception is a period whose invoice has not yet been finalised or sent, where amending is genuinely cheaper than adjusting.

What if the customer disputes an adjustment? The label and the event records are the answer: the period the usage belongs to, the quantity, and the ingestion time. That evidence resolves most disputes in one message, which is the reason to carry the label at all.

Does this interact with revenue recognition? Yes. Usage recognised in the period it occurred, but invoiced later, creates an accrual. Keep the adjustment_for_period label on the ledger entry so revenue recognition can place it in the correct period rather than the invoiced one.