Chapter

Invariants, Idempotency and the Outbox

Part D · Correct Changes|3,925 words|about 17 min read|Volume D

26.0 What this chapter gives you#

A reliable operation must survive more than a clean first attempt. Requests can arrive twice, replies can disappear, and external delivery can fail after the database has committed.
This chapter combines three ideas: an invariant defining what must stay true, an identity defining which attempts represent one intent, and an outbox recording external work implied by an accepted local change.
The resulting pattern has precise boundaries. It can make a bounded local reservation protocol repeatable; it does not establish universal exactly-once execution, physical-stock accuracy, authentication or fault-free external delivery.

26.1 Rules across operations#

26.1.1 PLAIN — in simple words#

An invariant is a rule that must hold across the relevant accepted operations. “Available stock never becomes negative” is one possible rule. “Every accepted reservation has a matching stock allocation” is another.
A workflow can satisfy one while breaking the other. Stock can remain non-negative even when the application forgets to record who reserved it.
Write the rules before selecting the mechanism. A transaction, a unique key and a queue each protect different boundaries; none is a general replacement for understanding the intended state changes.

26.1.2 PLAIN — a picture in your head#

Mira keeps a stock ledger, a reservation register and a dispatch list. A useful rule connects them: every accepted reservation explains a stock reduction and creates the required confirmation task.
Checking each notebook separately is not enough. Their relationships are part of the work.
Where the comparison breaks: the notebooks can represent promises about records, but actual goods and delivered messages remain outside the local database. A consistent ledger is evidence about the ledger first.

26.1.3 PLAIN — a worked example#

In a new local teaching system, available stock begins at 5. Request IDEM-A asks for 3. Acceptance should leave available=2, one accepted request outcome for 3, one reservation and one outbox message identity.
A repeated delivery of IDEM-A should return its recorded outcome without reducing stock again or creating a second confirmation intent.
A different request IDEM-B asking for 3 should be rejected while only 2 remain. A duplicate of IDEM-B should follow the explicitly chosen rejection-replay policy rather than unexpectedly becoming accepted later.
These example quantities do not rewrite the canonical agreed orders or explain the earlier unknown stock discrepancy.

26.1.4 PLAIN — what is really happening inside#

The protocol connects request identity to the state transition it authorizes. Unique constraints help prevent two concurrent records claiming the same identity, while a transaction groups the related local writes.
The code must handle accepted, rejected and exceptional outcomes distinctly. A deliberate business rejection can be recorded; an unexpected exception should not accidentally create a successful-looking outcome.
Cancellation, replenishment and correction need their own rules. Adding idempotency to one endpoint does not prove that every other mutation preserves the same invariant.

26.1.5 TECHNICAL — the engineer’s version#

Define the operation as a state transition over (stock, requests, reservations, outbox). State the preconditions and which fields may change for accepted, rejected and replayed operations.
In the educational implementation, a single owned SQLite write transaction coordinates the guarded stock update and the relevant request/outbox records. SQLite writer serialization is part of this local implementation, not a claim about a distributed service. [S25] [S54]
Add reconciliation identities, such as initial stock plus replenishments minus accepted reservations plus valid releases equals expected available stock, under explicitly bounded record classes. Unrecorded physical loss remains outside that equation’s evidence.

26.1.6 WORDS — remember these#

Invariant: a rule accepted operations must preserve — a predicate over the relevant system state that the protocol is designed to maintain. State transition: one defined move from an old state to a new state — an operation with explicit preconditions, writes and outcomes. Reconciliation identity: an equation connecting related records — a check that quantities and recorded state changes agree under stated inclusions and exclusions.

26.2 Request identity#

26.2.1 PLAIN — in simple words#

A retry needs a stable name for the original intent. Without that name, the server may be unable to distinguish “try that same reservation again” from “make another identical reservation.”
Two equal payloads are not automatically the same intent. A customer can legitimately place two separate orders for the same product and quantity.
Scope the name. The same local request string used by two different tenants should not make their operations collide or reveal one tenant’s outcome to another.

26.2.2 PLAIN — a picture in your head#

A customer receives a claim ticket for one request. Presenting the same ticket asks about the same decision; receiving a new ticket creates a separately identified request.
The ticket belongs to a particular desk and customer scope. Ticket 17 at another shop is not necessarily your ticket 17.
Where the comparison breaks: a request key is not automatically a secret or proof of ownership. Authorization must be established independently before the server reveals or changes anything associated with it.

26.2.3 PLAIN — a worked example#

Tenant T-A submits key REQ-17 for branch BR-A, product P-DEMO and quantity 3. A network retry repeats the same scoped key and the same validated meaning.
A new purchase with the same branch, product and quantity uses a new intent identity, such as REQ-18. Deduplicating solely by a hash of the payload would wrongly collapse these legitimate separate purchases.
Tenant T-B can independently use REQ-17 under a key scoped by tenant. A request outcome lookup must include the tenant established by the trusted service boundary, not merely a client-supplied string.
The companion’s local function accepts a trusted tenant parameter for teaching; it does not implement that authentication boundary.

26.2.4 PLAIN — what is really happening inside#

Validate the input into a well-defined semantic representation before comparing attempts. Decide whether whitespace, field order or equivalent numeric spelling changes meaning under the API contract.
Store enough validated request meaning to detect incompatible reuse. A digest can help indexing or comparison, but a collision-prone shortcut must not replace the equivalence rule.
The server must define who generates keys, their allowed length, their retention and whether a key can ever be reused. An unlimited arbitrary string is also an input-resource concern.

26.2.5 TECHNICAL — the engineer’s version#

Model identity as a tuple such as (tenant_id, operation_kind, request_key). Include additional scope only when required by the contract, and enforce the tuple’s uniqueness in storage.
Idempotent APIs commonly distinguish caller-provided request intent from merely equal parameters. The equivalence and key-retention contract is central to safe retries. [S126]
Canonicalization is not generic “sort JSON and hash.” It must preserve domain meaning, reject ambiguous duplicate members where relevant, represent units explicitly and avoid silently equating values that the application treats differently. Chapters 3, 5 and 7 supply the prerequisites.

26.2.6 WORDS — remember these#

Request identity: the stable name of one intended operation — a scoped identifier linking repeated attempts to one business decision. Payload equivalence: when two attempts mean the same thing — an explicit comparison contract over validated request semantics. Identity scope: the domain in which a key is unique — the tenant, operation and other components defining a request key’s namespace.

26.3 Replay detection#

26.3.1 PLAIN — in simple words#

When a known request returns, the server checks whether it represents the same intent and then returns the recorded outcome. It does not perform the business change again merely because the caller asked again.
Reusing the same key with different meaning is a conflict, not a request to rewrite history. The server should reject the incompatible reuse under a clear contract.
Decide what happens to recorded rejections. In this book’s local protocol, a valid business rejection is stable for that request identity. A later change in stock does not silently change the old decision.

26.3.2 PLAIN — a picture in your head#

A reservation desk keeps the decision attached to the claim ticket. Returning with the ticket retrieves that decision.
Changing the requested quantity while presenting the same ticket is not merely asking again. It is attempting to attach a different request to an existing identity.
Where the comparison breaks: some real APIs deliberately choose other policies, including retention limits or selected retryable rejections. Those choices must be documented; this chapter’s policy is not a universal standard.

26.3.3 PLAIN — a worked example#

IDEM-A with quantity 3 is accepted from stock 5, leaving 2. Replaying IDEM-A with quantity 3 returns the same accepted outcome and leaves stock 2.
IDEM-A with quantity 4 conflicts with the stored request meaning. It must neither allocate a fourth unit nor overwrite the original record.
IDEM-B with quantity 3 is rejected for insufficient stock. Suppose a separate replenishment later raises available stock to 7. Replaying IDEM-B still returns its original rejection under the chosen contract.
A caller who now intends a new allocation submits a new request identity. That is an explicit new decision, not an automatic retry that changes the meaning of the old outcome.

26.3.4 PLAIN — what is really happening inside#

Replay detection must occur within a protocol that coordinates competing first attempts. A lookup followed by an unprotected insertion has the same check-then-insert race discussed in Chapter 22.
Storing an outcome separately after the stock commit leaves a gap: a replay could see no outcome even though stock was already reduced. Group the outcome and business mutation atomically.
Do not return another scope’s result to resolve a conflict. Identity comparison, authorization and response filtering all remain relevant on the replay path.

26.3.5 TECHNICAL — the engineer’s version#

The local example persists both accepted and valid business-rejected outcomes, with normalized payload fields. A unique request key and an owned transaction prevent inconsistent duplicate outcome records within the tested database protocol.
Unexpected exceptions roll back that attempt’s local changes. They are not converted into stable success or an invented business rejection. An externally unknown commit outcome is resolved by querying the same request identity.
Retention bounds the deduplication horizon. Deleting request records while late retries remain possible can allow a formerly recognized intent to be treated as new. State that limit explicitly. [S126]

26.3.6 WORDS — remember these#

Replay: another attempt carrying the same defined intent — a repeated request that should resolve through its existing identity and outcome. Incompatible key reuse: one identity is presented with different meaning — a conflict requiring rejection or explicit reconciliation, not silent replacement. Deduplication horizon: how long repeated intent remains recognizable — the retention interval and protocol conditions under which replay records are available.

26.4 Atomic intent recording#

26.4.1 PLAIN — in simple words#

The reservation, its request outcome and its required message intent should be accepted together when the protocol requires all three.
This does not mean the message is already delivered. It means that a committed reservation has a durable local record of the delivery work that follows from it.
The order of application steps matters. A failure at any pre-commit point must not leave an accepted half-operation in the database.

26.4.2 PLAIN — a picture in your head#

Mira accepts a reservation only when its stock entry, decision slip and dispatch instruction are filed as one case.
The messenger may arrive later, but the instruction no longer depends on Mira remembering it after a crash or interruption.
Where the comparison breaks: an in-memory educational database does not demonstrate physical durability. The protocol’s atomic grouping can be tested locally while durable storage assumptions remain a separate chapter’s responsibility.

26.4.3 PLAIN — a worked example#

Begin an owned transaction. Look up the scoped request key. If it exists, compare meaning and return its recorded outcome without another allocation.
For a new valid request, attempt the guarded stock reduction. If accepted, insert the reservation, accepted outcome and one outbox row. If rejected under the business contract, insert the stable rejected outcome without a reservation or accepted-confirmation intent.
Commit only after every required local write succeeds. Inject an exception after stock reduction and verify that rollback removes all changes from that attempt.
The transaction is not allowed to call an email provider before commit. It records what should be sent, not an external send that rollback cannot undo.

26.4.4 PLAIN — what is really happening inside#

An outbox record needs a stable message identity, type, relevant payload or durable reference, and enough state to manage delivery attempts under the chosen design.
Payload design has retention consequences. Copying an entire customer profile into every message makes later correction or deletion more complicated than sending a bounded required subset.
A worker that reads the outbox also needs a coordination protocol if several workers can claim the same work. This book’s simplest local dispatcher is deliberately single-worker; it does not imply a production lease implementation.

26.4.5 TECHNICAL — the engineer’s version#

The transactional outbox places business state and delivery intent in the same local atomic boundary, avoiding the basic dual-write gap between a database commit and an unrelated message send. [S125]
Use constraints to connect outbox identity and request identity as required. Do not regenerate a new message identifier on every delivery attempt; that defeats consumer deduplication by message identity.
For multiple dispatchers, specify claiming, lease expiry, crash recovery, ordering and fencing where needed. Merely adding a sending=true flag can strand work permanently after a worker dies.

26.4.6 WORDS — remember these#

Atomic intent recording: accept the business change and its follow-up instruction together — committing state and outbox intent in one local transaction. Dual-write gap: one system changes while another does not — a failure window between separately committed operations in different resources. Dispatcher: the worker attempting recorded external tasks — a process that reads delivery intent and manages sends and acknowledgements.

26.5 Outbox delivery and consumers#

26.5.1 PLAIN — in simple words#

A dispatcher can send the same message more than once if it loses certainty about a previous attempt. The receiver must know what repeated delivery means.
A database-backed consumer can record “I processed message M” together with its local database effect. Replaying M then finds the record and avoids applying that effect again.
This protects the consumer’s participating local database work. It does not make a separate external effect, such as sending a physical parcel, part of that database transaction.

26.5.2 PLAIN — a picture in your head#

The receiving desk stamps a message identity into a register at the same time it files the corresponding local instruction. A second copy with the same identity finds the stamp.
If the desk crashes before accepting either, it can safely try again. If it accepted both, it can recognize the replay.
Where the comparison breaks: a stamp made before an external action can falsely suggest completion, while a stamp made after it can leave a duplication gap. The atomic boundary must contain the actual effect you claim to deduplicate.

26.5.3 PLAIN — a worked example#

Outbox message MSG-A represents IDEM-A’s accepted reservation. The dispatcher sends it, and the consumer commits an inbox record plus a local notification-task record.
Before the dispatcher marks delivery complete, it crashes. On restart it sends MSG-A again.
The consumer finds MSG-A in its inbox with equivalent meaning and returns the recorded result without creating another notification task. There are two deliveries but one committed local task.
If that task later sends an email, email delivery remains another boundary. The local inbox result does not prove that exactly one email reached a person.

26.5.4 PLAIN — what is really happening inside#

Inbox identity and effect must share the consumer’s transaction. Marking a message processed first and performing the effect later can lose work; performing the effect first and marking later can duplicate it.
A duplicate identity carrying different payload is suspicious or incompatible under the protocol. Do not silently treat it as an equivalent replay simply because the identifier matches.
Order is separate from duplication. A consumer may need per-order sequence checks so “cancel reservation” does not apply before the reservation it references. A global message ID alone does not establish causal order.

26.5.5 TECHNICAL — the engineer’s version#

Consumer-side deduplication typically uses a unique inbox key and a transaction containing both the inbox decision and the local effect. Its horizon is bounded by retained identity records and participating storage guarantees. [S125]
An acknowledgement should correspond to the consumer’s defined accepted boundary. A queue acknowledgement sent before the local effect commits can lose work if the consumer then fails.
Model delivery as at-least-once attempts with possible unknown outcomes unless the actual infrastructure contract establishes something else. “Exactly once” must name the effect, identity scope, storage boundary and retention interval it describes.

26.5.6 WORDS — remember these#

Inbox record: remember which message identity was accepted — a consumer-side deduplication record committed with the participating local effect. At-least-once delivery: retries may deliver a message repeatedly — a delivery contract that prioritizes eventual attempts while permitting duplicates under its stated assumptions. Acknowledgement boundary: what a positive receipt actually confirms — the defined acceptance point represented by a queue, consumer or provider response.

26.6 End-to-end limits#

26.6.1 PLAIN — in simple words#

The complete journey contains several boundaries: caller intent, database acceptance, outbox dispatch, consumer acceptance and any later real-world action.
A local guarantee should be described at its own boundary. Extending it verbally to the whole journey does not make the missing coordination appear.
Strong engineering names the remaining uncertainty and provides recovery procedures. It does not hide uncertainty behind a confident success label.

26.6.2 PLAIN — a picture in your head#

A parcel has separate records for booking, warehouse acceptance, courier collection and recipient delivery. A booking receipt is useful, but it is not proof that the parcel reached its destination.
Each handover needs a defined meaning and a way to investigate missing evidence.
Where the comparison breaks: this book’s reservation lab does not implement a logistics system. The parcel story illustrates boundaries, not a tested physical delivery protocol.

26.6.3 PLAIN — a worked example#

IDEM-A commits locally, but the response is lost. The caller retries the same identity and learns the accepted outcome. No second allocation occurs under the tested local protocol.
MSG-A is delivered twice, but the consumer’s inbox and local task transaction apply one local task. The dispatcher eventually records its acknowledgement.
A later email provider returns an ambiguous timeout. Without a suitable provider identity contract or a reconciliation interface, the system cannot infer from that timeout whether the email was accepted.
The honest status separates “reservation accepted,” “notification task recorded,” “provider outcome unresolved” and “delivery verified,” rather than collapsing all four into one invented certainty.

26.6.4 PLAIN — what is really happening inside#

Retention, permissions and failure domains constrain the protocol. Erasing an inbox record can remove replay protection; losing both the business database and its only backup can remove outcome evidence.
Replays must remain authorized. Knowing an old request key does not grant permanent access after a user’s rights change.
Operational recovery needs bounded procedures: identify stuck intents, verify payload identity, inspect consumer state, decide safe replay scope and preserve the evidence of the decision.

26.6.5 TECHNICAL — the engineer’s version#

The supplied implementation is a local educational state machine. It uses synthetic records and trusted function parameters; it is not a multi-process lease service, authenticated API, immutable audit store or proof of durable power-loss behaviour.
Test invariants after every injected failure point, replay and incompatible key reuse. Include stable rejection replay, inbox payload mismatch, tenant scope and stock reconciliation. A passing test means its stated behaviour occurred, not that unseen external systems were verified.
Before production, separately review transaction isolation, key retention, worker coordination, external provider contracts, privacy, monitoring and restore evidence. Idempotency is a protocol property with assumptions, not a decorative endpoint label. [S126]

26.6.6 WORDS — remember these#

End-to-end boundary: the complete chain whose result is claimed — all participating systems and effects required for a stated business outcome. Outcome reconciliation: resolve uncertainty using recorded identity and state — a procedure for determining what a prior attempt actually accepted. Scoped guarantee: a promise with explicit limits — a correctness claim tied to defined effects, participants, failures and retention assumptions.

26.97 Practice and worked answers#

Name the invariant. Stock falls from 5 to 2, but no reservation exists. Answer: non-negative stock holds, but the required allocation-to-reservation relationship fails. One invariant does not imply the other.
Separate equal requests. A customer intentionally buys the same three pens twice. Answer: equal payloads can represent two legitimate intents. Give the intents separate request identities.
Replay acceptance. IDEM-A for 3 is accepted once and delivered again. Answer: return its stored outcome without a second subtraction or a new outbox identity.
Reject incompatible reuse. IDEM-A returns with quantity 4. Answer: reject the mismatched meaning under the protocol; do not replace the original request record.
Replay rejection. IDEM-B was rejected, then stock was replenished. Answer: under this book’s explicit policy, the old identity retains its rejection. A genuinely new allocation request uses a new intent identity.
Inject a local failure. An exception occurs after stock reduction but before outbox insertion. Answer: the owned transaction rolls back all of that attempt’s local changes.
Trace duplicate delivery. MSG-A is sent twice after a dispatcher crash. Answer: an equivalent inbox identity and local effect in one consumer transaction can preserve one local task despite two deliveries.
Limit the claim. Does one local task prove exactly one delivered email? Answer: no. The provider and recipient boundaries require separate contracts and evidence.

26.98 Common wrong ideas#

Wrong: a transaction automatically knows the business invariant. Right: the application must define and protect the relevant relationships.
Wrong: equal payload hashes always mean one intent. Right: identical purchases can be legitimate separate requests.
Wrong: a request key is an authorization token. Right: scope and permission checks remain independent.
Wrong: a duplicate key with different data should overwrite the old request. Right: incompatible identity reuse requires an explicit conflict policy.
Wrong: every rejected request should become successful when retried later. Right: replay semantics must be stable under the chosen contract.
Wrong: an outbox means the message has been delivered. Right: it records durable local intent within the participating storage boundary.
Wrong: a consumer inbox deduplicates arbitrary outside effects. Right: it protects only effects inside the same proven atomic boundary.
Wrong: exactly once is meaningful without naming the effect and scope. Right: define identity, participants, failure assumptions and retention horizon.

26.99 Chapter summary in 20 lines#

Invariants define what accepted state transitions must preserve.
Related stock, reservation, outcome and message records need explicit relationships.
A retry must carry the stable identity of the original intent.
Equal payloads can represent different legitimate intents.
Scope request identity by the required tenant and operation namespace.
Validate and compare request meaning under a documented equivalence contract.
Matching identity with different meaning is a conflict, not an ordinary replay.
Store the business outcome with the local state change atomically.
A replay returns the recorded outcome without performing the allocation again.
This book’s valid business rejections remain stable for their original request identity.
Unexpected pre-commit exceptions roll back the attempt’s local changes.
A lost response after commit can be resolved through the same request identity.
An outbox records external work implied by an accepted local change.
Outbox intent is not evidence of completed external delivery.
Dispatch can duplicate messages after an ambiguous send or lost acknowledgement.
An inbox can deduplicate participating local consumer effects in one transaction.
Message identity does not by itself guarantee causal ordering.
Retention bounds how long replay protection and outcome evidence remain available.
Authorization and privacy checks apply to replays as well as first attempts.
State every guarantee at its actual boundary and preserve unresolved outcomes honestly.

Return to contents