From Events to State: Keeping History
Introductions, exercises and summaries stay visible.
6.0 What this chapter gives you#
- You will distinguish a record of something happening from a value describing the current position.
- You will reconstruct a notebook stock figure without treating a physical count as a movement of goods.
- You will read a state-transition rule and explain why some requests must be rejected.
- You will separate recorded order, physical time, delivery order and causal order.
- You will understand what a snapshot saves, what a projection computes and what neither proves.
- You will correct an earlier record without pretending that an external action has been undone.
- You will describe the limits of reconstruction when records, interpretation rules or supporting evidence are no longer available.
The stock stream in this chapter is a synthetic continuation of
Mira’s Corner. The order O-1042 still contains two
notebooks and one pen, for INR 171.00. No price or quantity from that
order is changed. A later count and an explicitly authorised stock
adjustment are new teaching events. The separate order-state and
reversal exercises use different identifiers so their histories cannot
be mistaken for the original order.
6.1 An event versus a current value#
6.1.1 PLAIN — in simple words#
- A current value answers a question such as “how many notebooks does our stock record say we have now?” An event answers a different question: “what did we record as happening?”
- “Ten notebooks” is a position. “Two notebooks were issued against this order” is a change that helps explain a position.
- The current position alone does not tell us how we arrived there. Ten could result from twelve minus two, eight plus two, or a much longer history.
- Keeping a history can make an answer explainable. It does not automatically make every entry in that history true. A mistaken count remains a mistaken count even when stored carefully. [S05]
- We must also distinguish an observation from a movement. Looking at nine notebooks on a shelf does not itself remove one notebook from a record that expected ten.
- A useful system can show both: “expected stock: ten” and “last observed count: nine”. It can then show the unresolved difference without inventing a reason for it.
6.1.2 PLAIN — a picture in your head#
- Imagine a water tank with a level indicator and a notebook beside it. The indicator says how much water appears to be there. The notebook records deliveries, withdrawals and inspections.
- A delivery entry explains why the expected level rose. A withdrawal explains why it fell. A person reading the indicator makes an observation, not a delivery or withdrawal.
- If the indicator and the notebook disagree, writing “leak” in the book is not a diagnosis. A leak is one possible explanation, but so are a bad reading and a missed entry.
- Where this comparison breaks: goods are often counted as separate units, while water is measured continuously. Our notebook example also assumes that “one notebook” means one item of the same product and stock location, not a pack, a reservation or an item in another branch.
- The analogy helps separate movement from observation. It does not tell us which stock policy the shop should adopt after a discrepancy.
6.1.3 PLAIN — a worked example#
- The stream identifier is
STOCK-P-NOTE. It describes the expected physical stock of productP-NOTEat the one shop location in this example. It is not an “available to promise” quantity with reservations subtracted. - Here are the first three entries. Sequence means accepted position in this one stream.
| Sequence | Recorded event | Effect on expected stock | Position afterwards |
|---|---|---|---|
| 1 | Opening balance: 12 notebooks | Establish 12 | 12 |
| 2 | Goods issued: 2, for O-1042 line 1 | Subtract 2 | 10 |
| 3 | Physical count observed: 9 | No movement | 10 |
- After sequence 3, the latest observed count is nine, the expected
position is ten, and the discrepancy is
9 - 10 = -1notebook. - We know that the two figures differ. We do not know from these three entries whether an item was lost, miscounted, issued without an entry, or assigned to the wrong location.
- In this teaching continuation, Mira reviews the discrepancy and explicitly authorises an adjustment of minus one. That decision becomes sequence 4 and refers to observation 3.
- The expected position becomes nine because of the authorised adjustment, not because the software silently treated every observation as a movement.
- A later receipt of five notebooks, with reference
R-500, becomes sequence 5. The expected position becomes fourteen.
opening balance 12
issue for O-1042 line 1 -2
count observation of 9 0 (no movement)
authorised adjustment linked to count 3 -1
receipt R-500 +5
--
expected position after sequence 5 14
Figure 6.1. Expected stock changes only under the declared movement and adjustment rules; the observation is separate evidence.
6.1.4 PLAIN — what is really happening inside#
- The system reads the event type before deciding how an entry affects the position. A number without its meaning is not enough.
- An opening entry creates the initial position. An issue reduces it. A receipt increases it. A count observation stores separate evidence. An authorised adjustment applies an explicit change.
- The calculation starts from a stated boundary and applies the appropriate rule to each accepted entry. The answer depends on both the entries and those rules.
- The program also tracks the last sequence processed. “Fourteen notebooks through sequence 5” is a stronger description than an unexplained “fourteen”. It tells us the boundary of included history.
- A person may still dispute the history. Linking the adjustment to the observation allows a reviewer to follow the decision; it does not prove that the observation was accurate or that every required approval occurred outside this teaching example.
- The honest version: a record can preserve what the system accepted and why it produced an answer. It cannot travel back in time and directly inspect the shelf.
6.1.5 TECHNICAL — the engineer’s version#
- A state is the information a model holds at a
particular logical boundary. An event records an
occurrence relevant to that model. A transition function can be written
next_state = apply(previous_state, event). - A projection folds a sequence of events into a derived
representation. For an ordered sequence
e1 ... en, the state isapply(...apply(apply(initial, e1), e2)..., en). That notation is repeated application, not a requirement to hold the entire history in memory. [S40] - Event sourcing is a particular architectural choice in which an event history is the authoritative record used to reconstruct application state. A system having an audit table is not, for that reason alone, event sourced. The relationship between the event store and any current-state tables must be specified. [S40]
- Our stock projection has an explicit schema: expected quantity, latest observation and its sequence, processed sequence, and stream identifier. The teaching implementation rejects unknown event types instead of guessing that every numeric payload is a delta.
- Opening balance and subsequent movements are different operations. Replaying an opening-balance event in the middle of an established stream would replace rather than explain the position; our lab rejects that malformed sequence.
- An invariant in this bounded model is that expected physical stock must not become negative. That is a chosen rule for this fixture, not a universal database rule or a claim that every inventory application must reject negative book stock.
- Retain provenance for important derived statements: input identifiers, transformation identity and the boundary of included input. W3C PROV provides a vocabulary for entities, activities and derivations, rather than an automatic guarantee of truth. [S05]
- The lab tests the transition rules in memory. It does not implement a durable event store, multi-process concurrency, crash recovery or production access control. Those require additional mechanisms introduced later in the book.
6.1.6 WORDS — remember these#
State: the position the model currently holds — a representation at a defined logical boundary.
Event: a record of something relevant happening — an occurrence interpreted under a stated event schema.
Projection: an answer built from other records — a derived representation produced by applying transformation rules to source data.
Event sourcing: keeping the accepted event history as the principal record — an architecture that reconstructs state from authoritative events.
Observation: something recorded as seen or measured — evidence that need not itself authorise or cause a state-changing business action.
Stock adjustment: an explicit change to the expected stock record — a domain operation distinct from a goods movement or a count observation.
6.2 State transitions#
6.2.1 PLAIN — in simple words#
- Not every requested change makes sense from every starting point. A draft order can be confirmed. An order already fulfilled cannot simply become an untouched draft again.
- A state-transition rule says which change is allowed, from which starting state, and what the resulting state will be.
- A request is not the same as an accepted event. “Please cancel” describes an intention. “Cancellation accepted” says that the system checked a request and accepted a change.
- A rejection is also useful information, but it must not be mistaken for the successful action the requester wanted.
- Two people can act on the same old view. The system needs to notice that one person’s accepted change may make the other person’s request stale.
- These rules are choices made for the work being modelled. A database cannot invent the shop’s cancellation policy merely by knowing what a table is. [S02]
6.2.2 PLAIN — a picture in your head#
- Picture a parcel moving through labelled trays: “preparing”, “ready to send”, and “handed to carrier”. Moving it between trays is allowed only through specified steps.
- A clerk who still sees an old list saying “preparing” must not erase the carrier handover just by placing a cancellation slip in the first tray.
- The clerk needs the current position and a rule for what a cancellation means at that position. It might be a rejection or a different process, such as a return request.
- Where this comparison breaks: physical trays encourage us to imagine one visible location. In software, several screens can display copies with different ages. A page that looks current is not proof that no accepted change has happened since it loaded.
- Nor does a tray enforce the business rule by itself. We still need an authority that checks and records the move.
6.2.3 PLAIN — a worked example#
- Use a separate fictional order,
P-3001. These rules do not assert anything new about the fulfilment status ofO-1042. - We choose four states:
DRAFT,CONFIRMED,FULFILLEDandCANCELLED.
| Current state | Request | Result in this teaching policy |
|---|---|---|
| DRAFT | Confirm | CONFIRMED |
| DRAFT | Cancel | CANCELLED |
| CONFIRMED | Fulfil | FULFILLED |
| CONFIRMED | Cancel | CANCELLED |
| FULFILLED | Cancel | Reject; a different returns process would be needed |
| CANCELLED | Fulfil | Reject |
- Mira and Dev both load
P-3001at version 2, in stateCONFIRMED. Mira requests fulfilment and Dev requests cancellation. - Suppose the system accepts Mira’s request first. It records fulfilment and advances the version from 2 to 3.
- Dev’s request says that it was based on version 2. The system rejects it as stale rather than silently overwriting version 3.
- On refresh, the current state is
FULFILLED. Under our chosen policy, cancellation is not an allowed transition from that state. - Retrying does not mean repeatedly forcing the same desired result. It means reading the new state and re-evaluating whether the requested action is still valid.
6.2.4 PLAIN — what is really happening inside#
- The request includes an identity for the object being changed and the version on which the requester based the decision.
- The system checks the current version and the transition rule together with accepting the new state. Separating the check from the acceptance leaves a gap in which another change could win.
- A rejected request must not increase the version as though a successful transition occurred. Nor should a failed attempt send a success notification.
- We can preserve a record of the failed attempt where appropriate, but it belongs to a different kind of record from the accepted order event.
- The version number is not a timestamp. Version 3 means a position after version 2 in this object’s accepted history; it does not mean three seconds have passed.
- The honest version: checking versions in a single Python function demonstrates the idea. It does not make a shared production system safe unless the storage operation enforces the check under real concurrent access.
6.2.5 TECHNICAL — the engineer’s version#
- A finite state machine specifies a set of states and permitted transitions. A command is evaluated against a state; a successful command may produce one or more domain events. Keep command identity, delivery identity and event identity distinct. [S39] [S40]
- Optimistic concurrency attaches an expected version to a write. The storage boundary must condition acceptance on the expected version still matching. In a relational system, related checks and updates belong in an appropriate transaction with suitable constraints or locking. A transaction boundary alone does not choose the isolation behaviour needed for every multi-row invariant. [S01] [S02]
- The lab’s
transition_order(state, version, expected_version, action)is a pure educational check. It returns a new state and version or raises a validation error. It never contacts a payment provider or changes the actual shop. - A repeated request after a lost reply creates a second problem: the client may not know that its first request succeeded. Request-level idempotency can let it retrieve the earlier outcome, but only within the stated identity and retention policy. Expected-version checking by itself is not a complete idempotency design. [S40]
- Domain validation is different from authorisation. “Cancellation is allowed from CONFIRMED” does not mean any caller may cancel any confirmed order. The production operation must also check the caller’s authority at the appropriate boundary.
- An accepted transition can have effects outside one database. An order-state transaction does not atomically recall a parcel already handed to a courier. Separate external effects, retries and compensations must be modelled explicitly. [S40]
- Test forbidden edges as well as allowed ones. Include unknown states, unknown actions, mismatched versions and a rejected action that leaves the original state unchanged. These are assertions about this chosen transition graph, not exhaustive proof of a complete commerce system.
6.2.6 WORDS — remember these#
Command: a request to do something — an intention evaluated against current state and applicable rules.
State transition: an allowed move between positions — a specified change from one model state to another.
Expected version: the history position a request relies on — a concurrency precondition used to detect stale writes.
Stale request: a request based on an older position — an operation whose assumed version no longer matches the accepted state.
Invariant: a rule that must remain true — a property the chosen model preserves across accepted operations.
Authorisation: permission to perform an action — an access decision separate from whether the action is a valid domain transition.
6.3 Ordering and causality#
6.3.1 PLAIN — in simple words#
- “Which came first?” can ask several different questions. Which event happened first? Which was written first? Which message arrived first? Which action depended on another?
- These orders can disagree. A slow connection can deliver an earlier event after a later one.
- Two devices can also have clocks that disagree. A printed time is a useful observation, but it is not automatically a trustworthy instruction about causal order.
- If one action sends a message and another action receives that same message, the send comes before that receive in the communication relationship. The receive cannot be the cause of the send it receives. [S41]
- An accepted sequence number can define the replay order for one stock stream. It does not automatically tell us the order of every action in every shop.
- Before sorting a history, name the question the sort is supposed to answer. Sorting by the easiest available column may produce a tidy but misleading story.
6.3.2 PLAIN — a picture in your head#
- Imagine two clerks posting letters. Each writes the time shown on their own desk clock. One clock is five minutes fast.
- A reply depends on the original letter arriving, even if the timestamps make the reply look earlier. The communication relationship is evidence that the clock labels alone are not the whole story.
- Now consider two unrelated letters sent to different people. Sorting them by envelope colour gives an order, but tells us nothing about whether one caused the other.
- Where this comparison breaks: software can exchange enormous numbers of messages with retries, relays and duplicates. Establishing that a receive corresponds to a particular send requires identifiers and protocol evidence, not a human guess from similar wording.
- The analogy separates an ordering convention from a causal claim. It does not provide a universal clock.
6.3.3 PLAIN — a worked example#
- Our stock projection has processed sequences 1 and 2 and holds expected stock ten. It next receives sequence 4, the authorised adjustment.
- The strict replay rule in this lab requires sequence 3 next. It rejects sequence 4 as a gap rather than pretending that the preceding history is complete.
- A real ingestion service might buffer 4 until 3 arrives or fetch the missing entry. Our small lab does neither: rejection makes the missing prerequisite visible.
- Once 3 is available, applying 3, 4 and 5 in sequence gives expected positions ten, nine and fourteen.
- Here is a separate logical-clock exercise. A logical counter is a number used to preserve communication order; it is not a time of day.
Process A starts with counter 0.
A performs an event: counter becomes 1.
A sends a message: counter becomes 2; message carries 2.
Process B already has counter 5.
B receives that message: counter becomes max(5, 2) + 1 = 6.
B performs its next event: counter becomes 7.- The send is labelled 2 and its receive 6, preserving their order. Those four steps between 2 and 6 are not four seconds, and they do not count four intervening events in the whole system.
- An unrelated process could also have an event labelled 6. Equal logical values do not prove the events happened at the same physical instant.
6.3.4 PLAIN — what is really happening inside#
- The stock stream uses one explicit sequence for interpretation. An arrival with a missing predecessor is held outside the accepted projection boundary in our lab.
- An event timestamp can still be stored for reporting. It must not silently replace the sequence rule when clocks disagree or old events arrive late.
- A logical clock advances locally and incorporates values carried by messages. That gives it information about communication dependencies without requiring perfectly synchronised wall clocks. [S41]
- The clock provides a useful guarantee in one direction: if an event causally precedes another under the model, the first receives a smaller logical value. A smaller value alone does not establish a causal link. [S41]
- A system may break ties using a process identifier to produce a deterministic total order. That tie-break is a chosen order, not a newly discovered fact about the physical world.
- The honest version: the identifiers and clocks only describe the events the system models. A phone call or a human action outside the recorded communication can matter without appearing in that model.
6.3.5 TECHNICAL — the engineer’s version#
- Lamport’s happened-before relation includes local process order,
message send before its receive, and transitive consequences. It is a
partial order: some distinct events are not ordered by it. Logical
clocks can satisfy
a -> bimpliesC(a) < C(b); the reverse implication is not generally valid. [S41] - The ordinary scalar-clock update is local increment for an event
and, at a receive,
C = max(C, received_C) + 1. A deterministic process-ID tie-break can extend clock order to a total order without proving additional causal relationships. [S41] - Distinguish event time, recording time, delivery time and stream sequence. Chapter 3 introduced representations of instants; this chapter explains why a correctly formatted timestamp still does not define every required processing order. [S09] [S40]
- Our projector requires an exact next sequence within an exact stream. It rejects duplicates, gaps and unexpected stream identifiers. Duplicate transport delivery is handled before this strict fold; silently applying it twice would corrupt state. [S39] [S40]
- Ingesting several streams does not create a globally agreed sequence just because each stream is internally ordered. Cross-stream invariants need a separate coordination model, addressed in the later parts on transactions and distributed systems.
- Integer deltas can commute in an unconstrained arithmetic sum, but admission rules may not. Starting at zero, receiving one and then issuing one passes a non-negative-stock rule; issuing before receiving fails it. Final arithmetic equality does not erase the intermediate rule violation.
- The lab demonstrates that difference with a small sequence and checks that a gap raises an error. It does not simulate network consensus or promise a global real-time order.
6.3.6 WORDS — remember these#
Stream sequence: a record’s position in one accepted history — a scoped ordering value used for interpretation or concurrency checks.
Causal order: one event preceding another through a dependency — the order induced by the model’s local actions and communications.
Logical clock: a counter that helps preserve event order — a timestamping mechanism that need not represent physical time.
Partial order: an order that does not compare every pair — a relation under which some distinct events remain unordered.
Sequence gap: an expected predecessor is missing — a discontinuity that prevents this strict projector from claiming a complete prefix.
Commutative operation: an operation whose result is unchanged by swapping order — a property that does not automatically extend to validation rules or external effects.
6.4 Snapshots and projections#
6.4.1 PLAIN — in simple words#
- Re-reading a long history from the very beginning can cost time. A snapshot is a saved position at a known point in that history.
- To continue, we load the snapshot and read only the later events. That works only when the snapshot belongs to the correct stream and interpretation rules.
- A snapshot without its boundary is like an answer without the question. We need to know which entries it already includes so we do not skip or repeat them.
- A projection is the answer produced for a particular purpose. The same accepted events can support a stock balance, a daily movement report and a list of unresolved count discrepancies.
- Those answers need not update at the same instant. A report can lag behind the accepted history, so it should make its processed boundary visible where that matters. [S40]
- A snapshot and a backup are not interchangeable words. A snapshot used to accelerate one calculation may be stored on the same machine and fail with the rest of that machine.
6.4.2 PLAIN — a picture in your head#
- Imagine reading a long account book and writing a subtotal at the bottom of each page. Tomorrow you can begin from a checked subtotal rather than adding every old line again.
- “Subtotal after page 12” is useful. “Subtotal: 10” with no page number is dangerous because you do not know where to resume.
- A second clerk might make a different summary from the same pages, such as the number of deliveries rather than the number of items. Both are projections of the same records for different questions.
- Where this comparison breaks: a software interpretation can change between versions. A subtotal made by a buggy program can be reproduced exactly and still be wrong. Its existence is not a certificate of correctness.
- Nor is a subtotal a replacement for the original lines when a reviewer needs the details the subtotal discarded.
6.4.3 PLAIN — a worked example#
- Save a snapshot of
STOCK-P-NOTEimmediately after sequence 2. It says expected quantity ten, no count observation yet, and last processed sequence two.
{
"stream": "STOCK-P-NOTE",
"projection_version": "stock-v1",
"last_sequence": 2,
"expected_quantity": 10,
"last_observation": null
}- Resume with sequence 3. The expected position stays ten and the latest observation becomes nine.
- Apply sequence 4. The authorised adjustment reduces expected stock to nine. Apply sequence 5 and it rises to fourteen.
- Replaying all five entries from the beginning also gives fourteen, with the same last observation and sequence boundary. The companion lab compares the full resulting state, not just one coincidentally equal number.
- Now deliberately label the snapshot
STOCK-P-PENwhile keeping notebook events. The lab rejects it. A plausible number in a snapshot for the wrong product is not a valid starting point. - Deliberately change the projection version to an unknown value. The lab rejects that too; it does not assume an unfamiliar interpretation is compatible.
6.4.4 PLAIN — what is really happening inside#
- The snapshot contains both the derived state and metadata that says how to use it. Together they define a starting point for replay.
- The projector accepts only events beyond that boundary, in the required order. Reapplying sequence 2 would issue the same two notebooks twice if the duplicate were not rejected.
- A snapshot made while new entries are arriving must correspond to one coherent point. Copying the balance after event 5 but the checkpoint after event 2 would produce a mixed record that no valid replay created.
- A report’s checkpoint similarly needs to agree with the result it describes. A crash between updating a report and updating its checkpoint can cause a retry problem unless those writes are coordinated. [S40]
- Rebuilding a projection should calculate data, not casually repeat physical actions. Reading an old “goods issued” event must not send a second instruction to issue the goods again.
- The honest version: successful replay proves agreement between two calculations under the tested rules and inputs. It does not prove that the input history includes every real-world event.
6.4.5 TECHNICAL — the engineer’s version#
- A snapshot is an optimisation associated with a stream boundary and a projection schema or version. It is not necessarily the authoritative history and can often be discarded and rebuilt when the required source history is retained. That rebuilding condition matters. [S40]
- Deterministic replay requires that interpretation not depend on uncontrolled current values. Calling “today’s price”, the current exchange rate or a random generator inside the fold can change a past result. Store the relevant historical input or define a versioned, reproducible rule where appropriate. [S05] [S40]
- Our
stock-v1projection uses integers, explicit event types and a stated sequence. A snapshot serialises expected quantity, observation information and last sequence. The lab validates matching stream and version before continuing. - A materialised projection may be updated asynchronously. A checkpoint such as “processed through sequence 5” is a statement about incorporated input, not a wall-clock guarantee that every source anywhere has been observed. [S40]
- Persisting a projection and its consumer position requires an appropriate atomicity or idempotency strategy. In a single database, both can sometimes be committed in one transaction; across external systems that assumption must not be made without evidence. [S01] [S25] [S40]
- Side effects deserve their own boundary. Sending an email, dispatching a parcel or charging a payment cannot safely be treated as an ordinary pure replay step. Recording delivery intent and handling retries are later topics; this chapter’s fold deliberately performs none of those actions. [S40]
- Test full replay against snapshot replay, wrong-stream snapshots, unknown versions and a missing first suffix event. Those checks expose specific errors. They do not replace tests of persistent storage, concurrent snapshot creation or recovery after process failure.
6.4.6 WORDS — remember these#
Snapshot: a saved position at a known boundary — a serialised state with enough metadata to resume a compatible interpretation.
Checkpoint: how far processing has reached — a recorded boundary of incorporated source input.
Deterministic replay: the same retained inputs and rules give the same state — reconstruction without uncontrolled dependencies on current external values.
Materialised projection: an answer stored for later reading — a persisted derived view rather than a query result recomputed on every request.
Projection version: which interpretation produced the answer — an identifier for the relevant transformation or state representation.
Side effect: an action beyond calculating the returned value — an external change that must not accidentally be repeated during reconstruction.
6.5 Corrections and reversal events#
6.5.1 PLAIN — in simple words#
- Keeping history does not mean pretending mistakes never happen. It means deciding how a mistake will be corrected and how the correction will be explained.
- Overwriting an old entry can remove evidence of what the system previously believed. Adding a correction can preserve both the earlier record and the later repair.
- A reversal often applies an opposite effect to an earlier entry. That arithmetic can repair a recorded position, but it does not necessarily undo anything outside the record.
- If a parcel left the shop, writing an opposite quantity does not teleport it back. The physical return and the record of that return are separate matters.
- A correction needs a target. “Subtract one” without saying which mistake it repairs can make today’s total look right while leaving tomorrow’s reviewer unable to explain it.
- Different kinds of records need different correction policies. We should not impose a complicated event system on every small spreadsheet, or erase important history merely because editing a cell is easy. [S40]
6.5.2 PLAIN — a picture in your head#
- Imagine a teacher marking an exercise. The first mark was written beside the wrong answer. A later note says which mark was mistaken and gives the corrected result.
- Crossing out the first mark without a trace makes it harder to explain why the student saw a different score yesterday. Keeping both marks without indicating which one now applies is also confusing.
- The correction therefore has two jobs: make the current answer right and preserve an understandable relationship to the earlier answer.
- Where this comparison breaks: reversing an accounting entry or correcting an inventory issue can affect several related records, permissions and external processes. A note in the margin alone does not enforce any of those relationships.
- The analogy is about explanation, not a universal instruction to retain every detail forever.
6.5.3 PLAIN — a worked example#
- Use a separate teaching stream,
R-EX-01. Do not apply these entries toO-1042or to the five-event notebook stream above. - This stream opens at twelve notebooks. In the scenario, two notebooks were actually issued, but the operator mistakenly recorded an issue of three.
| Sequence | Entry | Recorded effect | Expected position |
|---|---|---|---|
| 1 | Opening balance | Establish 12 | 12 |
| 2 | Erroneous issue record | -3 | 9 |
| 3 | Reversal referring to entry 2 | +3 | 12 |
| 4 | Correct replacement issue | -2 | 10 |
- The combined correction is
+3 - 2 = +1. It takes the mistaken recorded position from nine to ten. - The physical issue remains two. We are repairing its representation, not asserting that three notebooks were physically returned and then two issued again.
- A report of accepted entries can show all four events. A report of the corrected issue should explain the reversal relationship so it does not count two unrelated real-world issues.
- A second reversal of entry 2 would add another three and corrupt the result. Our teaching lab rejects a second reversal of the same target.
- A reversal referring to an unknown or non-reversible entry is also rejected. The word “reversal” is not permission to add any convenient number.
6.5.4 PLAIN — what is really happening inside#
- The system locates the target event and checks that the chosen correction policy allows it to be reversed.
- It derives the reversal’s effect from that target instead of trusting a caller’s arbitrary positive quantity. In this example, reversing minus three produces plus three.
- It records the link between the reversal and the original. A later reader can inspect why the extra positive entry exists.
- It tracks whether the target was already reversed. Otherwise, a retry could apply the repair twice.
- A replacement entry expresses the corrected fact. Reversal and replacement may need to be accepted together when a temporarily reversed-only position would violate the intended operation.
- The honest version: our arithmetic example explains a correction model. Production acceptance must also address concurrent requests, transaction boundaries, permissions and the availability of evidence supporting the correction. [S01] [S40]
6.5.5 TECHNICAL — the engineer’s version#
- A compensating event changes subsequent interpretation without mutating the accepted earlier event. Compensation is domain-specific: not every effect has a simple mathematical inverse, and an inverse in a ledger does not imply reversal of an external action. [S40]
- The lab’s reversal fixture permits reversal only of a prior issue in the same fixture and only once. It computes the inverse of the recorded delta and links to the target sequence. It deliberately does not expose a general “reverse anything” API.
- The pure implementation can check these rules sequentially. In persistent storage, “not already reversed” must be enforced under concurrent access, for example through an appropriate unique relationship and transaction design. A preliminary application query alone leaves a race. [S01] [S02]
- Preserve the distinction between event validity at acceptance and later interpretation. An entry can be a faithful record of what the system accepted even after a later correction establishes that its business content was wrong.
- Queries need a declared perspective. “What did we record by yesterday?” and “What is our corrected account of yesterday’s activity using information available now?” can return different answers without either being a database error. Source and derivation boundaries must be explicit. [S05]
- A schema change can also affect how old events are read. A versioned reader or an explicit transformation may be needed; it should not silently reinterpret an old quantity’s unit or turn an optional field into a fabricated historical fact. [S05] [S40]
- Do not assume a sum is a sufficient audit. A wrongly duplicated issue and an unrelated positive adjustment can cancel numerically. Reconciliation should inspect identities, links and expected relationships as well as totals.
6.5.6 WORDS — remember these#
Reversal: an opposite recorded effect tied to an earlier entry — a constrained correction operation whose scope must be defined.
Compensation: a later action that addresses an earlier effect — a domain-specific response that need not restore the exact original world.
Replacement entry: the newly accepted corrected representation — a record distinguished from the earlier entry it supersedes or repairs.
Correction target: the particular entry being repaired — an explicit reference needed to validate and explain a correction.
Reconciliation: checking whether related records agree as expected — a comparison of identities, relationships and totals rather than a single balancing sum.
6.6 History with retention boundaries#
6.6.1 PLAIN — in simple words#
- “We keep history” needs a boundary. Which records are kept, for how long, for which purpose, and under whose control?
- An append-only rule means ordinary changes add records rather than editing earlier ones. It does not, by itself, mean that every detail must be retained forever.
- A history can contain unnecessary personal details or sensitive text. Adding more copies can increase the work of protecting and eventually removing those details.
- Reconstruction depends on what remains available. If old source events are removed, a retained snapshot may preserve a position while losing the ability to explain every earlier movement.
- That can be an intentional choice. The honest report says what was retained and what can no longer be reconstructed, rather than promising both complete deletion and unlimited recovery from the same removed evidence.
- This chapter describes engineering boundaries, not a legal retention schedule. Applicable obligations and permissions must be established separately for the actual system and jurisdiction. [S40]
6.6.2 PLAIN — a picture in your head#
- Imagine an archive that retains annual totals but disposes of some old working slips under its approved policy. A total can still be read, but the archivist cannot honestly promise to produce every disposed slip.
- A catalogue saying a slip once existed is not the slip itself. A checksum saying a retained file has not changed is not proof that the file originally contained a correct observation.
- The archive might keep different categories for different periods. That requires an inventory and a process, not a single switch labelled “history”.
- Where this comparison breaks: digital records can be copied into logs, exports, backups, caches and analytical systems. Removing a main entry does not automatically remove every copy or every derived identifier.
- The analogy is useful only if we remember that the archive has many rooms, not one cupboard.
6.6.3 PLAIN — a worked example#
- Suppose the teaching stock system retains a checked snapshot through
sequence 5: expected quantity fourteen, latest observation nine at
sequence 3, with interpretation
stock-v1. - Under a hypothetical retention decision, the detailed events 1–5 are no longer available to this reader. The snapshot is still present and later events remain readable.
- The reader can continue a compatible calculation from that boundary, provided the snapshot and its metadata are trustworthy enough for the intended use.
- The reader cannot replay the missing prefix from scratch or inspect the original wording of the authorised adjustment. That evidence is absent from the available package.
- A remaining event identifier or hash may help compare a future recovered copy. It cannot recreate the deleted content by itself.
- Write the limitation plainly: “Current state continues from snapshot through sequence 5; the underlying prefix is not available here. Full prefix reconstruction was not performed.”
- This is a hypothetical retention exercise. No source files, user records or published materials were deleted to demonstrate it.
6.6.4 PLAIN — what is really happening inside#
- The system separates data categories and their purposes. A minimal operational event need not repeat every detail of the customer profile that happened to be on screen when the action occurred.
- It records what a projection actually needs. Where an event refers to another record, we must decide what happens if that referenced detail changes or is no longer retained.
- A replay that depends on deleted external content may fail or produce a less detailed result. The system should not silently fill the gap with a current profile or a guessed value.
- Backups and exported copies need their own handling. A restored old copy can reintroduce information that a later process removed unless restoration accounts for subsequent actions.
- A retained audit statement can say that a deletion or expiry action was recorded. It should not overclaim that every possible copy on every system has been examined unless that was actually established.
- The honest version: recordkeeping, reproducibility and minimisation can pull in different directions. Good design makes the trade-off explicit and tests the promises it actually makes. [S05] [S40]
6.6.5 TECHNICAL — the engineer’s version#
- Event-store immutability is an application or storage property with a defined enforcement boundary. It is not an automatic retention policy and does not imply immunity to privileged changes, storage loss or deletion. [S40]
- Avoid putting unnecessary sensitive payloads into long-lived events. One design keeps an event’s minimal operational fact separate from mutable or separately governed attributes. That separation creates reference-resolution and replay questions that must be designed, not ignored. [S40]
- A reconstruction claim should identify input availability, schema versions, transformation versions, snapshots and the scope of the resulting output. A provenance model can describe those relationships, while verification remains an additional activity. [S05]
- If a projection was rebuilt from a retained snapshot rather than the full history, record that distinction. Equal current totals do not prove equality of all earlier event histories.
- A hash-based integrity check detects disagreement with a trusted reference value under its stated assumptions. It does not establish that the source was true, restore missing content, or prove that no unexamined copies remain.
- The lab tests a snapshot continuation and its limits as data structures. It does not implement retention enforcement, encryption-key destruction, backup expiry or a legal erasure workflow. Do not treat its successful tests as evidence that those unimplemented capabilities exist.
- Operationally, inventory the stores, define category-specific decisions, record authorised changes, test restoration and disclose gaps. The later chapters on retention and operations will develop that workflow; it is not complete merely because an event log exists.
6.6.6 WORDS — remember these#
Append-only: ordinary updates add rather than replace — a mutation policy whose enforcement and exceptions must be specified.
Retention boundary: which history remains available — a stated limit on kept records, detail, time or access.
Reconstruction boundary: how far an answer can be rebuilt — the scope supported by retained inputs, snapshots and interpretation rules.
Data minimisation: avoiding unnecessary stored detail — collecting and retaining only the information justified for the defined purpose.
Integrity check: a check that retained content agrees with a reference — evidence of consistency, not automatic evidence of real-world truth.
Restoration: bringing back a stored copy — a recovery operation that must account for the copy’s age and subsequent required changes.
6.97 Exercises with worked answers#
Exercise 1 | A count is not a movement#
The stream opens at 12, records an issue of 2, and then records a count observation of 9. What are expected stock, latest observed count and discrepancy?
Worked answer. Expected stock is
12 - 2 = 10. Latest observation is 9. Observed minus
expected is 9 - 10 = -1. No adjustment has yet been
authorised, so changing the expected balance to 9 would add a decision
not present in these three entries. The discrepancy does not identify
its cause.
Exercise 2 | Continue from a snapshot#
A compatible STOCK-P-NOTE snapshot through sequence 2
holds expected quantity 10. Sequence 3 observes 9, sequence 4 authorises
an adjustment of -1 linked to 3, and sequence 5 receives 5. What is the
result, and what must be checked before replay?
Worked answer. The observation leaves expected quantity at 10. Adjustment gives 9. Receipt gives 14. Check the stream identifier, projection version, snapshot state and last sequence. Require the suffix to begin at 3 and continue without gaps under this lab’s strict rule. Record that the final position includes sequence 5.
Exercise 3 | Two requests, one old version#
An order is CONFIRMED at version 2. Fulfilment is accepted first, producing FULFILLED at version 3. A cancellation request based on version 2 then arrives. Should the system blindly retry the state change?
Worked answer. No. Its precondition is stale. After reloading version 3, cancellation is forbidden by the teaching transition graph. A separate returns process could exist, but it is not implemented here and cannot be invented by relabelling the fulfilled order as cancelled.
Exercise 4 | Read the logical clock#
A process has counter 5 and receives a message carrying counter 8. What is the new counter? Does an unrelated event with a smaller counter necessarily cause this event?
Worked answer. max(5, 8) + 1 = 9.
No. Logical-clock order preserves causal precedence in one direction; a
smaller scalar value alone does not establish a causal relationship. The
counter is not a physical duration. [S41]
Exercise 5 | Repair a mistaken issue#
In separate stream R-EX-01, the opening quantity is 12. An issue was incorrectly recorded as 3 when the scenario says 2 were actually issued. Show a reversal and replacement, and explain why a second reversal should fail.
Worked answer. Record -3, giving 9. Reverse that target with +3, returning the recorded position to 12. Record the correct replacement -2, giving 10. A second reversal would add another 3 despite referring to the same already repaired entry, so the fixture rejects it. These entries repair the record; they do not describe goods physically travelling back and forth.
Exercise 6 | State what missing history prevents#
You retain a compatible snapshot through sequence 5 and all later events, but not events 1–5. Can you claim to have reconstructed the entire history from its original source events?
Worked answer. No. You can report continuation from the retained snapshot under stated assumptions. You cannot inspect or replay the missing prefix. The snapshot’s metadata and any retained verification help characterise its basis; they do not replace the absent source details.
6.98 Common wrong ideas#
- Wrong: a current balance explains its own history. Right: several different histories can produce the same balance.
- Wrong: observing a stock count is the same as authorising an adjustment. Right: evidence and state-changing decisions are distinct operations in this model.
- Wrong: a cancellation request proves that cancellation occurred. Right: a request may fail validation, authorisation or its version precondition.
- Wrong: sorting timestamps discovers every causal relationship. Right: clocks, delivery and dependencies require separate interpretation.
- Wrong: a smaller logical counter proves causation. Right: causal precedence implies increasing counters, but the converse does not generally hold. [S41]
- Wrong: a snapshot is useful without knowing where it belongs. Right: stream identity, interpretation version and sequence boundary are essential.
- Wrong: replaying old events should repeat their physical effects. Right: rebuilding a projection should not resend parcels or repeat charges.
- Wrong: a reversal automatically undoes the outside world. Right: an opposite recorded effect and an external compensation are different things.
- Wrong: append-only means every payload must remain forever. Right: mutation policy and retention policy answer different questions.
- Wrong: passing this in-memory lab proves a production event platform is reliable. Right: it checks bounded rules and fixtures, not unimplemented storage, security, concurrency or recovery.
6.99 Chapter summary in 20 lines#
- A state describes a position; an event records an occurrence relevant to a model.
- A retained event is not automatically a true account of the physical world.
- Our notebook stream starts at twelve and issues two for O-1042, giving ten.
- Observing nine records a discrepancy; it does not itself authorise a movement.
- An explicit minus-one adjustment and a receipt of five produce fourteen.
- A projection applies declared interpretation rules to a defined input history.
- Event sourcing makes an event history authoritative; an audit table alone does not establish that architecture.
- A command requests a change and can be rejected rather than becoming an accepted event.
- State-transition rules identify permitted moves from particular starting states.
- An expected version helps detect requests based on stale state.
- Physical time, recording time, delivery order and stream sequence are different concepts.
- Logical clocks can preserve causal order without being clocks of physical time.
- Smaller logical values do not, by themselves, prove causation.
- A strict projector must not quietly claim completeness across a sequence gap.
- A snapshot needs a stream, interpretation version and included-history boundary.
- Rebuilding a projection must not casually repeat external side effects.
- A reversal needs a valid target and a rule preventing duplicate application.
- Correcting a record does not necessarily undo a real-world action.
- Retention decisions can limit which earlier details remain reconstructible.
- State the evidence boundary honestly, then move to Chapter 7: files, folders and data formats.