Keep the contract. Replace the machinery.
A conversion takes an existing system (a broker, a database, a service) and keeps everything its clients can see: the wire protocol, the API, the configuration and its own test suite, unmodified and passing. Everything inside is yours. Two phases get you there: first learn what the incumbent actually does, then build the same contract on the memory.
The model, in five sentences.
The contract is kept; everything inside it is yours.
Wire protocol, API, configuration, the incumbent's own suite. Not its process layout, its server, its internal messages, its schema or its threads.
The memory is memory.
State lives in the tuple space. A participant reports what is (reads) and sets what shall be (writes). Nothing asks another machine for anything.
Each participant writes its own truth, once.
One writer per location. No acks, no confirmations, no retries of a write that landed, no waiting for a reader.
Everyone else reads it from there.
A held read on a tuple family fires when something in it changes. Triggers are the callbacks. The client is told everything is fine unless a trigger changes what it was promised.
The three indices are free.
Pick them so every common question is one partial-index read on what the reader already knows, and every trigger is a family. There is no required naming convention.
Would you do this on one machine?
If this were a multithreaded program on one machine, would you send a thread a message and wait for its reply to update a shared table? Ack a memory write? Run a server so threads could ask what the table says? Then don't do it here.
What you keep, and what you throw away.
| Keep: the contract | Throw away: the incumbent's machinery |
|---|---|
| The wire protocol its clients speak, byte for byte | Its server process and its internal RPC, REST and IPC |
| Its public API and return values | Request and response between its own parts |
| Its configuration surface | Its locks, queues, worker pools and coordinators |
| Its own test suite, unmodified | Its schema, its caches, its maintained counters |
| Its documented semantics: the ordering, delivery and durability it actually promises | Its acks, retries and heartbeats between its own parts |
Clients outside the contract still do what their protocol says. A QoS 1 MQTT publisher waits for PUBACK. Answer it at once, from the memory write. Inside, nothing waits.
Mechanical conversion: learn what the incumbent actually does.
The goal is understanding, not the product. Change the incumbent as little as possible and watch what breaks. The phase-1 code is scaffolding and is not shipped.
| Step | What you do | What it leaves |
|---|---|---|
| 1. Baseline stock | Build the incumbent. Run its own suite and record pass/fail per test, with wall time and bytes. Build the side-by-side load harness now, against stock, so every later number has a stock column measured the same way. | The stock baseline |
| 2. Map the suite | Every test: what it asserts, which part of the incumbent it exercises, and whether it applies. This is the roadmap. | <NAME>-SUITE-MAP.md |
| 3. Find the talking | Every place the incumbent's parts talk to each other or share state: REST calls, RPC, shared tables, queues, globals touched by more than one connection. File and function for each. | study-*.md |
| 4. Replace it, one at a time | Same shape as the incumbent. If it called a service, call the memory; if it read a table, read the tuple family. No redesign yet. Run the suite after each one. | Scaffolding |
| 5. Feed it good and bad data | Boundary values, malformed input, adversarial ordering, concurrency the suite doesn't test (many clients, fan-in, fan-out, takeover mid-flight), and kills. What the incumbent does with each is the contract wherever its suite is silent. | The unwritten contract |
| 6. Write down the algorithms | What each core algorithm solves, its invariants, the order it guarantees and to whom, what it does under contention, its edge cases. Written as if the reader will never see the source. | study-*.md |
| 7. Keep the register | Every red, with the three questions in order: is the implementation wrong; was the contract misread; does the incumbent expose something the memory declines to reproduce? An entry is a decision with its reason, not a bug. | QUESTION-3.md |
Phase 1 is finished when you can say, for every piece of shared state, who changes it, who reads it, what order its readers need, and what the suite and your bad data say about it.
Number three is the gold. See how we test.
Greenfield: the same contract, built on the memory.
Start again. Do not port the phase-1 code, and do not keep the incumbent's structure because the incumbent had it.
The truth table comes first, before any code.
One row per piece of state the contract depends on, in DESIGN.md:
| Column | What it answers |
|---|---|
| Truth | The piece of state. |
| Owner | Its one writer: the participant whose fact it is. If two participants want to write it, the address is wrong. Give each its own location and let readers union them, or make the contest state (claims, decided by write order). |
| Indices | Chosen for the read. An exact question is one tuple; "everything for X" is a family with X fixed. |
| Latest or distinct | One tuple overwritten (a slow reader gets the newest), or one tuple per value (every reader sees each, in write order). |
| Readers and trigger family | Who holds a read on it, and what they do when it fires. |
| Derived instead? | Counts, sizes, liveness and "who is behind" are computed when asked, not stored. |
If any row's reader gets the truth by sending a message to its owner, the design is wrong.
Contested read-modify-write
Where the contract needs one order over a contested truth (INCR, LPOP, MULTI/EXEC, an election, a takeover), never read, compute and write back across participants. Either give the truth one owner that applies the others' published intents, or publish each change as an intent and let every reader apply them in write-order id. The write-order id is the one order.
The participant
It serves the contract's clients, holds what it needs as held views, writes its own truths and acts on triggers: 100 to 250 lines each. It answers the client from the memory write. Exceptions only come back; silence means fine. No timers for expiry, polling or retry. No participant-level mutex.
Verify every step
Each step adds one truth or one participant, then runs the incumbent's suite (unmodified), the good and bad data from phase 1, and the side-by-side bench. Red first: a behaviour gets a failing test before the change. Never modify the incumbent's tests to pass.
Report what you measured
Side by side against stock, on at least two machines with traffic crossing between them. One machine is just embedded RAM, and single-machine numbers against stock say nothing about what Ribbit is for. Throughput, latency, CPU per message, memory, loss and the suite. Lines and complexity, counted as statements. What is not claimed, on the same page.
What a conversion leaves behind.
Every conversion leaves these files in its example directory, so the next person or the next AI session can pick it up and re-run the suite and the bench before changing anything.
| File | Phase | What it holds |
|---|---|---|
| <NAME>-SUITE-MAP.md | 1 | The incumbent's suite, test by test, applicable or not and why |
| study-*.md | 1 | The core algorithms as understood, with source references |
| QUESTION-3.md | 1, 2 | The register: every red and its decision |
| DESIGN.md | 2 | The truth table, the indices, the triggers, the participants |
| RESULTS.md | 2 | Measured, side by side, tied to a version |
| README.md | 2 | Build, run, bench: one command each |
| VERSION | 2 | What the logs and the bench print, so every result names what produced it |
How it goes wrong.
The MQTT conversion's first design. Every one of these was built, measured and then thrown out. Check your design against them.
| Mistake | What it looked like | What it should be |
|---|---|---|
| A server for the state | A separate process holding the region, with its own operations (hello, sub, claim, pub) | The region is the memory. There is no server to ask. |
| Messages instead of memory | Batches of publishes sent to the server, results sent back, deliveries fetched by held "wait" requests | A participant writes its truth; the others' triggers fire. |
| Acks on writes | Each PUBACK held until the server answered: 136–152 µs per QoS 1 publish against 36 µs for stock, 0.8× stock throughput on one publisher | The publish is in memory when it is written. Tell the client at once. |
| Asking who is listening | The publisher looked up remote subscribers before writing | Write once. The subscribers' triggers find it. |
| Calls that block on a round trip | Subscribe, claim, own and clean each waited for an answer | Those are writes. |
| Ordering by draining | Waiting for every batch in flight to be answered, to keep Mosquitto's order | The memory's write order is the order. |
| Tuning the wrong design | Lanes, coalescing, chunked deliveries, per-lane sequencers, same-box shortcuts. Each was measured and each helped. | Every one existed only because of the mistakes above. When a fix exists only to make messaging cheaper, the fix is to stop messaging. |
| A shape that cannot leave the box | The fast resident shape, the region in one process, measured as if it were the product | A design that cannot span machines is not the product. Measure the shape that ships. |
| A naming convention nobody asked for | Indices chosen by habit | The three indices are free. Choose them for the reads. |
The second copy is the bug.
Redis's state is read-modify-write on keys every client may touch. Two earlier designs each kept a second copy of something and needed a layer to keep it true. That layer is the publish/subscribe class of code, and it is what to look for and remove in any conversion.
| Design | The copy | What kept it true |
|---|---|---|
| RAM host plus fronts with read caches | Each front's cache of values it had read | Per-front invalidation feeds, register-before-read ordering |
| Every box a copy, one order (Oct 17) | The whole keyspace in every box | Intent rings, pacing to the slowest reader, heartbeats, gap checks, a snapshot join, the same clock and seed on every copy |
| One keyspace (Oct 19, current) | None | Nothing to keep true |
In the current design the keys have one owner, the Redis region in the memory. A client never writes a key: its request is the RAM call, and the region applies it at the key's tuple, one request at a time per tuple, every connection's in parallel. Two INCRs on one key are two requests to one owner, not two writers. A blocked command doesn't need a thread either: it keeps its call's answer, and the write that readies it wakes it.
If a design needs any mechanism whose only job is to keep two copies equal, the second copy is the bug.
Doing it with an AI.
The failures above are the default behaviour of a model trained on request/response code. They recur unless checked.
- Read the model and the best-practices failure list at the start of every session.
- Write the truth table before any code, and show it.
- Before adding any mechanism, ask the one-machine question. If the mechanism exists only because parts are talking to each other, remove the talking instead.
- Research before answering: the repository, these documents, the example's checkpoint files. Don't raise a question the files already answer.
- Report measured, side-by-side numbers: which shape, which machines, how many runs, and what is not claimed.
- Keep the contract green at every step. Never modify the incumbent's tests to pass.
Determine your truth. Write your truth. Read the other truths. Answer from memory. Go.