Use the incumbent as the oracle.
When an established implementation has a behavioral suite, use it. Do not replace somebody else's contract with a demo written to flatter the new architecture.
Every failure gets three questions.
Is our implementation wrong?
Fix it. Add a regression test. Re-run the full gate.
Did we misunderstand the contract?
Correct the model before correcting the code.
Should the new model decline this behavior?
Write it down. Do not turn a deliberate semantic difference into a fake pass.
Never turn question three into question one just to make the suite green.
A bug report is not the end of the analysis.
Known defects are carried with their engineering disposition so candor is not mistaken for ignorance.
| Status | What it means |
|---|---|
| Unknown | The mechanism has not been investigated far enough to make a stronger statement. |
| Root cause known | The failure mechanism has been inspected and understood. |
| Solution designed | A concrete correction is understood, but it is not represented as running code. |
| Deliberately deferred | The solution is known, but present requirements do not justify paying the implementation cost ahead of work that currently constrains the experiments. |
| Implemented | The correction exists in code. |
| Qualified | The implemented behavior has been tested against an explicit contract and condition. |
That distinction matters in a research implementation. A five-machine development network does not need a roughly 59,500-node allocator merely to prove that the replacement can be designed. Deferring it preserves code velocity; it does not turn a solved design problem back into an unknown one.
Qualification is not characterization.
| Question | Meaning |
|---|---|
| Qualification | Does the required contract hold under the stated condition? |
| Characterization | What operating envelope did this implementation, machine and path actually demonstrate? |
| Policy | What administrative limit do we choose inside that measured envelope? |
| Planning | Does a proposed workload fit the measured and policy-constrained system? |
Freeze the evidence.
- Every reported number should point to the code and configuration that produced it.
- Same-session controls are preferred when environment drift can dominate the result.
- Simulator oracles cover behavior and discovery convergence before claims move to physical hardware.
- External suites are used where an incumbent already defines the contract.
- Failures, skips and deliberate incompatibilities remain visible.
Eight application classes. One vocabulary.
Psychedelic FrogTorch, Redis-on-Ribbit, Communicator, Monitor, Sourri, residency/swapper, FrogNet discovery/resolution and physical control do not share a business domain or timing model. The question is whether independently addressable truths, relationships, participants and held reads remain useful across all of them.