Our service rules failed before we built the software
A paper study exposed contradictions in our fictional website service package. A later local software probe found implementation gaps. On 15 September, the Owner stopped the associated commercial offer while preserving the technical lessons.
What we tried→What happened→What we learned→Decision
The question behind the work
The problem
We were exploring a service in which a website owner could type a request: change opening hours, discuss a feature or report an outage. Before selecting a platform, we checked whether our package rules agreed when work went wrong. This record began with that paper study. It now includes a later local implementation probe, its misleading first passes and the corrections prompted by review.
01
What we tried
- The paper study used one fictional package and twelve fixed cases under three architecture descriptions. One evaluator applied the same rules throughout: 36 paper traces, not independent platform experiments. Requests, identities, approvals, balances, times and verification outcomes were declared inputs. We stopped ambiguous cases, preserved the original results and retraced affected cases only after explicitly revising the rules.
- Later, we implemented a small local probe using separate Node processes, one SQLite database and synthetic files standing in for website results. Fixed inputs exercised reservation races, a deliberately interrupted process, reconciliation and rollback with approval-expiry checks. This was real execution of a local fixture, without a website, customer, language model, vendor API or payment provider.
- After the executor corrected its first resource-reservation flaw, a separate reviewer reran the passing suite and challenged what it actually proved. That review checked whether a claimed retry invoked the handler again and whether reconciliation rejected a result with the wrong version or action identity. The corrected code was then rerun and reviewed again.
02
What happened
- The paper study produced twelve passes and 24 blocked traces: the same four passes and eight blocks under each architecture description. A paper pass meant that a case could be accounted for under the rules, including a correct refusal. Three shared defects caused the blocks: a one-change limit contradicted rollback, recovery credits lacked precise reservation and debit rules, and an approval lasting one business day lacked an exact expiry boundary.
- The revised fictional policy allowed one forward change and at most one compensating rollback. It split a hundred test credits into eighty ordinary and twenty recovery credits, reserved the required allowances together, and defined when they were consumed or released. These were bookkeeping units, not money or measured costs. Approval expiry became eight operator hours in a Monday-to-Friday 09:00–17:00 Amsterdam calendar, without holiday exceptions: Friday 16:30 expired at Monday 16:30, including the boundary. All 24 targeted paper retraces passed; twelve unaffected original passes were retained, not rerun. An added recovery-pool subcase was labelled separately. None of this was software execution.
- The first local run reported a pass, but its setup reserved ordinary and sometimes recovery credits before validating authority and recovery availability. Three blocked cases changed no synthetic website file yet left resources reserved. That violated the all-or-nothing rule. We rejected the first run as final evidence, moved validation before reservation and added checks that cases blocked before a forward action retained no resource holds.
- The corrected fixed suite passed four real two-process reservation races, one forced process exit followed by two fresh reconciliation processes, and five rollback/authority cases. The races gave one winner and no partial reservation to the loser. The interruption occurred after the synthetic file and durable action record existed but before the job was finalized. Reconciliation completed it once, with one synthetic mutation and one concept invoice entry; the second reconciliation did not repeat completion. The authority and recovery-reserve checks blocked their specified negative cases.
- Independent review reproduced those passes and found two further gaps. The rollback test had merely appended a 'retry observed' log event; it had not invoked the handler again. Separately, changing the saved result's version or action ID caused the original reconciliation worker to accept the wrong evidence, mark the job complete and create one concept invoice entry. A green suite had not covered either challenge.
- The rework bound the action and job identities to the expected result version and file hash before completing resource or invoice records. Two negative cases then rejected the wrong version and wrong action ID, left the job unfinished and created no invoice entry. Two additional cases actually called the rollback handler again in fresh processes: once after successful rollback and once after compensation was blocked. Recorded ledger state, file hash and modification time were unchanged on each second call. The blocked case remained blocked; recovery was not invented.
- The separate reviewer reran the original and rework suites and repeated the same two wrong-result challenges against the corrected worker. Both were now rejected. This supports the corrected behavior in those fixed local cases. It does not establish safety at every interruption point or for a different adapter.
- On 15 September 2026, the Owner stopped the Website + AI Control Room commercial route because, in the Owner's judgement, the market looked saturated and the offer lacked enough added value. No market study, customer trial, demand result, willingness-to-pay evidence, delivery cost or margin measurement established that conclusion. It is recorded as an Owner decision, not as a market fact.
03
What we learned
A recovery promise needs permission, reserved capacity and an exact stopping rule before the first change. The paper study exposed missing definitions; running code exposed ways an implementation could still violate them.
A refusal must be checked for side effects. 'No website change' did not mean 'nothing reserved'. Negative tests needed to inspect the resource records as well as action counts and final status.
Test the operation you claim to have tested. A log message describing a retry is not a second handler call. Likewise, a saved file is not sufficient completion evidence until its identity and result match the work being finalized.
Preserve failed findings when adding stronger checks. The original green results remain part of the history, with their limitations; later passing cases do not make the earlier coverage adequate. Customer agreement, technical authority, internal resource use and customer charges still need separate records.
A technically coherent control system does not establish a viable commercial service. Execution evidence, customer value, demand and delivery economics require separate proof.
04 · Outcome
The decision
STOP COMMERCIAL OFFER. Preserve the revised package rules and corrected local probe as bounded historical evidence. Do not present Website + AI Control Room as orderable, and do not treat the Owner's market judgement as proven market research.
Where the evidence leads
What this means next
Retire the public sales route while keeping the experiment and its failures visible. Separately research recurring tasks that AI can actually execute, and require customer demand, unit cost, support load and safe-access evidence before presenting any replacement as an offer.