Why passing tests missed the real work
A review of WerkgeverWijzer, an ordinary CV and the real WordPress authentication chain exposed three different gaps in our evidence. We changed what acceptance had to prove.
The question behind the work
The problem
WerkgeverWijzer 2.0 combines employer information, pay guidance, application preparation and knowledge about working in the Netherlands. Technical checks had passed, but on 1 October the Owner rejected treating that technical tranche as the intended completed product. A page being reachable did not show that a person could finish a useful journey. Later demo and security tests exposed the same wider problem: a test can be correct about its chosen fixture while leaving the actual user or host behavior unproved.
01
What we tried
- We changed the public acceptance target to six concrete intentions: finding work, understanding pay, researching an employer, finding Dutch work information, preparing an application and identifying pay or contract questions. Dedicated product, frontend and content executors prepared the integrated Dutch and English experience; a separate reviewer tested the actual public journeys. The Manager coordinated and reviewed rather than serving as the primary implementer.
- We separately built a private guided job-search demo. It used synthetic vacancies and travel facts, deterministic document extraction and human confirmation of extracted fields. Independent review followed the fourteen-step route, downloaded the prepared documents and checked the boundaries against real sending and public activation.
- After the Owner reported demo problems, we tested readable CV PDFs with different layouts, distinct notice-period choices and a person requiring full-time work. We also exercised a zero-result explanation and its correction route instead of accepting the absence of results as a sufficient answer.
- For authentication, we moved beyond helper-method tests into an isolated restored WordPress runtime with synthetic accounts. The check asked whether correct synthetic credentials were still denied after the failed-attempt threshold when the complete authentication filter chain ran. A later protected host probe checked the deployed integration without signing in to a real account.
02
What happened
- The rebuilt public version passed twenty-four Dutch and English journey checks, with responsive and negative-feature checks recorded separately. Exact deployed files matched the reviewed release. That was acceptance of the visible public scope, not a claim that every planned private feature or the whole product was complete. It was not fabricated Owner acceptance or market validation.
- The earlier PDF fixture had a deliberately regular structure. An ordinary readable layout produced wrong role and employer fields. A second previously unseen layout then exposed a missing education record. Both failures were preserved and the final repair passed both layouts. Unknown fields remained editable and unconfirmed. Image-only scans still required text or manual input; no OCR or universal CV understanding was established.
- The corrected demo retained distinct notice choices through confirmation, reload and Back. It added a declared synthetic full-time example while preserving the earlier hard-boundary examples. An explicit impossible wage produced zero results with reasons and correction actions; changing the wage restored the eligible sample. The independent repair review passed thirty-nine outcome assertions. This remained isolated staging with no application sent and no production personal features enabled.
- In the restored WordPress runtime, an early rate-limit error could be replaced by a later core authentication result. A final enforcement step repaired the tested bypass. Twenty-nine actual WordPress checks passed after correction. The runtime also verified numeric-author restrictions and explicit synthetic identity migration and rollback; it did not establish a defect in ordinary profile updates.
- On 2 October, independent host review accepted the security overlay: six deployed files matched the frozen candidate and five protected files matched their preimages. Twenty-six public GET checks had no failures. The host authentication probe used an in-memory synthetic precursor through the actual filter chain; actual synthetic-account authentication had been tested separately in the isolated restore. Temporary helpers and synthetic counter keys were removed. Existing author-identity migration remained a separate unfinished action, and two-factor enrollment was not tested.
03
What we learned
Define acceptance around the result a person can obtain. Build, route and data checks remain useful, but they cannot stand in for a coherent journey with a readable output and a recovery path.
A regular fixture can conceal assumptions about document layout and coverage. An additional varied input should challenge the extraction, confirmation and matching steps together. Preserve the failure before claiming the correction; distinguish a reproduced failure from a diagnosis read in another executor’s evidence.
A registered authentication callback does not prove the final login decision. When another callback can replace its result, the host’s complete execution chain becomes part of the test. A synthetic host probe and actual account authentication answer different questions.
Acceptance belongs to a named scope and frozen evidence. Public guidance, a private synthetic demo and a deployed security overlay have different finish lines. A green result in one cannot activate the others or close their remaining work.
04 · Outcome
The decision
ADOPT. Keep useful public journeys, varied user inputs and final runtime behavior as separate acceptance requirements. Retain regression cases for the PDF layouts, notice choices, full-time and zero-result recovery, and authentication filter ordering. Require source review, independent checks, recoverable release and public verification for the scope actually being activated.
Where the evidence leads
What this means next
Keep the guided job-search route private until its separate product and publication decisions are made. Recheck authentication when the relevant host or plugin chain changes. Later on 2 October, the separate author-identity migration was completed and independently verified for one existing account. The reviewed public-slug/login equality count became zero; credentials, email, roles and the displayed byline remained unchanged. The old author route redirected to the expected new route, which returned HTTP 200. This closes that bounded migration, while two-factor enrollment and broader authentication evidence remain outside its proof. Broader document understanding, live vacancy data and private account features need their own evidence.
See the public WerkgeverWijzer guidance