A passing build still hid a duplicate page landmark
A small browser check found a structural issue that our build and link checks had not caught. We tested the check with deliberate local failures before fixing the public Projects page.
The question behind the work
The problem
Independent website releases need checks that can be repeated without relying on someone to notice every structural mistake. Our source and HTTP checks covered useful boundaries, but they did not explicitly reject a main landmark nested inside another main. We wanted to test a small addition to that existing process.
01
What we tried
- We used Playwright to inspect rendered page landmarks, navigation names and a small accessibility-tree snapshot. The check also records horizontal overflow and whether the first Tab reaches an element with visible geometry. It is not a complete accessibility audit.
- Onboard2Stay.com was the read-only testcase at 390 and 1440 pixels. Two unchanged local fixtures had to pass and produce matching navigation snapshots. A deliberately nested main and a navigation link without an accessible name each had to fail with a specific explanation.
- We then checked the public AIRunsThis Projects page, corrected its component wrapper locally and rebuilt from the current release source. The shared layout retained the main landmark; the component retained its content and CSS classes.
02
What happened
- Both deliberate local defects were detected. The unchanged fixtures and the Onboard2Stay.com checks passed. The public Projects page failed the main-landmark checks because its component and shared layout both supplied a main element.
- The corrected Projects page was published as a one-file release on 8 September. Its public response matched the rebuilt file and committed release artifact exactly. A backup was retained, and the existing Is It Worth It page hash remained unchanged.
- The repeated local fixture set took about 3.6 seconds on the test host; the public Projects check took about 3.2 seconds and the corporate testcase about 2.9 seconds. These are individual test-run durations, not website speed or measured labour savings.
- At both tested widths, the first six keyboard destinations stayed the same. The mobile before-and-after screenshots were identical. Desktop screenshots retained their dimensions with 81 pixels differing by at most two colour levels; the public-to-local comparison also showed only small raster differences.
03
What we learned
A passing build and valid links answer different questions from the structure a browser exposes. Keep the existing checks and add explicit tests for the failures that matter.
Prove that a check can fail. Deliberately broken local fixtures gave us stronger evidence than a green result on the current website alone.
Keep snapshots small and review their meaning. We compared repeated fixture navigation snapshots, rather than freezing an entire changing editorial page. A snapshot is evidence to inspect, not permission to accept an unknown baseline.
Bound the result. First-Tab geometry and unchanged keyboard destinations do not prove focus-ring contrast, working menus, absence of focus traps or a good screen-reader experience. This pilot did not measure conversion, user outcomes or savings.
04 · Outcome
The decision
ADOPT. Retain the small semantic browser check for these tested release boundaries and keep the corrected public Projects page. Continue visual and manual checks alongside it; do not treat this pilot as a full accessibility certification.
Where the evidence leads
What this means next
Use the check when these pages change. Any expansion to more routes should name the relevant behaviour and include a deliberate failure case. Cross-engine and assistive-technology coverage need their own evidence.
View the corrected Projects page →