Methodology

A rigorous, evidence-based method.

Every criterion is evaluated from captured evidence against a fixed rule set, and every finding is independently reviewed before it is reported.


The method

Seven rules we do not bend.

  1. Scope from data, not from guesses

    We build the scope from your sitemaps, your app's own route table and a read of each kind of page, then group pages by template. You see what is in and what is out before any work starts.

  2. Capture once, judge many times

    Every page and screen is stored with its structure, pixels and behaviour. Any finding can be judged again later, by a newer rule or a second auditor, without touching your live site.

  3. Decide every criterion

    Each criterion gets a position: Pass, Fail, Not applicable or Needs human review, with the reason written down. Nothing is left silent, and "not evaluated" is said out loud.

  4. Prove every finding

    A finding names the element, shows the screenshot and, on mobile, quotes what the screen reader said. If it cannot be shown, it is not reported.

  5. Try to disprove it

    A second, adversarial review attacks every finding before it ships. Findings are reproduced on the stored page with an exact locator; what does not reproduce is referred for human review, never confirmed.

  6. Grade the barrier, not the rule

    A missing label on the only Continue button of an account-opening form outranks a hundred cosmetic warnings. Severity follows what the user cannot do.

  7. Not raised is not fixed

    In a re-audit, a finding that a scanner stopped raising is measured again on the same element, with a control, before it is called solved.


One decision, step by step

How a single criterion gets its answer.

How one criterion is decided Is the criterion applicable to the screen? If not, Not applicable, with a census of what was looked for. If yes, the captured evidence is judged. A candidate failure must survive an adversarial review and be reproduced to become a Fail; if it cannot be reproduced it goes to Needs human review. If no failure is found, the result is Pass. Applicable to this screen? Judge the captured evidence Survives review and reproduces? Fail, with evidence Needs human review Pass Not applicable yes no candidate failure no failure found yes no
Every criterion on every screen ends in one of four positions, each with its reason written down.

Documents

The same checks, before and after.

Every document is audited when it arrives and again when it leaves, through the same set of checks, and the two results are compared. A new or worse result blocks delivery.

Opens intact

The file opens, renders and matches the original page for page.

Machine conformance

Validated against the PDF/UA standard's machine-checkable rules.

Human checks

Headings, reading order, tables, alternative text and forms, judged by a person.

Every criterion

A position on all 54 WCAG criteria that apply to documents.

What a reader hears

A record of what a screen reader will say, page by page.

What a reader sees

Every page rendered and compared with the original, so nothing moves without being stated.


Limits we state

We tell you what we could not test.


Standards we cite

So any finding can be checked by anyone.

WCAG 2.1 and 2.2 · WCAG-EM, the W3C's evaluation methodology · EN 301 549 · PDF/UA (ISO 14289) and the Matterhorn Protocol · WAI-ARIA authoring practices.

Ask us to show the evidence.

Pick any finding from a sample report and we will walk you through how it was captured, judged and checked.

Talk to us →