What automated testing can and cannot detect

All 55 WCAG 2.2 Level A and AA success criteria, and whether a scanner can find a failure of each one. Computed from axe-core 4.12.1's own rule metadata, not asserted by us.

Published estimates of automated coverage range from roughly 13% to 57%, and both figures are honest — they measure different things. Count by success criteria and you get the number above. Count by issue volume on real pages and it goes much higher, because the failures automation is good at finding are also the most repeated: one missing alt pattern in a template becomes four hundred violations.

Neither number tells you a page is accessible. That is the point of this table: the 32 rows marked needs a person are invisible to every scanner on the market, ours included, and they include some of the ones that most determine whether a page can actually be used — focus order, whether alt text is accurate, whether a custom widget behaves the way its role promises.

1. Perceivable

20 criteria at A and AA. 11 have at least one automated rule; 9 do not.

2. Operable

20 criteria at A and AA. 8 have at least one automated rule; 12 do not.

3. Understandable

13 criteria at A and AA. 3 have at least one automated rule; 10 do not.

4. Robust

2 criteria at A and AA. 1 have at least one automated rule; 1 do not.

The 32 no scanner will mention

Worth reading as a list, because this is the actual scope of what a clean scan leaves unanswered.

1.2.3 Audio Description or Media Alternative (Prerecorded) A

Prerecorded video has either an audio description of what is shown or a full text alternative.

1.2.4 Captions (Live) AA

Live audio content has real-time captions.

1.2.5 Audio Description (Prerecorded) AA

Prerecorded video has an audio description track — a text alternative is no longer sufficient at this level.

1.3.2 Meaningful Sequence A

When the order of content matters to its meaning, that order is available programmatically — the DOM order matches the reading order.

1.3.3 Sensory Characteristics A

Instructions do not rely on shape, colour, size, position or sound alone. Not "click the round button on the right".

1.4.5 Images of Text AA

Text is real text rather than a picture of text, unless the presentation is essential — a logo, for instance.

1.4.10 Reflow AA

Content reflows into a single column at 320 CSS pixels wide without requiring scrolling in two directions.

1.4.11 Non-text Contrast AA

Interface components and meaningful graphics have at least 3:1 contrast against adjacent colours — including the visible boundary of an input and the focus indicator.

1.4.13 Content on Hover or Focus AA

Content that appears on hover or focus can be dismissed, hovered over without vanishing, and stays visible until dismissed or no longer valid.

2.1.2 No Keyboard Trap A

Keyboard focus can always be moved back out of a component using the keyboard.

2.1.4 Character Key Shortcuts A

Single-character shortcuts can be turned off, remapped, or are active only while a component has focus.

2.3.1 Three Flashes or Below Threshold A

Nothing flashes more than three times a second, unless it is below the general flash and red flash thresholds.

2.4.3 Focus Order A

Focus moves through the page in an order that preserves meaning and operability.

2.4.5 Multiple Ways AA

There is more than one way to reach a page within a set — search, a site map, or navigation.

2.4.6 Headings and Labels AA

Headings and labels describe the topic or purpose of what they introduce.

2.4.7 Focus Visible AA

The keyboard focus indicator is visible. Removing outlines without replacing them fails this.

2.4.11 Focus Not Obscured (Minimum) AA

When a component receives keyboard focus, it is not entirely hidden by other content — a sticky header or a cookie bar must not cover it.

2.5.1 Pointer Gestures A

Multi-point or path-based gestures have a single-pointer alternative, unless the gesture is essential.

2.5.2 Pointer Cancellation A

An action is not completed on press-down; it can be aborted or undone before release.

2.5.4 Motion Actuation A

Anything triggered by device or user motion can also be operated by conventional controls, and motion actuation can be disabled.

2.5.7 Dragging Movements AA

Anything that uses a dragging movement has a single-pointer alternative that does not require dragging.

3.2.1 On Focus A

Receiving focus does not itself change the context — no dialog opening or navigation on focus alone.

3.2.2 On Input A

Changing a setting does not automatically change context unless the user has been told it will.

3.2.3 Consistent Navigation AA

Navigation repeated across pages appears in the same relative order each time.

3.2.4 Consistent Identification AA

Components with the same function are identified consistently across the set of pages.

3.2.6 Consistent Help A

Help mechanisms repeated across pages — contact details, a help link, a chat widget — appear in the same relative order.

3.3.1 Error Identification A

An input error is identified in text, and the item in error is described.

3.3.3 Error Suggestion AA

When an error is detected and a correction is known, it is suggested — unless doing so would compromise security.

3.3.4 Error Prevention (Legal, Financial, Data) AA

Submissions with legal or financial consequences are reversible, checked, or confirmable before finalising.

3.3.7 Redundant Entry A

Information already entered in a process is either auto-populated or available to select, rather than asked for again.

3.3.8 Accessible Authentication (Minimum) AA

No cognitive function test — remembering a password, solving a puzzle — is required to authenticate, unless an alternative or a mechanism to assist exists. Pasting into a password field must work.

4.1.3 Status Messages AA

Status messages can be read by assistive technology without receiving focus — a live region, correctly used.

Rules that map outside A and AA

axe-core also ships rules for 5 Level AAA criteria, and for one criterion that no longer exists.

• 1.4.6 Contrast (Enhanced) (AAA) — color-contrast-enhanced
• 2.1.3 Keyboard (No Exception) (AAA) — scrollable-region-focusable
• 2.2.4 Interruptions (AAA) — meta-refresh-no-exceptions
• 2.4.9 Link Purpose (Link Only) (AAA) — identical-links-same-purpose
• 3.2.5 Change on Request (AAA) — meta-refresh-no-exceptions
• 4.1.1 Parsing — removed in WCAG 2.2, but axe-core still reports duplicate-id, duplicate-id-active against it. A result citing 4.1.1 is not evidence of a 2.2 failure.