Skip To main content

Automated Accessibility Testing Tools: How Much Do Scans Catch?

Published:

Automated testing tools are an essential component of web accessibility testing, but can they be used as a standalone tool to assess whether your website meets the Web Content Accessibility Guidelines (WCAG)?

Automated Accessibility Scan Tools

No.

Automated scans (sometimes called checkers) can flag approximately 25% of issues occurring on your website, but to get a full and accurate assessment of your website’s conformance with WCAG 2.1 AA, both an automated scan and manual audit is necessary.

Why Automated Web Accessibility Testing Tools Aren’t Enough

Automated testing tools are cost-effective, easy to use, and can catch a multitude of issues across websites of all sizes in less than a minute.  This efficiency is extremely helpful to your organization’s team.

With an automated scan, you can get a good idea of where your website’s accessibility stands and some important issues to fix. However, it’s important to understand automation’s limits in testing for accessibility.

For WCAG 2.1 AA tests, automated scans can only check for 20-25% of accessibility issues. This means 75-80% of issues will not even be detected.

Scans fall short because they’re bound by yes or no conditions.  To fully assess accessibility, there must be a dynamic mind capable of processing whether certain WCAG success criteria are met.

To date, no artificial intelligence or AI has been able to replicate human minds in testing accessibility.

Experts are able to review the issues flagged by automated testing tools, and — more importantly — go much deeper to test critical areas and capabilities on your site that automated tools aren’t capable of assessing.

Some of these areas include:

Keyboard-only navigation

Navigating a website without a mouse is a necessary capability for countless users with disabilities. A manual test is the only way to determine if your digital properties are optimized for keyboard-only navigation, which will allow users to browse your website, access page menus, interact with links, etc., using only keyboard commands.

Descriptive Content

Titles, headings, and alt text all must be appropriately descriptive.  Scans can tell you whether a title or alt text is present.  Scans can flag heading structure as potentially out of order.  But scans cannot assess whether these elements are appropriate for the page.

User experience

Today’s websites are complex, and automated testing tools can’t interact with them the same way a real user would. For example, consider testing your website’s checkout experience. An automated testing tool cannot add items to a cart and complete a purchase, leaving one of the most critical components of your website untested. Conversely, a manual tester would be able to simulate the entire purchase process and identify any accessibility issues occurring throughout.

False Negatives

Automated testing tools are also prone to missing issues in conditions that they check for. This amounts to a false negative because someone can be led to believe there are no accessibility issues where some exist.

For example, if an image has an empty alt attribute – indicating it’s a decorative image – this will pass an automated test because decorative images are acceptable.  However, an alt text value may actually be needed if the image is meant to convey information.

___

This is why an automated scan is extremely helpful and a manual audit is essential when assessing the accessibility of your website.

With accessibility lawsuits soaring, it’s best practice to make sure your scan errors are down to 0, then have an independent audit conducted and remediate your website per the audit recommendations.

Our experts can help you perform a full audit of your website to ensure full WCAG conformance.

Connect with us today to learn more about our automated and manual auditing capabilities and extensive range of digital accessibility services designed to help keep you compliant with ADA, AODA, Section 508, and other global regulations.

 

Five approaches to avoid when making your digital properties compliant.

Just starting your accessibility journey? Learn more about commonly considered approaches to avoid. Download our 2021 Guide to Web Accessibility.

Subscribe to Blog Updates