BairesDev

Accessibility Testing Beyond Automated Checks

Automated accessibility testing finds technical gaps fast, but real users reveal what tools miss. Learn how to balance both for stronger coverage.

Last updated:
10 min read
Verified Top Talent

By Renata Matos

Product Manager18 Years of Experience

Renata is a product manager with 18+ years of experience combining technical expertise with strategic product vision. She has worked at Accenture and advanced from quality assurance to product ownership at CESAR.

Abstract illustration representing accessibility testing with geometric patterns, layered interface elements, visual inspection symbols, and testing indicators on a green gradient background.

Checking website accessibility requires more than running an automated accessibility test. Accessibility testing is a human issue that tooling alone cannot resolve. Does everyone get a great user experience?

That doesn’t mean that tooling isn’t helpful. It can identify many common technical problems efficiently. Automated accessibility testing misses many website accessibility issues for those who use assistive tools. Automated accessibility tests are good for checking if accessibility features are available or functional, but real users know if the features make sense.

So the real challenge is balancing these two strengths. Risk is the starting point: the more your users depend on accessible interactions to complete critical tasks, the more testing you need. Teams need enough testing to find meaningful accessibility issues without requiring a full manual audit for every release. Integrating automated checks into CI/CD pipelines is a practical strategy for maintaining that baseline without slowing down delivery.

Key Points

  • Automated methods are a good baseline that requires additional testing layers to provide full coverage. They identify many technical problems quickly; manual methods evaluate contextual and experiential aspects of digital accessibility.
  • Different accessibility testing methods identify different problems. Manual testing, assistive technology testing, and testing with people with disabilities each provide information that automated tools miss.
  • Your release gates should block on failures that stop core task completion but can defer minor low-frequency issues.
  • Your UX team needs to address accessibility during design so QA handles less of the remediation cost.
  • Rigorously testing a shared component once prevents the same barrier from repeating across every screen that uses it.

What Does Accessibility Testing Evaluate?

Accessibility testing is an inclusive evaluation of the usability of digital products. Can everyone use the product, including users with disabilities? It promotes universal design for a broader audience. A power user that prefers to use keyboard shortcuts benefits alongside users with disabilities who cannot use a mouse.

Web content accessibility guidelines are a good place to begin. The Web Content Accessibility Guidelines (WCAG) provide criteria for evaluating web accessibility, including conformance levels that help teams understand how thoroughly their site meets accessibility standards. Some of those criteria can be found with automated scans. Others require a person to conduct accessibility testing to ensure the site’s accessibility.

For example, form fields can be challenging for users with disabilities to use. The input field is a separate element from any labeling of the input, so it can be challenging for a screen reader to understand what information should go in a field. This can be further confused for visually impaired users if color is used to indicate an error in a form instead of readable text.

An automated tool may catch whether an input has an accessibility label, but code that is technically valid does not necessarily meet web accessibility standards. A user still needs to be able to navigate to the field, understand the purpose of the field, identify entry errors, and fix those errors. Testing the workflow generally requires more than just checking the HTML via automated tooling.

Layered Accessibility Testing

This is why accessibility testing requires manual and automated methods. Some problems can be identified by scanning the code with automated tools, while others need user feedback. A layered testing approach can balance these needs. **Automated accessibility testing** excels at finding common technical issues. **Manual testing** gives more insight into accessibility barriers. **Assistive technology testing** reveals how assistive technology interacts with the digital product. **User testing** provides a deeper understanding of aspects that have a significant impact on users.

The Role of Automated Accessibility Testing

Automated tooling should be the first step. It is fast and repeatable. Automated tools can scan pages and interactive elements for basic accessibility guidelines and catch some ARIA (Accessible Rich Internet Applications) issues.

Accessibility testing tools are well suited to development and CI/CD workflows. A developer can run an accessibility checker to verify accessibility guidelines while building an interactive element. The tests are repeatable, so as the component changes, the accessibility checks can be rerun. This allows most accessibility success criteria to be evaluated before a formal review. 

A recent study found that a combination of automated accessibility testing tools can help meet 39% of WCAG success criteria. This is a great start, but also demonstrates that automation is not enough. Web pages that pass automated checks may still have gaps in equal access and full accessibility compliance.

Manual Testing Finds Problems Automation Cannot

Automated accessibility testing checks for issues programmatically. This approach misses the context of the interaction and thus the actual experience of the user. For example, an interaction may be technically available via keyboard navigation while remaining unintuitive for those with visual impairments.

It is difficult to reduce interfaces to an accessibility score. Interactive elements depend on components working together and the context can change based on user interaction. For example, a screen reader is helpful for understanding digital content, but might be less helpful if the screen reader starts reading out every auto complete response suggestion in a search field as a user types.

Manual accessibility checks allow a tester to evaluate those relationships and interactions directly. It can reveal whether a workflow makes sense to users and whether the interface communicates what is happening as they interact with it. Human evaluation can also expose problems that are only noticeable when someone interacts with the interface using a different input method, such as a screen reader or browser extension.

Manual testing is slower than automated testing and produces higher-quality findings. However, it is often not possible, or cost effective, to use manual accessibility testing everywhere. Concentrate manual testing on important and reused components as well as common workflows.

Understanding Compatibility with Assistive Technology

Assistive technology testing examines how the interface behaves when assistive tools (e.g. screen readers, keyboard-only navigation, or voice control) are used. This can identify problems missed during conventional manual testing.

For example, a tester may be able to navigate to a form with the keyboard, but still find that the screen reader does not provide enough information for the user to understand the form fields or changes made in the form fields as the user interacts with the field. Similarly, content that is displayed based on user actions may not be communicated appropriately when using assistive technology.

This type of testing requires time, so it is best to implement it strategically. Not every page or component needs to be tested. Focus on areas that are dynamic, have complex interactions, and are important aspects of the user experience or workflow.

Testing with People with Disabilities

Testing with users with disabilities is the best way to evaluate the actual experience of those users. It can identify accessibility barriers that technical testing misses, or testers do not think of. Real users are also more likely to identify issues that, although technically correct, are inefficient or cumbersome for the user. The World Wide Web Consortium (W3C) recommends combining user evaluation with accessibility testing tools.

User feedback is most useful when developing highly interactive elements where it can be hard to predict what the user experience will be using assistive technology, particularly for digital products like mobile apps where interaction patterns vary widely. Or, if the web content is updated and the change could significantly affect user experience.

Although this approach can identify unique insights, it still leaves gaps in accessibility coverage. Different people use assistive technology differently. But it is still an important layer of testing because real users can give real insights that the other testing layers miss.

Choosing the Right Testing Layer

Each type of testing answers a different question.

Testing method What it can detect Expertise required Scalability Frequency Main limitation Development lifecycle
Automated Accessibility Testing Common markup, labeling, contrast, semantic and ARIA problems Developer or QA knowledge High Every build or pull request Cannot evaluate many contextual and interaction problems Development and CI/CD
Manual Keyboard navigation, focus, interaction, and contextual problems Accessibility-aware developer or QA tester Moderate When features change and during regression testing Takes more time and depends on tester skill Development and QA
Assistive technology Problems experienced through screen readers, and other assistive technologies Experienced accessibility tester Lower Important workflows and major releases Results can vary across environments QA and release validation
User testing Problems encountered by people with disabilities during real tasks Accessibility or usability research expertise Low Major changes and high-risk workflows Limited participants cannot represent every user Design and validation

Not every testing layer will be used for every change or release. It is important to choose the layer of testing based on the problem being tested.

How Much Accessibility Testing is Enough?

Accessibility testing works best when it is integrated into the development lifecycle. 

Developers can test for digital accessibility when building components. These tests can be integrated into the CI/CD process to catch basic issues early. Manual testing can be added when creating interactive components and important workflows. Assistive technology testing can focus on the workflows where digital accessibility problems would have the greatest impacts.

Release gates should be based on risk. A failure that prevents a user from completing a core task is a bigger blocker than minor irritants that occur infrequently. Low-impact issues still require tracking and remediation. The goal is to determine which issues should block a release and which can be tracked to fix after release.

The amount of testing needs to reflect the risk profile of the product. An application that accesses sensitive data or manages critical processes requires more testing. Aspects that define the user experience should be tested more closely too. 

Release frequency matters too. Manual testing every deployment is impractical for products released several times a day. Automated testing should be a baseline, but it is best to use a targeted approach for manual testing.

Accessibility Testing is Part of the Development Process

Accessibility testing is the most effective when it is completed throughout the development process instead of at the end as an afterthought, during the final QA test. Developers need to understand accessible HTML and keyboard interactions. The UX team needs to consider accessibility during the design process and incorporate it into their testing. Accessibility specialists can provide additional detailed testing, or information about accessibility compliance and legal requirements.

Creating and maintaining component libraries can be a good strategy to reduce the amount of manual accessibility testing across web applications and mobile apps alike. A component can be more rigorously tested when developed and then re-used throughout the web application. This allows developers to address accessibility issues once and prevent the same usability barrier from repeating throughout the application.

Most releases require only the level of testing appropriate to the changes made. When the appropriate level of testing is incorporated into the development process, more in-depth testing can be used judiciously when user experience is central. Organizations that do not have UX capacity or accessibility expertise in-house can use UX design services to support adding accessibility considerations into the design process. Ultimately, accessibility works best when it is considered as part of creating the product, rather than something added after it is built.

Frequently Asked Questions

  • Is automated accessibility testing enough?

    No. Automated tools cover only a portion of WCAG criteria. They’ll catch missing labels and contrast failures, but they can’t tell you whether a screen reader reads autocomplete suggestions in a way that makes the field unusable. You need manual and assistive technology testing to find those problems.

  • Does every release need manual accessibility testing?

    Not every release does. Run automated checks on every build since they’re fast and repeatable. Save manual and assistive technology testing for releases that touch important workflows, dynamic components or anything your users depend on to complete core tasks.

  • When should accessibility testing happen?

    Start during development. Developers can run accessibility checks while building a component and wire those into the CI/CD pipeline. Your UX team should be evaluating accessibility during design reviews. QA picks it up in regression testing. Waiting for a final audit means you’re fixing problems that were cheaper to prevent.

  • Should we test with people with disabilities?

    Yes, when the timing is right. User testing surfaces barriers that automated and manual methods don’t catch because real users interact with assistive technology in ways testers don’t anticipate. It’s most valuable for highly interactive workflows and for changes that could significantly affect how users experience your product. Keep in mind that a small participant group won’t represent every assistive technology setup, so it works best as one layer among several.

By Renata Matos

Product Manager18 Years of Experience

Renata is a product manager with 18+ years of experience combining technical expertise with strategic product vision. She has worked at Accenture and advanced from quality assurance to product ownership at CESAR.