Tim.dev

You as a developer can start making more accessible products today by implementing various tools and techniques and using the output to improve accessibility (a11y). Ensuring the a11y of a web app will make the product more usable for all users.

Automated a11y testing & checking can be baked in during the engineering phase so a lot of low-hanging fruit can be tackled. Doing a lot of this low-effort work from the get-go will greatly reduce the amount of work you will have to do later on if you’re implementing or fixing a11y as an afterthought. This article focuses on the why and how for implementing these automated a11y quality checks during the development phase for React web apps. However, the principles are universal and many of the highlighted libraries have analogues for other JavaScript frameworks/libraries.

Introduction

Let’s first review the definition of accessibility to make sure we have a common understanding. I find the definition by Laura Kallbag in her book Accessibility for Everyone one of the clearest:

Web accessibility is the degree to which web products are usable by as many people as possible.

We often think of disabilities when thinking about accessibility. It’s undeniably true that you have to design and develop with disabilities in mind, however, the benefits of accessibility extend far beyond that. Apps and websites designed with accessibility principles in mind are also more usable for people without disabilities. Just think about a website that in addition to being able to be operated by mouse, also works well with the keyboard. This makes the site more inclusive by allowing non-mouse users to access the site, but at the same time makes it more usable for all users by enabling proper keyboard navigation.

A common misconception is that a11y is only for a small minority of users. This is not true. According to the World Health Organization, people with disabilities make up about 15% of the world’s population in 2020. Anyone without a disability can acquire one at any given time or gradually, be it temporary or permanent. Accessibility shouldn’t be a nicety added on as an afterthought for people with disabilities. Without accessibility, we directly negatively impact users because they can’t use our products.

When we set ourselves the goal of creating accessible products, the goal of this article becomes to limit as much as possible, during the engineering phase, to ship things take cause barriers for people with disabilities. It is important to mention however that accessibility should be part of all processes in the organization, of every discipline related to creating web products: not just development, but also graphic design, UX (research), copywriting, QA, business management, and HR.

“Without accessibility, we directly negatively impact users because they can’t use our products.”

There are a lot of open-source accessibility testing APIs out there that we can use in different phases during the development process, so we don’t have to write rules for checking a11y ourselves. The rules of these APIs are based on the WCAG (Web Content Accessibility Guidelines) specification and support checking different levels of conformity as well as some general accessibility best practices. Most of the libraries we will talk about in the following sections are based on the axe-core testing engine.

The main questions that we will aim to solve with these automated checks are:

How do you know what you’ve built is accessible, and,
How do you know that your changes don’t break anything (i.e., make something inaccessible)?

Linting

Linting allows us to do static checks on our code during development, on commit, or before builds in a pipeline. You should already be using a linter, most likely ESLint, to find problems with your code, so baking a11y standards into your application in real-time becomes real’ easy.

Configuring a11y linting:

For React projects, you can use the following ESLint ruleset to extend your ruleset to enable static checking for accessibility violations or problems.

eslint-plugin-jsx-a11y

Note that the eslint-config-airbnb or create-react-app rulesets (and possibly others) extend eslint-plugin-jsx-a11y so you don’t need to include it if you are using this configuration.

In editor a11y linting hints

You can configure your IDE to pick up your linting rules and to give you hints about violations during your editing experience like in the image above. Use the VSCode ESLint extension to automatically pick up your project’s ESLint configuration file to enable linting warnings and errors for VSCode.

In addition to in-editor hints, you can also configure your linter to run programmatically over on (a subset of) your files. With ESLint you can run for example:

eslint src/**/*.{ts,tsx}

to lint all the matched files whenever you want, for example, before a commit or before a build.

Linting limits

CSS can impact the end result
Rendering of your code makes a difference
Rules sometimes are subjective or situational

The previous factors should be considered when using linting. Static linting does not know about what the end-user sees which can impact the actual accessibility of your final product, so linting alone is not enough.

Runtime linting

As seen in the previous section, render result is a huge limiting factor for linting. The @axe-core/react library allows you to run a11y linting on the rendered DOM of your React app during development. It will log any issues to the Chrome DevTools console and will point to the exact elements affected as well as provide links to documentation on the issue.

Example of @axe-core/react a11y issue logging in the Chrome DevTools during development.

Unit testing

Unit testing allows us to test different components of our web applications in isolation. You should already be doing unit testing as a way of assuring quality: making sure your component’s API keeps doing what you expect it to do. We can test the accessibility of UI components in the same way.

With Jest’s JSDOM environment we have access to the DOM which allows us to check for accessibility. You could for example check for the following:

Focus states
Key presses and other interactions
Text alternatives
Aria attributes
Element visibility

Configuring a11y unit tests

The easiest way to do a11y unit tests for React components is by using Jest as a testing framework and @testing-library/react for testing utilities, which contains element selectors that encourage best practices and inclusive, accessible, code.

Asserting

Jest’s expect API gives you access to several matchers to validate different things. However, this set of matchers is limited and will force you to write some custom code for repetitive patterns when checking the DOM for all kinds of things, including a11y features.

jest-dom: The @testing-library/jest-dom library provides a set of custom jest matchers for DOM-related checks, that you can use to extend jest. These will make your tests more declarative, clear to read, and to maintain.
jest-axe: The jest-axe package provides a custom matcher to assert validity on an axe violation check on your HTML markup.

Example unit tests

The following snippet showcases some a11y unit tests from one of our codebases, using jest-axe and testing-library. Of course, your implementation of unit tests will depend on what you’ve built.

/**
 * @jest-environment jsdom
 */
import { ThemeProvider } from '@material-ui/core';
import { act, fireEvent, render } from '@testing-library/react';
import { axe } from 'jest-axe';
import React from 'react';
import theme from '../../../../theme';
import { TextFormControl } from '../TextFormControl';

describe('TextFormControl', () => {
  it('creates an accessible form control', async () => {
    const onValueChangeSpy = jest.fn();
    const { getByRole, container } = render(
      <TextFormControl
        labelText="label text"
        id="test-id"
        value="currentValue"
        onValueChange={onValueChangeSpy}
      />
    );

    const results = await axe(container);
    const input = getByRole('textbox');

    expect(results).toHaveNoViolations();

    expect(onValueChangeSpy).toHaveBeenCalledTimes(0);
    act(() => {
      fireEvent.change(input, { target: { value: 'new value' } });
    });
    expect(onValueChangeSpy).toHaveBeenCalledTimes(1);
  });

  it('has accessible help and error text', async () => {
    const { container, getByRole } = render(
      <ThemeProvider theme={theme}>
        <TextFormControl
          labelText="label text"
          id="test-id"
          value="currentValue"
          onValueChange={jest.fn()}
          helpText="help me!"
          hasError
          errorText="I have an error, fix me!"
        />
      </ThemeProvider>
    );

    const input = getByRole('textbox');

    expect(input).toHaveAttribute(
      'aria-describedby',
      'test-id-helpText test-id-error'
    );

    const results = await axe(container);

    expect(results).toHaveNoViolations();
  });
});

Integration testing

When a component or layout (a composition or combination of different units) has so many concerns that all parts unit-tested individually cannot assure that we meet requirements, we must resort to integration testing. The question here is: are our components interoperable? Do they work well together on a page and is the end result still properly accessible?

Integration tests allow you to do the following:

render and test in a real browser
check for document and page-level rules (i.e., colour contrast foreground-background, head tags, landmark usage, heading order)
component interoperability (i.e., can I still pass focus from my custom dropdown to the next tabbable item properly?)

Configuring Cypress a11y testing

Cypress is one of the main integration or end-to-end testing frameworks. Cypress has a handy plugin for testing for accessibility based on axe-core called cypress-axe.

Once you have this added to your cypress setup, you can call the `checkA11y` method to check for a11y violations at various points during your test. For example:

After the initial render of the page
After interacting with widgets on the page, for example, opening a modal
After async data has loaded and new content is rendered

These checks will give you insights into a11y violations and warnings and give pointers on how to fix them. Of course, you customize rules and configurations as needed. Please see the plugin documentation for more information.

Lighthouse

Lighthouse by Google is an automated tool for improving the quality of web pages. It has several audits and generates reports including one for a11y, which can be used to detect a subset of accessibility issues. Lighthouse runs against any web page and can be launched in multiple ways:

From the command line,
As a Node module,
As part of your CI process,
From the Chrome DevTools,
From a browser plugin,
From a web UI

You can use Lighthouse to check your site for issues during development and integrate it into your CI process to prevent regression.

During development, simply use the Chrome DevTools Lighthouse tab to run against your development web server URL to generate reports and fix issues.

Running from the Chrome DevTools will look like the screenshots below:

The Lighthouse tab in the Chrome DevTools will allow you to run audits on your live site for different categories, including a11y.

The generated report shows a score and a list of opportunities to improve the accessibility of your website:

Example of a Lighthouse a11y report in the Chrome DevTools with clear hints on how to improve a11y further.

Give it a spin to see if this is something you could benefit from in your project. The quickest way is the Web UI at PageSpeed Insights.

Browser (plugins) & other tools

There are other browser plugins like Lighthouse that you can leverage during development to check for a11y. These plugins are also useful for less technical users to check violations on live sites.

Axe DevTools

Axe DevTools allows you to run axe-core directly in the browser to generate a report of (a part of) your page. The report includes detailed information about violations and links to more information about how to solve them.

WAVE

The web accessibility evaluation tool (WAVE) browser plugin will allow you to test a11y directly in the browser and will identify many a11y and WCAG errors. The tool will report and highlight issues and explain how to fix them.

Accessibility Insights for web

Accessibility Insights for Web is a browser extension for Chrome and Edge that helps developers find and fix accessibility issues in web apps and sites. It provides several handy functionalities:

A “Fast Pass” automated checks reporting of most common issues
Tab stops visualization tool that allows you to visually see how the tab stops work and are ordered on your site, allowing you to identify issues with incorrect tab order or missing tab stops.
A manual tests checklist that provides step-by-step instructions on how to test and fix issues on your page. Many of the tests have visual helpers that highlight the elements in question to review.

Accessibility Tree inspector

In most browser’s DevTools you have an “Accessibility” tab in the “Elements” inspector where you can inspect the parsed accessibility tree, which is very useful for developers to debug element roles and descriptions when working with HTML tags and aria-attributes.

What can’t we automate yet?

According to the UK Government Digital Service, roughly 70% of issues can be caught using automated testing tools, depending on what tool and rulesets you are using. They recommend using manual testing as well because tools cannot be relied on by themselves to check the accessibility of a website completely.

Current limitations:

Event handlers: There is no standard way to detect event handlers on non-interactive elements like divs, but there are many ways to attach events to elements.

Screen readers: screen readers are an issue because there is a great diversity of external screen reader software that all control and read the browser DOM in slightly different ways. Why you can’t test a screen reader yet by Rob Dodson explains concisely why automated tests for screen readers are problematic. He recommends focusing on making sure our accessibility trees are good rather than on what each different screen reader might read out.

Note: Currently there is automation support for controlling the VoiceOver screen reader on macOS through VoiceOver.js and autoVO. However, as VoiceOver is only available on macOS, this would make it impossible on your colleague’s windows machine or to run as part of your pipeline, unless your ci runs on a virtual machine that runs AppleScript.

A final note on (automated) screen reader testing: a common consensus seems to be that sighted developers or non-regular screen reader users shouldn’t be manually testing your application with screen readers for accessibility. The same could be said for writing automated screen reader tests. How will you know if the automated test actually tests for proper accessibility if you don’t know how a screen reader user would use the application?

What shouldn’t be automated?

As shown in the previous section, there are currently many limitations on automated testing for assistive technologies such as screen readers. A certain part of a11y checking should always be done manually by specific users. Smashing Magazine has a great resource on the Importance of Manual Accessibility Testing. It is recommended to test with real users: people with disabilities that are experts at using assistive technology. You can either do this in-house or get help from specialised companies that provide accessibility testing services like Fable.

Wrap up

We have reviewed the why and how of implementing and using automated checks for accessibility during the development phase. By leveraging various tools and techniques, you can get more insight into the level of accessibility of your React web app and it will help you address issues in a focused way. However, accessibility should be an organisation-wide effort to make sure that products such as applications and websites are as accessible and inclusive as possible from the beginning.

Make your React web app more accessible today!