Accessibility Testing, Pt 2 – Screen-reader

Screen-reader testing is a bit more challenging at first. On the plus side, high quality, free screen-readers are readily available, such as NVDA for Windows, and Voice Over on Mac. If you follow my advice, there are not a lot of controls to learn. The tricky part is just getting used to the experience.

I still remember the first time I tried a screen-reader. It was very disorienting. It was hard to understand what was going on and how your were supposed to interact with it. For someone who is used to a visual experience interacting with the web, it is just completely foreign, and possibly panic inducing.

Luckily, this feeling fades pretty fast after a couple sessions. My advice to ease yourself into it is to start with a very simple, preferably already quite accessible page. If you are a developer, you might want to set up a simple page yourself with some basic content and a couple active elements. IF you need an accessible (though not necessarily simple) page to start with, visit a government website. They usually have at least basic accessibility in place already.

Once you’ve chosen a sample page to start with, take a moment to find out how to slow down the speech rate of your chosen screen-reader. The defaults are fairly fast, and “native” users tend to listen at a rate that is just too fast for even experienced occasional users to understand, so make it easier on yourself by starting with a slow rate of speech.

To begin, load your chosen page, and simply let the screen-reader read the page from top to bottom. Try to follow along with the reader with your eyes, seeing how various features and content of the page are rendered, and in what order. It might be tricky at first, so don’t be afraid to reload the page and start from the top again if you get lost.

This process will give you a feel for the nature of the screen-reader experience (and will probably justify to you the importance of accessible webpages for non-visual users).

When you get to real testing, it will often be convenient to have more control over the reading of the page, and the default way to do this is “arrowing through” the page. All screen readers will have line-by-line, forward and backward controls. For example, for NVDA, these are the up and down arrow cursor keys. This technique gives a fine grain of control to see how things are read, and allows you to go back and reread a line if you missed something. This is also the lowest common denominator of how real screen-reader users will read a page, especially new, relatively inexperienced users. Most of your testing will use arrowing, since it is also the most thorough way to check all the content in the page methodically.

Another useful command to learn is the one that most screen-readers have that give a list of headings and links in the page. (On NVDA, the default command for this is Insert+F7). This gives a quick way to check out how well-organized the page content is, if the links have sufficiently descriptive text to understand out of their context, and sometimes brings to light structural problems in the page.

As with the keyboard-only testing, you will want to make sure that all the links and other active elements work, and typically these are triggered with the same keys as you used with the keyboard-only testing.

For someone who just needs to be able to test pages or apps for accessibility, I would recommend resisting the temptation to learn further features of your screen-reader, for a few reasons. First, it makes is simpler to learn. Second, it is a more thorough way to test the content and functionality of your app or webpage. Third, it mimics how a beginner, who will have the hardest time navigating, will actually be using your product. And fourth, if testing works with this simple method, the chances are that the more advanced features will also work correctly for advanced users.

So that is all you really need to know to get started with screen-reader testing. With a bit of experience using these techniques, you will be able to get a very good idea of how accessible your product is, and what needs to be done to improve it.

Advertisement

Accessibility Testing, Pt1

In my previous post, I suggested that the best place to begin if you want to produce accessible websites, apps or documents is to start testing with keyboard-only input and with a screen-reader.

In part 1 of this post, I’d like to explore the effectiveness of accessibility testing in a bit more depth, and cover the basics of keyboard-only testing. In part 2, I’ll cover using a screen-reader for testing.

Why Testing is an Effective Place to Start

One of the challenges of achieving accessibility is that most people don’t relate to it, since they are so used to accessing computers visually with a mouse or touchscreen. They can’t imagine what a different approach would look like. This can make the rules of accessibility seem like abstract red tape that “doesn’t really matter”, or something which everyone has good intentions towards, but without knowing how to make sense of the applicable rules.

Testing solves this problem by giving a practical way for people new to accessibility to experience first hand the impact of inaccessible products. It also gives them a concrete goal to work towards that doesn’t require learning the subtleties of the WCAG standard up front.

Even if you do master the rules of the standard, testing is the most essential component for practical accessibility, because it is entirely possible to build a product that could be argued by a lawyer with weaselly technicalities to be compliant to the standard, but which still fails completely to be usable by the target audience.

So starting to learn testing techniques sooner in the process is also the best for the long run.

Keyboard-only Testing

For most people who are visual users of websites, apps or documents, keyboard-only testing is a relatively small adjustment. All you have to do is perform your normal test plan or test activities using only the keyboard. The one thing I would recommend is, until you get used to eschewing the mouse, it helps to physically set your mouse aside, out of easy reach, so that you don’t unconsciously grab it when you hit a snag. Using the mouse to bail you out would of course defeat the purpose of the testing, which is to ensure you don’t ever need a mouse to access functionality, or to solve a navigation problem. It is surprising how automatic much of our computing behaviour is. I’ve seen many people new to accessibility automatically revert to using the mouse midway through testing without realizing it!

The main keys that you will typically use for keyboard testing are page up and page down for accessing content, tab to access active elements in order, and likely cursor keys to manipulate active elements, such as drop downs.

Overall you want to make sure that all content and functionality is perceivable and operable just with the keys, that you don’t get stuck in any of the functionality (e.g. active elements you can’t tab out of), and that the tab order is logical and follows the visible ordering. It should also be visually obvious which item has the current focus in the tab order when tabbing.

When you start, I would recommend doing the keyboard-only testing as a separate step to make sure you cover all the bases. Once you have mastered it, and have also mastered screen-reader testing (covered next post), you can often do both kinds of testing at the same time if you want streamline the process. If you can afford the extra steps, or want to split up the work across the team, doing them separately is never bad, since it makes it easier to focus on a subset of the potential issues, which might allow more problems to be caught more quickly.

I would expect that anyone with some experience at testing should be able to master keyboard-only testing after two or three sessions, since it really is quite straightforward, and is still a primarily visual approach to interacting with the product under test.

These basic keyboard-only skills will be built upon with screen-reader testing, which I’ll cover in the next post.

The Magic Wand of Accessibility

I can’t count the number of times someone has come to me with a nearly completed document, app or website wanting me to quickly “make it accessible”. They seem to think that I will have some special accessibility dust I can sprinkle on it, or a magic accessibility wand that will just tweak everything right.

The bad news is that no such magic wand exists — if it existed, and I owned it, I would already be rich from commoditizing it. The really bad news is that, depending on how complex the implementation of your document, website or application is, making it accessible after it is completed may be as hard as tearing it down and starting over — though sometimes you get lucky and it’s only major surgery.

But there is good news: it isn’t that hard to make an accessible product from scratch. You just need to have a little bit of accessibility knowledge spread through your whole team and one or two accessibility experts to handle accessible design and finding solutions to the difficult problems that arise now and again.

In this regard, accessibility isn’t any different than any other global requirement for your project: performance, usability, security, effective content, etc. With all of these, you need to design, execute and test for these properties at all stages of development, and while there may be one or two experts to focus on this requirement, it is always assumed that everyone on the team is pitching in, being mindful of it as they do their main work.

Admittedly, it takes some time to get all these pieces into place, so there has to be a practical first step to get you there, and what I would recommend as the most effective place to start is by having testers — be they full time QA, developers unit-testing their own work, or end-user acceptance testers — to start periodically performing their testing with keyboard-only input and with a screen-reader. This testing, if done throughout the project, will catch 90% of your accessibility problems.

I hope to write a longer post soon focusing on just this topic, but for now, I recommend acquiring a free screen-reader, such as NVDA on Windows, for all your testers, get them to learn only the very basics — which I’ll cover in that upcoming post — and get them to start testing with it at all stages of development. This will make the problems visible and stimulate the acquisition of accessibility knowledge and skills through your team organically and incrementally.

I’ll repeat this as the closing moral of this post:

Testing with keyboard-only input and with a screen-reader at every stage of development is the most effective way to start creating accessible products.