Real user centric testing goes a long way in ensuring a quality rich product on release. Today, end users are willing to actively partake right from the early stages of product development in providing valuable inputs to product teams. While this is a nice to have aspect that goes a long way in smart engineering from early stages, it is not really a mandate that all teams take on. There are however certain quality attributes where real user testing almost becomes inevitable. Accessibility testing, specifically web accessibility testing is one such quality attribute that is not fool proof without real users. As we know real users here are differently abled users, who may have issues with vision, hearing, speech, cognition, motor activities, as some of the core ones.
At QA InfoTech, we have been helping several global organizations with a solid accessibility testing strategy for the last decade now. We have a team of visually challenged users, with disabilities of a varying spectrum. We have taken real user testing to the next level in accessibility, with a unique proposition called, Accessibility with Paired Testing Approach. Herein the pair involves an accessibility subject matter expert (SME) and a real user (a differently abled user) who has been trained in software quality and testing. In our experience, of having worked on several projects for accessibility testing, an SME alone on an average is able to bring in about forty percent test coverage, a real user tester individually is able to bring in about sixty percent coverage, whereas when they work as a pair they are able to bring in fool proof coverage, based on track of post-production defects. Herein, it is not that they work as a pair throughout the accessibility testing effort. They individually focus on their areas of competence, while in specific areas work as a pair. Let’s take a quick look at what they include.
A few issues that are better surfaced by real users include, unexpected focus changes in screen readers, the order in which content is read out, performing timed activities etc. – in scenarios such as these sighted SMEs often make assumptions and may miss issues, making them the most relevant to be handled by differently abled users.
Looking at what issues are best handled in a paired manner, testing for hidden issues is a main piece – these include hidden text, hidden links. The real user may assume these to be a part of the web page and attempt access, resulting in issues. Similarly, elements that are available on screen but may be skipped screen reader are best tested in paired approach. These are especially the case when retail applications are tested with price promotions, strike through in UI etc. The other low hanging but significant piece of web accessibility testing is the correctness of alt text values. An SME can leverage a tool such as AXE to verify the presence of absence of alt text, but the correctness of the values for each element is best tested as a pair.
Putting together a custom accessibility test strategy for each application under test, keeping a combination of SME, real user and paired tester approach has helped us bring in an edge in enabling several global products become digitally inclusive – we are proud of our accessibility team and continue to invest in newer and smarter test approaches including automation in accessibility using AI, ML – accessibility on one hand is a business offering – but on the other, is also very important to us in helping build digitally inclusive global solutions, enabling us be a responsible corporate citizen.