1. Bring Accessibility into the Development Lifecycle with CI Testing
Alistair Garrison
Director of Accessibility Research

2. Formerly Known As SSB BART Group
Who is Level Access?
Formerly Known As SSB BART Group
New Corporate Identity – Name and Branding
New Website – Coming Spring 2017!!
No Change in our Innovative Technology or Excellent Service
CSUN 2017 Assistive Technology Conference

3. What is Continuous Integration?
Continuous Integration (CI) is a development practice;
It requires developers to check-in their code to a shared repository – ideally several times a day.
Each check-in forces an automatic re-build; then re-test of the software (from the integrated code)
So integration issues can be detected immediately.
CSUN 2017 Assistive Technology Conference

4. CSUN 2017 Assistive Technology Conference
CI process flow…
Developer develops
Checks-in changes
CI fetches changes
CI builds
CI tests
CI notifies – Build Success / Failure
CSUN 2017 Assistive Technology Conference

5. So, let’s see a perfect-world example…
The ACME Corporation wants a web page for selling its products;
CSUN 2017 Assistive Technology Conference

6. So, let’s see a perfect-world example…
To interested customers (who previously had to use a mail order catalog);
CSUN 2017 Assistive Technology Conference

7. The Product Owner, a Designer and a Business Analyst sit down to talk…
The Business Analyst (BA) asks questions;
The Product Owner states what they want;
The Designer draws ideas;
CSUN 2017 Assistive Technology Conference

8. CSUN 2017 Assistive Technology Conference
After a short while…
Concepts / ideas start to solidify…
They want:
A web page;
Pictures of their products;
With show / hide buttons above each picture; to reveal more product information.
CSUN 2017 Assistive Technology Conference

9. CSUN 2017 Assistive Technology Conference
So…
Designer creates a wire- frame of page;
BA creates user stories for the show / hide function (component-level);
BA creates user stories for completing a task on the page (integration-level);
CSUN 2017 Assistive Technology Conference

10. For clarity - a user story is…
A description in the everyday language of the end-user that captures what a user does, or needs to do, in order to complete a task.
CSUN 2017 Assistive Technology Conference

11. User stories: special format…
In our example, the ACME Corp uses the Gherkin format when creating user stories.
Given (the situation);
When (I do something or something happens);
And (optionally I do something else);
Then (this is what I expect has happened)…
CSUN 2017 Assistive Technology Conference

12. Component-level acceptance test
Example, from ACME…
Component-level acceptance test
Scenario 1: Show / Hide Vanilla Widget – Show state.
Show X
CSUN 2017 Assistive Technology Conference

13. Component-level acceptance test
Example, from ACME…
Component-level acceptance test
Given the Show / Hide Widget button is focused;
And the Show / Hide Widget button has the value “Show X”;
Show X
CSUN 2017 Assistive Technology Conference

14. Component-level acceptance test
Example, from ACME…
Component-level acceptance test
When the Show / Hide Widget button is activated;
Show X
CSUN 2017 Assistive Technology Conference

15. Component-level acceptance test
Example, from ACME…
Component-level acceptance test
Then the Show / Hide Widget button has the value “Hide X”;
And the Show / Hide Widget panel is visible;
And the Show / Hide Widget panel contains the text “X”.
Hide X X
CSUN 2017 Assistive Technology Conference

16. Integration-level acceptance test…
Example, from ACME…
Integration-level acceptance test…
Scenario 1: Show product details
Basically, the vanilla widget is used in context.
Show product details
CSUN 2017 Assistive Technology Conference

17. Integration-level acceptance test…
Example, from ACME…
Integration-level acceptance test…
Given the Show / Hide button is focused;
And the Show / Hide button has the value “Show product details”;
Show product details
CSUN 2017 Assistive Technology Conference

18. Integration-level acceptance test…
Example, from ACME…
Integration-level acceptance test…
When the Show / Hide button is activated;
Show product details
CSUN 2017 Assistive Technology Conference

19. Integration-level acceptance test…
Example, from ACME…
Integration-level acceptance test…
Then the Show / Hide button has the value “Hide product details”;
And the Show / Hide panel is visible;
And the Show / Hide panel contains the text “Super magnet”.
Hide product details
Super magnet
CSUN 2017 Assistive Technology Conference

20. FYI, in terms of development methods ACME Corporation uses…
BDD - Behavior-driven development;
BDD specifies that software should be built / tested in terms of its desired behaviors.
Where "desired behaviors" are generally defined through user stories in a special format e.g. Gherkin.
CSUN 2017 Assistive Technology Conference

21. ACME uses Cucumber for BDD testing…
With Selenium Webdriver + Assertion Libraries e.g. Chrome Webdriver / Chai;
But, it could have chosen loads of other frameworks…
CSUN 2017 Assistive Technology Conference

22. So, turning user stories into Acceptance Test for Cucumber…
The user stories formatted in Gherkin are associated with code functions e.g. step definitions, page objects, etc… so they become Acceptance Tests.
CSUN 2017 Assistive Technology Conference

23. CSUN 2017 Assistive Technology Conference
So, we now have…
Designs;
Acceptance Tests (Gherkin);
BDD Component-level acceptance tests;
BDD Integration-level acceptance tests;
CSUN 2017 Assistive Technology Conference

24. So where do they fit into ACME’s CI process?
Pre-Check-In
CSUN 2017 Assistive Technology Conference

25. In the pre-check-in process…
Continuous Integration
Update from Version Control
Build (local)
Inputs
Designs
Component-Level Acceptance Tests
Test: Unit tests + Acceptance Tests
Code
Test
Check-In
CSUN 2017 Assistive Technology Conference

26. CSUN 2017 Assistive Technology Conference
And…
Post-Check-In
CSUN 2017 Assistive Technology Conference

27. In the post check-in test phase…
Continuous Integration
CI runs all unit tests (fast);
CI runs a smoke test of acceptance tests (slow);
tests that cover the highest traffic tasks / journeys;
happy paths;
CI runs all acceptance tests (nightly).
Unit Tests
Smoke Test
Test type
Nightly
Number of tests
CSUN 2017 Assistive Technology Conference

28. So, how do we bring Accessibility testing into this process?…
CSUN 2017 Assistive Technology Conference

29. Accessibility testing…
Accessibility testing is effectively an extension of User Acceptance Testing (UAT).
Ensuring:
Users – including defined groups of disabled & older users;
Using – defined assistive technologies, browsers, and operating systems;
Can – effectively and efficiently undertake necessary journeys within a product (e.g., web site or web application);
To – complete all core goals.
CSUN 2017 Assistive Technology Conference

30. Moving forward, it is not…
In the context of Continuous Integration
Simply putting the url of a page into an on-line tool; and pressing test – in order to check:
The source code; or
The initial DOM state of the page (post-load).
As, a user’s journey through a page can result in many DOM stages; and each state needs all its components to be accessible in order for a user to have an accessible journey;
CSUN 2017 Assistive Technology Conference

31. In today’s development pipelines it is…
Accessibility Testing
The integration of stand-alone Accessibility Testing Libraries with QA Frameworks for BDD, Unit Testing, etc…
To allow each DOM state configuration attained through-out a user story (component level / integration level) to be tested for accessibility.
CSUN 2017 Assistive Technology Conference

32. Where Accessibility Testing libs are…
Stand-alone libraries of accessibility checks – often coded in JavaScript (e.g. Level Access’s Access Engine).
Which can be integrated with loads of testing frameworks.
Called through APIs which allow all tests to be run, single tests to be run, etc.
CSUN 2017 Assistive Technology Conference

33. Methods of injecting a lib…
Injecting lib into web page
Adding the JavaScript through a <script tag…
Achieved dynamically through JavaScript in (Cucumber) step definitions;
Through a manual testing toolbar;
CSUN 2017 Assistive Technology Conference

34. Methods of injecting a lib…
Inserting lib into mobile build
For Mobile it can be better to “instrument the code” – in effect, include the testing lib in the non-release code.
So it’s baked-in from the start.
It’s just a little easier if you’re testing with frameworks like Appium – driving real phones.
CSUN 2017 Assistive Technology Conference

35. So, let’s see a perfect-world example again…
The ACME Corporation wants a web page for selling its products;
CSUN 2017 Assistive Technology Conference

36. So, let’s see a perfect-world example again…
To interested customers (who previously had to use a mail order catalog);
CSUN 2017 Assistive Technology Conference

37. The Product Owner, a Designer and a Business Analyst sit down to talk…
The Business Analyst (BA) asks questions;
The Product Owner states what they want;
The Designer draws ideas;
CSUN 2017 Assistive Technology Conference

38. CSUN 2017 Assistive Technology Conference
After a short while…
Concepts / ideas start to solidify…
They want:
A web page;
Pictures of their products;
With show / hide buttons above each picture; to reveal more product information.
All to be accessible!
CSUN 2017 Assistive Technology Conference

39. CSUN 2017 Assistive Technology Conference
Why…
Amongst other reasons…
One of their very good customers has a lot of temporary disabilities.
CSUN 2017 Assistive Technology Conference

40. CSUN 2017 Assistive Technology Conference
So, they…
Determine the accessibility requirements their aiming to meet;
Select an Accessibility Testing Library that integrates with their BDD testing tools;
Define relevant design pattern of behaviors (in terms of accessibility) for their page / widgets.
CSUN 2017 Assistive Technology Conference

41. CSUN 2017 Assistive Technology Conference
The…
Designs are altered to ensure accessibility is considered;
Component-Level Acceptance tests are extended to include calls to the Accessibility Checking Library; and additional behaviours (in terms of accessibility).
CSUN 2017 Assistive Technology Conference

42. CSUN 2017 Assistive Technology Conference
Component-Level
Accessibility acceptance test using Accessibility Checking Library (BDD format)
Scenario 1: Accessibility Test Point - Show / Hide Widget – Show state.
Given the Show / Hide Widget button is focused;
And the Show / Hide Widget button has the value “Show X”;
When I run Accessibility Tests
Then no Accessibility Tests will fail.
Show X
CSUN 2017 Assistive Technology Conference

43. CSUN 2017 Assistive Technology Conference
Component-Level
Accessibility acceptance test using Accessibility Checking Library (BDD format)
Scenario 2: Accessibility Test Point - Show / Hide Widget – Hide State.
Given the Show / Hide Widget button is focused;
And the Show / Hide Widget button has the value “Hide X”;
And the Show / Hide Widget panel is visible;
And the Show / Hide Widget panel contains the text “X”.
When I run Accessibility Tests
Then no Accessibility Tests will fail.
Hide X X
CSUN 2017 Assistive Technology Conference

44. CSUN 2017 Assistive Technology Conference
Component-Level
Accessibility acceptance test based on behaviors that support accessibility
Scenario 3: Accessibility Behavior - Show / Hide Widget - Show.
Given the Show / Hide Widget button is focused;
And the Show / Hide Widget button has the value “Show X”;
When the Show / Hide Widget button is activated;
Then the Show / Hide Widget button has the value “Hide X”. (satisfying as status update is communicated).
And [optional] the Show / Hide Widget button has an aria-expanded attribute set to the value “true”.
Hide X
Show X X
CSUN 2017 Assistive Technology Conference

45. CSUN 2017 Assistive Technology Conference
And…
Integration-Level Acceptance tests are extended to cover additional behaviours (in terms of accessibility).
CSUN 2017 Assistive Technology Conference

46. CSUN 2017 Assistive Technology Conference
Example
Integration-level A11y acceptance test, based on behaviors that support accessibility
Scenario 1: Keyboard - Read more about product.
Given that the ACME product page is open.
And the tab key has been used to focus the “Show product information” button
When the enter button is pressed.
Then the “Show product information” button becomes “Hide product information”.
And an area becomes visible that shows “product information” for the “super magnet” product.
CSUN 2017 Assistive Technology Conference

47. CSUN 2017 Assistive Technology Conference
So, they now have…
Designs (that take into account selected accessibility requirements);
Component-level acceptance tests
Component-level accessibility acceptance tests – that include behaviours needed for accessibility support;
Integration-level acceptance tests
Integration-level accessibility acceptance tests – that include behaviours needed for accessibility support;
CSUN 2017 Assistive Technology Conference

48. What does this impact in CI?
Continuous Integration
Unit tests – no;
Functional tests:
Component-level acceptance tests - yes;
Integration-level acceptance tests - yes;
CSUN 2017 Assistive Technology Conference

49. In terms of the number of tests…
Continuous Integration
Acceptance Tests for A11y
Integration Tests for A11y
Component Tests for A11y
Acceptance Tests for A11y
Integration Tests
Component Tests for A11y
Component Tests
Component Tests
Pre Check-In
Post Check-In
Unit Tests
Unit Tests
CSUN 2017 Assistive Technology Conference

50. But, what does this bit of extra effort give you!!!
Far higher testing coverage relating to accessibility;
Far higher confidence that all users will be able to complete their journeys in an unhindered way.
= Happier users!
CSUN 2017 Assistive Technology Conference

51. Questions?