1. IT Accessibility Coordinator Summit Automated Web Testing
Department of Health
Robert Brutus
My Name is Robert Brutus. In December of 2016, following recommendation of policy 188, I became the IT Accessibility coordinator for Department of Health.
Since this program started, I have had the chance to review several tools available on the internet; specifically the ones found at the W3C site. Web Accessibility Evaluation Tools list .
Like most state agencies, DOH has a lot of applications that need to be evaluated. Our thinking was, we need to find a way to quickly find out: What is the current accessibility status of our applications. We knew trying to do this by hand would not easily be done, nor do we have the time/resources to accomplish that. We needed to find a more automated process or tool. After doing some research and testing several tools, we found a company named PowerMapper Software located in Edinburgh (UK). They make a product named SortSite. SortSite is currently being used in more than 50 countries by some of the world’s largest organizations, according to their web site.
SortSite is a tool that stood out to us and gave us good results. To date we have scanned multiple applications at DOH and the results have been informative and helpful is assessing the accessibility and in assisting us with what to fix. I had the opportunity to share our experience with this tool with Ryan Leisinger and he asked me to share with you the features of the tool.

2. Our Challenges Lots of applications to be evaluated (300+)
Needed to find:
Current status of our applications – quickly
Time and resources - Big factor
List of tools suggested by W3C
Web Accessibility Evaluation Tools list
What SortSite is not
It’s not a catch all, and still requires some manual intervention, and testing to confirm the results.

3. Solution After Research …. One application stood out
PowerMapper Software located in Edinburgh (UK).
Gave us good result
We have scanned multiple applications
Results have been informative and helpful in assessing what to fix
What SortSite is not
It’s not a catch all, and still requires some manual intervention, and testing to confirm the results.

4. SortSite Cost Standard Professional Developer Cost
$149 single user license
$349 single user license
$849 single user license
License type
Perpetual
All upgrades
$37.25 optional annual fee
$87.25 optional annual fee
$ optional annual fee

5. SortSite Overview Overview Checkpoints Compatibility Errors
Accessibility
Optimization
Usability
Standards
Scanning Options
Choosing Rules
Password Protected Sites
Dashboard
Reports
Issues
Inventory
Import
Export
Blocks Robots.txt
Broken links
Recording
Backup and Restore
Scheduling Job
What SortSite is not
It’s not a catch all, and still requires some manual intervention, and testing to confirm the results.

6. Who is using SortSite? Government Center for Disease Control
US Environmental Protection Agency
Library of Congress
US Marine Corps
NASA
US National Science Foundation
US Air Force
UK Central Office of Information
US Dept. of Energy
UNICEF
US Dept. of Interior
Woods Hole Oceanographic Institute
According to the company website

7. Who is using SortSite? II
Aerospace and Defense
Financial Services
Boeing
AXA
Honeywell
Bank of America
Lockheed Martin
Bank of New York
Northrop Grumman
Fannie Mae
Raytheon
Fidelity Investments
Nationwide
Wells Fargo
Aerospace and Defense and some financial services group

8. How it works Scan each page of a site by running Checkpoints
Examine the page for links
Scan newly found pages
Repeats the process
Pages are requested same manner as a web browser
Process is web crawling
Same used by search engine like Googlebot to find index.
You can add pattern to ignore *.PDF
Googlebot is the search bot software used by Google, which collects documents from the web to build a searchable index for the Google Search engine.

9. Checkpoints Part I
Each page is checked for quality issues using over 700 checkpoints.
Checkpoints are grouped into guidelines (e.g. Google Webmaster Guidelines or Section 508 Accessibility Guidelines)
Check points can be enabled or disabled as a group.
Individual checkpoints can also be enabled or disabled

10. Checkpoints Part II Missing and placeholder ALT tags
Links only accessible via Flash movies
Poor text contrast
Animated images and sounds that cannot be paused
Pages without structural markup
Elements behaving unexpectedly when keyboard is used for navigation

11. Errors Check for over 40 functional problems
Broken links and anchors in HTML,CSS, Flash, PDF
Missing or corrupt images
Configuration issues like inconsistent MIME
Domains about to expire
Placeholder like “TODO”
Inappropriate language
Disclaimer and analytics tags
MIME (Multi-Purpose Internet Mail Extensions) is an extension of the original Internet protocol that lets people use the protocol to exchange different kinds of data files on the Internet: audio, video, images, application programs, and other kinds, as well as the ASCII text handled in the original protocol, the Simple Mail Transport Protocol (SMTP). In 1991, Nathan Borenstein of Bellcore proposed to the IETF that SMTP be extended so that Internet (but mainly Web) clients and servers could recognize and handle other kinds of data than ASCII text. As a result, new file types were added to "mail" as a supported Internet Protocol file type.
Cascading Style Sheets (CSS) is a style sheet language used for describing the presentation of a document written in a markup language.
Domain

12. Browser compatibility
Check your HTML, CSS, JavaScript and images work with desktop and mobile browsers:
Internet Explorer
Firefox
Safari
Opera
Google Chrome
Check for over 150 issues including:
Use of HTML tags not supported by other browsers
JavaScript and DOM properties and methods that trigger bugs
Images that don’t display e.g.
PNG files that are 4097 or 4098 don’t display in IE6

13. Spellchecker
Check for spelling mistakes in:
English
French

14. Accessibility
Check that disabled users can access your site using national and international accessibility standards:
W3C WCAG 1 Levels A, AA, AAA
W3C WCAG 2 Levels A, AA, AAA
Section 508
Reading Age
PDF/US – Matterhorn Protocol
Matterhorn Protocol:
To promote adoption of PDF/UA by software developers and document testers alike, the PDF Association’s PDF/UA Competence Center developed the Matterhorn Protocol, a list of all the possible ways to fail PDF/UA.

15. Search Optimization
Check against search engine guidelines and best practices: Google, Yahoo, Bing, Keyword optimization
Check for over 60 issues including:
Duplicate and missing page titles
Dynamic pages with too many URL parameters to index
Pages with hidden text which can trigger ranking penalties
Presence of specified keywords in critical areas like page titles and headings

16. Web Standards Check over 50 user experience problems
Check against usability Standards
Usability.Gov guidelines
W3C Guidelines
Readability Guidelines
Check over 50 user experience problems
Slow loading pages
windows or Mac specific fonts
Page with no Navigation options
Page requiring University level reading skills
Unsolicited popup
Using underlined text that look likes link
Long URL etc.

17. Scanning options Check Entire Site example: http://www.doh.wa.gov
Check Folder
Every page within a particular folder
Example:
Check Page
Check Page and Links
Check …. Specify additional pages
If you choose to scan entire site, for large site, this can take a very long time
If the site is really large with thousands of pages, things can get locked
External Links
All scans check for valid links to external sites, unless explicitly told not to
( Options  Reports)
You can turn off the option to check external link to save time
You can tell SortSite to check a certain number of links
Checking External Links is a good valid test to ensure your external links are working, but it does add a significant amount of time in your scanning.

18. Choosing rules Scan against current 508 Standards
Scan against WCAG standards – Level A – AA – AAA
(Check  Choose Rules)
The Choose Rules window allows you to import and export rules, allowing you to share common settings with other team members. This allows a team lead to define a corporate standard, then distribute to other people on the team.
SortSite is itself a web browser with some specialized tools built-in
Enter the URL in the address bar
Scan Just a folder
Options available
Check all external Links
Check first 20 links to each external site
Advanced
Explore entire site
Only explore specific pages (3) clicks from the start page
Check  Choose Rules
Checking errors
Choose which guidelines
Up to priority : A, AA, AAA (At a minimum Choose AA)
The report will tell you which level
Other options include

19. Password Protected Login page
Login to the site in the usual way (either via a login page or the popup password dialog)
Once you’ve logged in, navigate to the page you want to start scanning from (usually the page shown after login, or the home page of the site) then map or check your site
*logout*
This stops the scanner following links with ‘logout’ in the URL.
: If you have a logout link inside the secure area, then the scanner may be logged out if it follows the logout link. To prevent this happening go to Block Links on the Links tab of the Options window and add something like the following:

20. Windows Authentication
Go to Options on the View menu, go to the Links tab, and make sure the Automatic Login box is checked (this option is not present and always set in SortSite 3.0 and PowerMapper 5.01 and earlier)
Now navigate to the page you want to start scanning from then map or check your site
: If you have a logout link inside the secure area, then the scanner may be logged out if it follows the logout link. To prevent this happening go to Block Links on the Links tab of the Options window and add something like the following:

21. Blocks Allows you to prevent pages being visited during the scan.
Obey Robots.txt When checked the web crawler will obey this protocol.
Blocked Links Allows you to specify URL patterns for pages to block. The syntax is the same as used in robots.txt. Examples: Blocks a single page Blocks all pages in stayout
Blocks all HTTPS links
*.doc Blocks links to all pages with a .doc extension
*print_friendly.htm Blocks links to URLs ending in print_friendly.htm
*action=edit* Blocks links containing action=edit
The crawler only visits pages on the same domain as the home page, so pages on a different domain do not appear on the map. To add other domains to the same report or sitemap, select the Options command from the View menu, and add the additional domain names to the Additional Domains box in the Links tab.
Some pages were blocked by the Robot Exclusion Standard (robots.txt) or explicitly blocked in the Blocks tab on the Options window
To find out which links are blocked by robots.txt for a site, for example, open the address If you get a 404 Not Found message then no links are blocked, if you get a text file back this lists which links are blocked.
Adding the following entries to the top of the robots.txt file, before any Disallow: directives, will bypass any blocks intended for other web crawlers:

22. Links Allows you to control which links are followed during the scan.
JavaScript DOM Changes
Lets you choose whether links in JavaScript are extracted, and changes to the DOM are checked.
None – changes made by JavaScript are ignored
Smart – important changes made by JavaScript are checked
All – all DOM changes are applied, including analytics
Follow links to related domains When checked the scanner will follow links to child domains (e.g. if you scan domain.com then links to store.domain.com and will be followed) and sibling domains at the same level (e.g. if you scan then links to secure.domain.com and www2.domain.com will be followed).
Automatic Login Check this box if you need to scan password protected sites.
Follow links to additional domains Links to any additional domains listed in this box will be followed.

23. Reports Options Maximum pages shown per issue
Maximum line numbers shown per issue
Checking links to other sites
Allows you to specify settings control how many issues are reported.
Maximum pages shown per issue The number of pages an issue is reported on in detail. This reduces noise from issues in page templates that affect every page of a site.
Maximum line numbers shown per issue The number of lines per page an issue is reported on. This reduces noise from issues like unescaped ‘&’ characters that can occur on nearly every line of every page of a site.
Checking links to other sites Either unlimited, or the number of outbound links checked per external site. Checking lots of links to social book-marking sites can result in your book-marking account being suspended temporarily.

24. Reports Reports are broken down into a number of tabs and sub-tabs:
Summary
o Dashboard overview of site issues
o Site Map shows distribution of issues across site
o What was Tested summary of what was tested
Issues
o Errors technical errors such as broken links
o Accessibility WCAG and Section 508 accessibility
o Compatibility browser compatibility issues
o Compliance [Pro] legal compliance problems
o Search [Pro] search engine guidelines & best practice
o Standards W3 validation and technical standards
o Usability [Pro] usability issues
Pages
o Inventory inventory of pages, images, scripts etc.
o Issues by Page issues grouped by page

25. Recording
Navigate to the page containing the form you want to record. For example,
Select Record Form Input from the Check menu to start recording.
Type your test values into your form and submit it. This will display a new page with the results of the form submission (e.g. a page of search results). This new page will now be scanned each time the site is scanned.
Select Record Form Input again to stop recording. The recorded form action will be re-submitted each time is scanned, and the results of form submission checked for problems.
Technical details: the Record Form Input command intercepts form GETS and POSTs and records (in encrypted form) the form fields in the post. Each time the page containing the form is visited during future scans, the form fields are re-submitted and the HTML returned by the GET or POST checked for problems.

26. Backup and Restore
SortSite stores site structure and issues in a binary .schk file, with generated reports stored in an associated folder with the same name as the .schk file (minus extension).
Files are only held open while reads and writes are in progress so saved data can be archived and restored using any backup solution.
Technical details: the Record Form Input command intercepts form GETS and POSTs and records (in encrypted form) the form fields in the post. Each time the page containing the form is visited during future scans, the form fields are re-submitted and the HTML returned by the GET or POST checked for problems.

27. Job Scheduling You can schedule a task (File  View Schedule) Once
Daily
Weekly
Monthly
At System Startup
At Logon
When Idle
Provide Start and end date
Repeat task
Specify Duration – if the task is still running, stop it at this time
Show multiple schedules
New Task is disabled until a site has been scanned or loaded from disk. Use the Options button next to each task to edit, delete or rename the task

28. Import and Export
Reports can be exported using the Export command on the File menu. Supported formats include:
HTML
Word RTF
Excel XML
Technical details: the Record Form Input command intercepts form GETS and POSTs and records (in encrypted form) the form fields in the post. Each time the page containing the form is visited during future scans, the form fields are re-submitted and the HTML returned by the GET or POST checked for problems.

29. Inventory
List of pages and content on site broken down by type and technology.
Clicking the expansion button next to each file type shows a list of URLs of that type.
Clicking on URLs in the inventory displays the Source View for each URL.

30. Microsoft Engineering Excellence
DEMO  Sample
Suggested Testing Process
Run SortSite until WCAG 2.0 is reached
Do human testing or functional testing using the following script
Additional usability testing can be done using WaTech Lab or reaching out the blind community
Caret Browsing is a feature of Firefox which allows you to navigate a page just like a document in MS Word or any other word processing application. You can move the cursor between lines using arrow keys and can select the text or click on any link using your keyboard.
Microsoft Confidential

31. Microsoft Engineering Excellence
Questions? Robert Brutus CSM IT Accessibility Coordinator Applications and Data Services Office (ADS) Washington State Department of Health Desk:        Mobile:   Public Health – Always working for a Safer and Healthier Washington
Microsoft Confidential