Presentation is loading. Please wait.

Presentation is loading. Please wait.

Prioritizing Remediation of Accessibility Issues

Similar presentations

Presentation on theme: "Prioritizing Remediation of Accessibility Issues"— Presentation transcript:

1 Prioritizing Remediation of Accessibility Issues

2 About Me Karl Groves, Dir. of Training, Deque Systems
@karlgroves Also, a rock star*

3 Agenda What is an accessibility issue? Why prioritize?
Understanding risk Challenges Remediation Approaches Considerations Simple Prioritization Advanced Prioritization

4 Things to keep in mind I am mathematically challenged
This topic is exploratory, not declarative Please participate, ask questions, offer new ideas

5 What is an accessibility issue?

6 What is an Accessibility Issue?
Bug: Term used to describe an error, flaw, mistake, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways. We need to move beyond the impression (that we’ve created, IMO) that accessibility issues are something separate. Accessibility issues aren’t “nice to haves”, or enhancements. They are bugs. We have created a flaw in the system which harms its ability to be used by real people. Photo Credit

7 Why Prioritize?

8 Why Prioritize? Apply resources most effectively
Minimize accessibility’s impact on business Motivate development staff Maximize positive impact for users Reduction of Risk During my career, I’ve performed scores of audits and expert reviews to look at the usability and accessibility of web-based systems (websites, web-based applications, intranets, etc.). Of those, I can count on one hand the number of systems tested that were in a pre-production phase prior to the accessibility work. In nearly all cases, the testing was performed on systems that were already released to users. In such a scenario, an organization’s level of risk is at its highest, as people are using the system. Therefore the organization and its developers require sufficiently informative data output from the accessibility testing which is detailed, clear, and actionable. It doesn’t do the client any good to throw them a report filled with a bunch of accessibility violations unless that report also includes information to help them fix their problems. They need to know what their problems are, where their problems are, and how to fix them. There’s also an additional item they need to know: When to fix them. Prioritization of remediation helps to apply resources most effectively. Post-deployment bug fixes are a needless expense. We should apply resources in a manner that reaches deepest toward the greater good Prioritizing remediation helps minimize accessibility’s impact on business. Generally, accessibility is too often regarded as expensive and hard. Properly prioritized, this expense and difficulty can be significantly diminished. This diminished impact can motivate staff and help get their buy-in for both this effort and future efforts The end goal, for us as accessibility advocates is the real reason, IMO, we should prioritize: It allows us to maximize the positive impact for users. Fixing the most important stuff first will mean that we can more quickly make things better for the end user Finally, prioritization also helps reduce our risk.

9 Understanding Risk

10 Understanding Risk Risk is the potential that a chosen action or activity (including the choice of inaction) will lead to a loss (an undesirable outcome). Photo Credit

11 Understanding Risk Ultimately, remediation of bugs is an effort at risk mitigation Risks of Poor quality (Users having problem with system) Lost income Ancillary losses Administrative Complaint (public sector) Litigation Although people generally think of legal risk as the only risk when it comes to accessibility, there are many types of risks that are created by bugs in a system. Poor quality: Operability or performance problems for the end user Lost income: The volume or severity of bugs is such that users who want to buy our products or consume our content do not or cannot do so and we lose money Ancillary losses: Losses caused by spending money you don’t need to: Primarily, a11y fixes after the fact. Oops, we decided a11y is important after the fact, now we must spend money we didn’t intend to spend and that negatively impacts the money we have available to do other things Administrative Complaint: Compliance is a requirement in some industries and noncompliance raises the chance that we can have the heavy hand of regulators beating down on us Litigation: Certain industries see an increased amount of litigation surrounding accessibility, which is a significant risk. Many of these risks are related and have a multiplying effect upon our business case. Intelligently remediating our bugs will work to rapidly reduce risk

12 Understanding Risk Probability
Probability = (number of negative events) / (population) What is your probability of an actualized risk? In its simplest form: Probability = (number of negative events) / (population). Your probability of anything happening is based on the size of your population and the number of negative events within that population. In other words, if 10 people out of every 100 get a speeding ticket every 5 years, then your probability of a speeding ticket every 5 years is 10% (10/100 = .10). In terms of accessibility lawsuits, it is important to understand how to determine your population. Your population isn’t “all the other websites on the Web”, but rather it is your peer companies. How you define your peer companies involves a number of factors such as industry, site features, audience demographics, reach, and engagement in risky behavior. Note that I use the word “peer” rather than “competition”. Your competition could be a brick & mortar entity, or it could be a company that is much smaller. While they may compete in the marketplace, they are not a “peer”. Again, to use a simple analogy: Lung Cancer Probability = number of cancer cases/ number of people engaging in cancer-causing behaviors. Although everyone has some level of probability for lung cancer, people who are engaging in cancer-causing behaviors are in a very different peer group from those who don’t. Their engagement in that behavior increases their probability of the negative event. Photo credit

13 Understanding Risk Risk Amount
Risk Amount = (probability of a negative event) * (expected loss in case of negative event) From the probability, we can determine how much we are risking. Our risk amount is our probability multiplied by our expected loss: Risk Amount = (probability of a negative event) * (expected loss in case of negative event). For example, if we have a 10% probability, our risk amount is 10% of the total expected loss  (.10*totalExpectedLoss). To use an extreme case, we’ll use the Target lawsuit. Target settled with the NFB for $6,000,000 plus $4,000,000 in attorneys fees, etc. Based on their population (very large e-commerce sites) they had an 8% probability: .08 * $10,000,000 = $800,000. If you’re a company that is a peer of Target, you are risking $800,000. Some people may argue that you’re actually risking $10,000,000 because that’s what Target settled for, but that doesn’t take into consideration your probability of the lawsuit.

14 ROI = ((Risk Amount - Investment)/ Investment)*100
Understanding Risk ROI ROI = ((Risk Amount - Investment)/ Investment)*100 Where: Risk Amount = Expected loss * probability Investment = Money spent on Accessibility Your Return on Investment (ROI) for accessibility work (assuming 8% probability) is $800,000 minus the money you spend on fixing or remediating accessibility related bugs. For those who are intending to try to sell accessibility internally, it may make sense to take this into consideration when trying to gather budget for the efforts.

15 What factors impact our ability to fix our system?

16 Challenges Not all accessibility problems are equal
Time Impact Impact on Users Impact on Business WCAG Level & SC is inappropriate for determining priority The Web Content Accessibility Guidelines structures its Success Criteria in terms of “Level”. Success Criterion can be Level A, Level AA, or Level AAA. Unfortunately, their explanation of the levels isn’t terribly clear. It was much more clear in WCAG 1.0, however the WCAG 1.0 Priority levels did not take into consideration any factors other than user impact. It is clear that in the early days of work toward WCAG 2.0, they understood that conformance level needs to take more things into consideration, as evidenced by this early draft. Unfortunately, the current WCAG documentation on Understanding Conformance currently lacks a description of any kind whatsoever on the topic of what the conformance levels mean and what differentiates each from the rest. Consequently, people tend to believe WCAG 2.0’s Level is synonymous with WCAG 1.0’s Priority What is clear, despite the lack of guidance in WCAG, is that they acknowledge through the use of different levels that not all accessibility issues are created equal. Some items are more impacting on users than others. Some items are technically difficult or impossible to achieve. Some items don’t have good support among user agents or assistive technologies. As a consequence of all these factors and more, it would be rather reckless to treat everything as if they were all equal. In some cases, doing so could risk budgets while not helping the end user. Instead, remediation should be targeted to provide the best benefit quickly and cheaply. Turning back to impact, for a moment, I decided to take my own list of Best Practices and sort them by impact, giving an impact of '1' for each user population affected (Low Vision, Blind, Hearing Impaired, Mobility Impaired, and Cognitive).  In other words violating a Best Practice that impacts all of them got a '5' for impact. Violating a Best Practice that impacts only Blind people got a '1'.  Here's what I found: 5 - 2A ; 1AAA  (66% A; 33%AAA) A; 10 AA; 15 AAA (55% A; 18% AA; 27% AAA) A; 5 AA; 12 AAA (67% A; 9% AA; 23% AAA) A; 3 AA; 12 AAA (67% A; 7% AA; 26% AAA A; 11 AA; 13 AAA (68% A; 15% AA; 17% AAA) What I found was that, at least in my subjective interpretation, reliance on WCAG level alone meant that we could be including best practices that have rather low user impact. Also, we could be giving high priority to things that are difficult or time-consuming to remediate The problem is, obviously, not all best practices are created equally. "Provide a help link on each page" (mapped to 3.3.5) is definitely not the same as "Provide ability to pause, stop, or hide content which updates automatically" (2.2.2) and "Provide the ability for users to turn on sound only at their request" (1.4.2).  The latter case illustrates the need to rank the impact of individual populations. The impact of auto-starting audio for Blind people is very high and the impact on Mobility Impaired and Low-Vision persons is much higher than hearing impaired or cognitive.  I'd say 3,2,2,1,1 on that one. 

17 Challenges Time Often at a premium
Time spent on after-the-fact bug repairs is time that is taken away from meeting other business needs See, “Technical Debt”, Martin Fowler Doing bug remediation takes time. Ideally this would be time we don’t need to spend because we’d get everything right in the first place. That is often not the case. Creating bugs causes us to incur Technical Debt - a metaphor referring to the eventual consequences of poor design. This debt must eventually be repaid in some form – the remediation of bugs, for instance. The cost of that debt in this instance is the time that the repair of the bug requires. “We’ll do it later” obligates you to pay the extra time cost of actually doing it later when you should have had a commitment to quality in the first place. (Hard to stop a moving train, repairing of bugs is chasing the train) Very rarely are development staff just sitting around, looking for something to do. So the repair of bugs takes time away from doing other things. Often times, those other things are things which have a direct line on ROI: Important new features that need to ship now. Repair of bugs is a time expense and time is money, so we need to ensure we prioritize on the important things so that effort is not used frivolously. Photo Credit:

18 Challenges Impact Budgets Resources System
So, given what’s been said already: Bug repair will also cause an impact on the budgets, resources and even to the system Do these bugs cause us to require outside help from a consultant? Do we need to outsource this repair to a third party? Or, conversely, do we need to take that work away from third party and use a specific internal resource who is an expert at accessibility? Also, how will fixing this bug impact the actual system? Will it impact the presentation logic? Will it impact the business logic? Are there other features in the system or in upcoming builds that are going to be impacted by fixing this item? These are all important things that can cause challenges in determining priority

19 Remediation Approaches

20 Simple Prioritization
Focused solely on time and (simple) impact How long will it take? How bad is the problem? In the simple prioritization scheme, you focus on only two factors: Time and Impact For time: How long will it take to fix this issue? For impact: How bad is the problem for users?

21 Simple Prioritization
Pros Focused on the user Super simple Often, “hunch” from expert is as good as something more formal Cons Does not take into consideration impact on business or system

22 Advanced Prioritization
User Impact Ease & Speed Impact on Interface Volume Location Secondary Benefit Each item ranked: None (0), Low (1), Medium (2), High (3) With advanced prioritization, there are six items to take into consideration: As before: User Impact Ease & Speed of Repair PLUS: Impact on the system: Will this change the interface? Will it cause modifications to business logic Volume: How often does this issue occur? Location: Where is this problem? Secondary Benefit: Can we get any additional business boost from fixing this? In all 6 cases we’ll be ranking these factors with 4 scores: 0 – None, in other words, either it is not a consideration or there is no weight given by this consideration 1 – Low, in other words the weight for this factor is low 2 – Medium, in other words there’s a moderate weight for this factor 3 – High, in other words this has a lot of weight in this case

23 Advanced Prioritization
Impact on Users with Disabilities Broken down by type of user & impact on each IB - Blind IV – Visually Impaired (non-blind) IH – Deaf & HoH IM – Motor IC - Cognitive IS – Speech Impact* = (IB + IV + IH + IM + IC + IS) The more people are impacted – and the greater that impact – the more important it is that we give this priority. In this case, impact on user is ranked Low, Medium, High where “High Impact” is how bad it would be if the error was left unresolved. High Impact. Users will be unable to perform important system tasks or unable to understand important content if this item is left not repaired. Medium Impact. Users will be able to perform important system task with some level of difficulty or will be able to understand important content with difficulty if this item is left not repaired. Low Impact. Users will be inconvenienced by leaving this item not repaired, but will be able to accomplish all tasks. Something that offers no impact should be taken from consideration. Note the asterisk(*) on impact. As we’ll discuss later, there may be better alternatives Some prioritization schemes offered by others also take this into consideration, but they consider user impact for all users, without distinction. I would argue that we should segment the different types of disabilities. This is because issues which impact multiple disability types will need to be given higher priority. Also, some items will impact different user populations in a different way. For instance, focus control will impact nearly all users with disabilities, but will be an absolute barrier for some users but mostly a frustration for others. Photo credit

24 Advanced Prioritization
Ease and Speed of Repair As before, if the goal of our remediation effort is to fix the system, it only makes sense to utilize our developers’ time wisely. Part of doing so is knocking out the easier things before the hard things. Easy things tend to take less time, which means we can get further down the road toward an accessible system by prioritizing the easy things over the hard. Doing so will give us the ability to get quick improvements with minimal effort. Ease & Speed of Repair relates to what it will take to fix the issue. Ease & Speed are related but may be exclusive of each other. It may be very easy to do but could take a long time to get it done. Consider both of these when calculating them. One thing to keep in mind: In this case, the Faster & Easier something is, the higher score (and therefore priority). “Super easy” = 3, “Super Difficult” = 1, etc. Photo Credit:

25 Advanced Prioritization
Impact on Interface & Operation This relates to both the appearance and the functionality of the system. There may be times when an accessibility-related repair may be necessary which would have a negative impact on the user interface and/ or operation of the interface as it is currently designed. One area where I see this happen very often is in color contrast of primary site templates. In these cases poor color contrast is used which is related to the organization’s branding as a whole. Improving the color contrast would essentially require a redesign of the site. In this scenario, things which will require significant redesign will need to be given much lower precedence over things which will have little-to-no impact on the interface. Something that has zero impact on the interface = 3 Something that has high impact on the interface = 1 Photo Credit:

26 Advanced Prioritization
Volume of Repeat Issues How many times does the exact same issue occur? How many times do (very) similar issues occur? The more often the exact violation occurs, the more impactful it is. What I often find when doing accessibility testing is that the same mistakes will be found over and over. Some of those mistakes will be repeated because they’re part of a template. Some of the mistakes will be repeated because common code is used throughout the site. For instance, forms may be created using a server-side library or class which is used to make all forms. In either case what happens is that the volume of similar issues will be quite high while the actual volume of offending code will be small. In this case, a large benefit can be realized quickly by fixing the templates or classes which cause the most common issues first. It is important to understand that in this case, we must refer to the exact same violation (in other words, the exact code) is found. Similar violations may exist (such as missing alt text), but that doesn’t mean the code which causes the issue is the same. Some prioritization schemes do not make a distinction regarding whether the violation is exactly the same vs. merely the same type of problem. There’s value in tracking similar violations as “patterns”, but volume of identical issues tends to be a multiplier in terms of priority. Photo Credit:

27 Advanced Prioritization
Location of Issues Traffic Criticality of location Accessibility issues that don’t impact anyone aren’t issues at all. One area where this is especially true is when it comes to the location of the issues. In any site, there will be those pages which get far more traffic than others. People commonly understand this concept as the 80/20 rule. In reality, it might be that 15% of your pages get 90% of your traffic. Whatever the case, you will want to remediate issues which occur on the pages with the highest traffic before those which don’t get much traffic at all. Also a factor is the criticality of the location of violation. If the violation exists in a primary template or in the content area of a critical workflow, its priority is higher. One example would be a multi-step interaction where screen 1 of the interaction is free of problems, screen 3 of the interaction is free of problems, but the middle screen has a high-impact flaw which causes a roadblock to success for the user. Because this causes the entire interaction to fail, the issues on screen 2 need higher priority

28 Advanced Prioritization
Secondary Benefit Older Users Low Literacy Users Low Bandwidth Users Reduced Dev/ Maintenance time Alternate Devices SEO Usability Tie to org goals Certain accessibility best practices also have a tie-in with other benefits that could be realized if violations are repaired. For instance, depending upon the nature of the issue, you may find that you’ll be able to improve on-page factors for SEO or improve display for alternate devices like tablets. These should be tied to the organization’s goals. For instance, if you’re already using standards based production, then the reduced development and maintenance time isn’t going to mean much. Photo Credit:

29 Advanced Prioritization
(Impact + Repair Speed + Location + Secondary Benefits) * Volume = Priority Sort all issues according to priority Fix em! Very simply, each priority factor for the issue can be added together, multiplied by the volume of occurance for this issue and you have your priority. Then sort the issues by the priority and get to work fixing them!

30 Advanced Prioritization
What’s missing? Normalize data? Do some of this ahead of time? Importance ranking for each factor Risk from Litigious user types? It seems sensible then, that we could just add these things up and be done with it. But, there are a lot of things not yet known. Do we do this ahead of time? There should be a means we can use to pre-calculate at least some portion of this ahead of time. As stated earlier, WCAG Level is of no help here. Furthermore, precalculation cannot take into consideration location of issues and doing so is beyond the abilities of any tool. But, in creating our checklist, it seems to make sense to also include data on the things we can consider up front. Impact is very easy to understand and for the most part very easy to factor. Repair speed, location, and secondary benefits, not so much. Each of these may mean different things for different organizations. Would it be beneficial to be able to provide a ranking factor for each item? For instance, do we want to say that user impact is 10x as important as everything else? Certainly secondary benefit is of lower importance than impact. For organizations in the United States, risk mitigation is a strong motivator. Do we want to take this into consideration when talking about user impact? Or, do we want to take into consideration population size? For instance, there are far more people with low vision than people who are blind. Is this a factor to consider? Or, do we risk implicitly considering the needs of some over the needs of others?

31 Dilution Managing Remediation

32 Managing Remediation You have a report full of bugs
Now what do you do?

33 Managing Remediation Managing Remediation is a process not unlike dilution in chemistry Dilution: The process of reducing the concentration of a solute in solution In our case: reducing concentration (defect density) in a system Concentration = numBugs/linesOfCode In chemistry, dilution is the process of reducing the concentration of a solute in solution, usually simply by mixing with more solvent. In our case, the solute are bugs. The solvent is the code itself. We want to add more “solvent”, which is clean code.

34 Managing Remediation Accessibility Errors Non-Compliant System
- Low Priority - Medium Priority - High Priority Non-Compliant System Mostly-Compliant System Partially-Compliant System Continuing our dilution analogy, we see three beakers indicating different states of concentration. Given the three examples, we see our start point, mid-point, and end point represented by beakers of solution. In our starting point we see a beaker that has a high concentration of problems – especially high priority problems In our mid-point, we see that the vast majority of high priority problems have been eliminated, leaving us to address our medium priority problems Finally, our end point, we see almost all of our problems are low priority. We can choose to fix them if we’d like – and I believe we should.

35 Managing Remediation Fully-Compliant System
And here is our unicorn – a perfectly accessible system

36 Managing Remediation Measuring improvement
(concentrationStart - concentrationEnd) / time Only 1st order accurate. Perfect for snapshots. We can measure the improvement of our system by tracking it over time. Very simply, we can measure our starting concentration minus our ending concentration (in other words, our improvement) divided by time. There is an inherent flaw with such a simple approach in that it is most accurate when the time interval is a short one. A two point representation for a first derivative such as above is deemed 1st order accurate since this approximation has an error that scales with the time interval to the 1st power. Another key point to keep in mind is that the # of lines of code WILL change when bugs are fixed, so that you cannot just continually count bugs each time, you must also count lines each time as well.

37 Conclusion

38 Conclusion In the quest for #perfectA11y, prioritization helps us get closer quicker We must maximize efficiency to have high positive impact Multiple factors exist that can be used to determine priority Iterate remediation efforts to progressively dilute them We can measure success

39 Connecting with Deque Twitter LinkedIn Web Email
@dequesystems Deque Systems

Download ppt "Prioritizing Remediation of Accessibility Issues"

Similar presentations

Ads by Google