6What is an Accessibility Issue? Bug: Term used to describe an error, flaw, mistake, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways.We need to move beyond the impression (that we’ve created, IMO) that accessibility issues are something separate.Accessibility issues aren’t “nice to haves”, or enhancements. They are bugs.We have created a flaw in the system which harms its ability to be used by real people.Photo Credit
8Why Prioritize? Apply resources most effectively Minimize accessibility’s impact on businessMotivate development staffMaximize positive impact for usersReduction of RiskDuring my career, I’ve performed scores of audits and expert reviews to look at the usability and accessibility of web-based systems (websites, web-based applications, intranets, etc.). Of those, I can count on one hand the number of systems tested that were in a pre-production phase prior to the accessibility work. In nearly all cases, the testing was performed on systems that were already released to users. In such a scenario, an organization’s level of risk is at its highest, as people are using the system. Therefore the organization and its developers require sufficiently informative data output from the accessibility testing which is detailed, clear, and actionable. It doesn’t do the client any good to throw them a report filled with a bunch of accessibility violations unless that report also includes information to help them fix their problems. They need to know what their problems are, where their problems are, and how to fix them. There’s also an additional item they need to know: When to fix them.Prioritization of remediation helps to apply resources most effectively. Post-deployment bug fixes are a needless expense. We should apply resources in a manner that reaches deepest toward the greater goodPrioritizing remediation helps minimize accessibility’s impact on business. Generally, accessibility is too often regarded as expensive and hard. Properly prioritized, this expense and difficulty can be significantly diminished. This diminished impact can motivate staff and help get their buy-in for both this effort and future effortsThe end goal, for us as accessibility advocates is the real reason, IMO, we should prioritize: It allows us to maximize the positive impact for users. Fixing the most important stuff first will mean that we can more quickly make things better for the end userFinally, prioritization also helps reduce our risk.
10Understanding RiskRisk is the potential that a chosen action or activity (including the choice of inaction) will lead to a loss (an undesirable outcome).Photo Credit
11Understanding RiskUltimately, remediation of bugs is an effort at risk mitigationRisks ofPoor quality (Users having problem with system)Lost incomeAncillary lossesAdministrative Complaint (public sector)LitigationAlthough people generally think of legal risk as the only risk when it comes to accessibility, there are many types of risks that are created by bugs in a system.Poor quality: Operability or performance problems for the end userLost income: The volume or severity of bugs is such that users who want to buy our products or consume our content do not or cannot do so and we lose moneyAncillary losses: Losses caused by spending money you don’t need to: Primarily, a11y fixes after the fact. Oops, we decided a11y is important after the fact, now we must spend money we didn’t intend to spend and that negatively impacts the money we have available to do other thingsAdministrative Complaint: Compliance is a requirement in some industries and noncompliance raises the chance that we can have the heavy hand of regulators beating down on usLitigation: Certain industries see an increased amount of litigation surrounding accessibility, which is a significant risk.Many of these risks are related and have a multiplying effect upon our business case.Intelligently remediating our bugs will work to rapidly reduce risk
12Understanding Risk Probability Probability = (number of negative events) / (population)What is your probability of an actualized risk? In its simplest form: Probability = (number of negative events) / (population). Your probability of anything happening is based on the size of your population and the number of negative events within that population. In other words, if 10 people out of every 100 get a speeding ticket every 5 years, then your probability of a speeding ticket every 5 years is 10% (10/100 = .10).In terms of accessibility lawsuits, it is important to understand how to determine your population. Your population isn’t “all the other websites on the Web”, but rather it is your peer companies. How you define your peer companies involves a number of factors such as industry, site features, audience demographics, reach, and engagement in risky behavior. Note that I use the word “peer” rather than “competition”. Your competition could be a brick & mortar entity, or it could be a company that is much smaller. While they may compete in the marketplace, they are not a “peer”. Again, to use a simple analogy: Lung Cancer Probability = number of cancer cases/ number of people engaging in cancer-causing behaviors. Although everyone has some level of probability for lung cancer, people who are engaging in cancer-causing behaviors are in a very different peer group from those who don’t. Their engagement in that behavior increases their probability of the negative event.Photo credit
13Understanding Risk Risk Amount Risk Amount = (probability of a negative event) * (expected loss in case of negative event)From the probability, we can determine how much we are risking. Our risk amount is our probability multiplied by our expected loss: Risk Amount = (probability of a negative event) * (expected loss in case of negative event). For example, if we have a 10% probability, our risk amount is 10% of the total expected loss (.10*totalExpectedLoss). To use an extreme case, we’ll use the Target lawsuit. Target settled with the NFB for $6,000,000 plus $4,000,000 in attorneys fees, etc. Based on their population (very large e-commerce sites) they had an 8% probability: .08 * $10,000,000 = $800,000. If you’re a company that is a peer of Target, you are risking $800,000. Some people may argue that you’re actually risking $10,000,000 because that’s what Target settled for, but that doesn’t take into consideration your probability of the lawsuit.
14ROI = ((Risk Amount - Investment)/ Investment)*100 Understanding RiskROIROI = ((Risk Amount - Investment)/ Investment)*100Where:Risk Amount = Expected loss * probabilityInvestment = Money spent on AccessibilityYour Return on Investment (ROI) for accessibility work (assuming 8% probability) is $800,000 minus the money you spend on fixing or remediating accessibility related bugs.For those who are intending to try to sell accessibility internally, it may make sense to take this into consideration when trying to gather budget for the efforts.
15What factors impact our ability to fix our system? Challenges
16Challenges Not all accessibility problems are equal TimeImpactImpact on UsersImpact on BusinessWCAG Level & SC is inappropriate for determining priorityThe Web Content Accessibility Guidelines structures its Success Criteria in terms of “Level”. Success Criterion can be Level A, Level AA, or Level AAA. Unfortunately, their explanation of the levels isn’t terribly clear. It was much more clear in WCAG 1.0, however the WCAG 1.0 Priority levels did not take into consideration any factors other than user impact. It is clear that in the early days of work toward WCAG 2.0, they understood that conformance level needs to take more things into consideration, as evidenced by this early draft. Unfortunately, the current WCAG documentation on Understanding Conformance currently lacks a description of any kind whatsoever on the topic of what the conformance levels mean and what differentiates each from the rest.Consequently, people tend to believe WCAG 2.0’s Level is synonymous with WCAG 1.0’s PriorityWhat is clear, despite the lack of guidance in WCAG, is that they acknowledge through the use of different levels that not all accessibility issues are created equal. Some items are more impacting on users than others. Some items are technically difficult or impossible to achieve. Some items don’t have good support among user agents or assistive technologies. As a consequence of all these factors and more, it would be rather reckless to treat everything as if they were all equal. In some cases, doing so could risk budgets while not helping the end user. Instead, remediation should be targeted to provide the best benefit quickly and cheaply.Turning back to impact, for a moment, I decided to take my own list of Best Practices and sort them by impact, giving an impact of '1' for each user population affected (Low Vision, Blind, Hearing Impaired, Mobility Impaired, and Cognitive). In other words violating a Best Practice that impacts all of them got a '5' for impact. Violating a Best Practice that impacts only Blind people got a '1'. Here's what I found:5 - 2A ; 1AAA (66% A; 33%AAA) A; 10 AA; 15 AAA (55% A; 18% AA; 27% AAA) A; 5 AA; 12 AAA (67% A; 9% AA; 23% AAA) A; 3 AA; 12 AAA (67% A; 7% AA; 26% AAA A; 11 AA; 13 AAA (68% A; 15% AA; 17% AAA)What I found was that, at least in my subjective interpretation, reliance on WCAG level alone meant that we could be including best practices that have rather low user impact. Also, we could be giving high priority to things that are difficult or time-consuming to remediateThe problem is, obviously, not all best practices are created equally. "Provide a help link on each page" (mapped to 3.3.5) is definitely not the same as "Provide ability to pause, stop, or hide content which updates automatically" (2.2.2) and "Provide the ability for users to turn on sound only at their request" (1.4.2). The latter case illustrates the need to rank the impact of individual populations. The impact of auto-starting audio for Blind people is very high and the impact on Mobility Impaired and Low-Vision persons is much higher than hearing impaired or cognitive. I'd say 3,2,2,1,1 on that one.
17Challenges Time Often at a premium Time spent on after-the-fact bug repairs is time that is taken away from meeting other business needsSee, “Technical Debt”, Martin FowlerDoing bug remediation takes time. Ideally this would be time we don’t need to spend because we’d get everything right in the first place. That is often not the case.Creating bugs causes us to incur Technical Debt - a metaphor referring to the eventual consequences of poor design. This debt must eventually be repaid in some form – the remediation of bugs, for instance. The cost of that debt in this instance is the time that the repair of the bug requires.“We’ll do it later” obligates you to pay the extra time cost of actually doing it later when you should have had a commitment to quality in the first place. (Hard to stop a moving train, repairing of bugs is chasing the train)Very rarely are development staff just sitting around, looking for something to do. So the repair of bugs takes time away from doing other things. Often times, those other things are things which have a direct line on ROI: Important new features that need to ship now.Repair of bugs is a time expense and time is money, so we need to ensure we prioritize on the important things so that effort is not used frivolously.Photo Credit:
18Challenges Impact Budgets Resources System So, given what’s been said already: Bug repair will also cause an impact on the budgets, resources and even to the systemDo these bugs cause us to require outside help from a consultant? Do we need to outsource this repair to a third party? Or, conversely, do we need to take that work away from third party and use a specific internal resource who is an expert at accessibility?Also, how will fixing this bug impact the actual system? Will it impact the presentation logic? Will it impact the business logic? Are there other features in the system or in upcoming builds that are going to be impacted by fixing this item?These are all important things that can cause challenges in determining priority
20Simple Prioritization Focused solely on time and (simple) impactHow long will it take?How bad is the problem?In the simple prioritization scheme, you focus on only two factors: Time and ImpactFor time: How long will it take to fix this issue?For impact: How bad is the problem for users?
21Simple Prioritization ProsFocused on the userSuper simpleOften, “hunch” from expert is as good as something more formalConsDoes not take into consideration impact on business or system
22Advanced Prioritization User ImpactEase & SpeedImpact on InterfaceVolumeLocationSecondary BenefitEach item ranked: None (0), Low (1), Medium (2), High (3)With advanced prioritization, there are six items to take into consideration:As before:User ImpactEase & Speed of RepairPLUS:Impact on the system: Will this change the interface? Will it cause modifications to business logicVolume: How often does this issue occur?Location: Where is this problem?Secondary Benefit: Can we get any additional business boost from fixing this?In all 6 cases we’ll be ranking these factors with 4 scores:0 – None, in other words, either it is not a consideration or there is no weight given by this consideration1 – Low, in other words the weight for this factor is low2 – Medium, in other words there’s a moderate weight for this factor3 – High, in other words this has a lot of weight in this case
23Advanced Prioritization Impact on Users with DisabilitiesBroken down by type of user & impact on eachIB - BlindIV – Visually Impaired (non-blind)IH – Deaf & HoHIM – MotorIC - CognitiveIS – SpeechImpact* = (IB + IV + IH + IM + IC + IS)The more people are impacted – and the greater that impact – the more important it is that we give this priority.In this case, impact on user is ranked Low, Medium, High where “High Impact” is how bad it would be if the error was left unresolved.High Impact. Users will be unable to perform important system tasks or unable to understand important content if this item is left not repaired.Medium Impact. Users will be able to perform important system task with some level of difficulty or will be able to understand important content with difficulty if this item is left not repaired.Low Impact. Users will be inconvenienced by leaving this item not repaired, but will be able to accomplish all tasks.Something that offers no impact should be taken from consideration.Note the asterisk(*) on impact. As we’ll discuss later, there may be better alternativesSome prioritization schemes offered by others also take this into consideration, but they consider user impact for all users, without distinction. I would argue that we should segment the different types of disabilities. This is because issues which impact multiple disability types will need to be given higher priority. Also, some items will impact different user populations in a different way. For instance, focus control will impact nearly all users with disabilities, but will be an absolute barrier for some users but mostly a frustration for others.Photo credit
24Advanced Prioritization Ease and Speed of RepairAs before, if the goal of our remediation effort is to fix the system, it only makes sense to utilize our developers’ time wisely. Part of doing so is knocking out the easier things before the hard things. Easy things tend to take less time, which means we can get further down the road toward an accessible system by prioritizing the easy things over the hard. Doing so will give us the ability to get quick improvements with minimal effort.Ease & Speed of Repair relates to what it will take to fix the issue. Ease & Speed are related but may be exclusive of each other. It may be very easy to do but could take a long time to get it done. Consider both of these when calculating them.One thing to keep in mind: In this case, the Faster & Easier something is, the higher score (and therefore priority). “Super easy” = 3, “Super Difficult” = 1, etc.Photo Credit:
25Advanced Prioritization Impact on Interface & OperationThis relates to both the appearance and the functionality of the system.There may be times when an accessibility-related repair may be necessary which would have a negative impact on the user interface and/ or operation of the interface as it is currently designed. One area where I see this happen very often is in color contrast of primary site templates. In these cases poor color contrast is used which is related to the organization’s branding as a whole. Improving the color contrast would essentially require a redesign of the site. In this scenario, things which will require significant redesign will need to be given much lower precedence over things which will have little-to-no impact on the interface.Something that has zero impact on the interface = 3Something that has high impact on the interface = 1Photo Credit:
26Advanced Prioritization Volume of Repeat IssuesHow many times does the exact same issue occur?How many times do (very) similar issues occur?The more often the exact violation occurs, the more impactful it is.What I often find when doing accessibility testing is that the same mistakes will be found over and over. Some of those mistakes will be repeated because they’re part of a template. Some of the mistakes will be repeated because common code is used throughout the site. For instance, forms may be created using a server-side library or class which is used to make all forms. In either case what happens is that the volume of similar issues will be quite high while the actual volume of offending code will be small. In this case, a large benefit can be realized quickly by fixing the templates or classes which cause the most common issues first.It is important to understand that in this case, we must refer to the exact same violation (in other words, the exact code) is found.Similar violations may exist (such as missing alt text), but that doesn’t mean the code which causes the issue is the same.Some prioritization schemes do not make a distinction regarding whether the violation is exactly the same vs. merely the same type of problem. There’s value in tracking similar violations as “patterns”, but volume of identical issues tends to be a multiplier in terms of priority.Photo Credit:
27Advanced Prioritization Location of IssuesTrafficCriticality of locationAccessibility issues that don’t impact anyone aren’t issues at all. One area where this is especially true is when it comes to the location of the issues. In any site, there will be those pages which get far more traffic than others. People commonly understand this concept as the 80/20 rule. In reality, it might be that 15% of your pages get 90% of your traffic. Whatever the case, you will want to remediate issues which occur on the pages with the highest traffic before those which don’t get much traffic at all.Also a factor is the criticality of the location of violation. If the violation exists in a primary template or in the content area of a critical workflow, its priority is higher.One example would be a multi-step interaction where screen 1 of the interaction is free of problems, screen 3 of the interaction is free of problems, but the middle screen has a high-impact flaw which causes a roadblock to success for the user. Because this causes the entire interaction to fail, the issues on screen 2 need higher priority
28Advanced Prioritization Secondary BenefitOlder UsersLow Literacy UsersLow Bandwidth UsersReduced Dev/ Maintenance timeAlternate DevicesSEOUsabilityTie to org goalsCertain accessibility best practices also have a tie-in with other benefits that could be realized if violations are repaired. For instance, depending upon the nature of the issue, you may find that you’ll be able to improve on-page factors for SEO or improve display for alternate devices like tablets.These should be tied to the organization’s goals. For instance, if you’re already using standards based production, then the reduced development and maintenance time isn’t going to mean much.Photo Credit:
29Advanced Prioritization (Impact + Repair Speed + Location + Secondary Benefits) * Volume = PrioritySort all issues according to priorityFix em!Very simply, each priority factor for the issue can be added together, multiplied by the volume of occurance for this issue and you have your priority.Then sort the issues by the priority and get to work fixing them!
30Advanced Prioritization What’s missing?Normalize data?Do some of this ahead of time?Importance ranking for each factorRisk from Litigious user types?It seems sensible then, that we could just add these things up and be done with it. But, there are a lot of things not yet known.Do we do this ahead of time? There should be a means we can use to pre-calculate at least some portion of this ahead of time. As stated earlier, WCAG Level is of no help here. Furthermore, precalculation cannot take into consideration location of issues and doing so is beyond the abilities of any tool. But, in creating our checklist, it seems to make sense to also include data on the things we can consider up front.Impact is very easy to understand and for the most part very easy to factor. Repair speed, location, and secondary benefits, not so much. Each of these may mean different things for different organizations. Would it be beneficial to be able to provide a ranking factor for each item? For instance, do we want to say that user impact is 10x as important as everything else? Certainly secondary benefit is of lower importance than impact.For organizations in the United States, risk mitigation is a strong motivator. Do we want to take this into consideration when talking about user impact? Or, do we want to take into consideration population size? For instance, there are far more people with low vision than people who are blind. Is this a factor to consider? Or, do we risk implicitly considering the needs of some over the needs of others?
32Managing Remediation You have a report full of bugs Now what do you do?
33Managing RemediationManaging Remediation is a process not unlike dilution in chemistryDilution: The process of reducing the concentration of a solute in solutionIn our case: reducing concentration (defect density) in a systemConcentration = numBugs/linesOfCodeIn chemistry, dilution is the process of reducing the concentration of a solute in solution, usually simply by mixing with more solvent.In our case, the solute are bugs. The solvent is the code itself.We want to add more “solvent”, which is clean code.
34Managing Remediation Accessibility Errors Non-Compliant System - Low Priority- Medium Priority- High PriorityNon-Compliant SystemMostly-Compliant SystemPartially-Compliant SystemContinuing our dilution analogy, we see three beakers indicating different states of concentration.Given the three examples, we see our start point, mid-point, and end point represented by beakers of solution.In our starting point we see a beaker that has a high concentration of problems – especially high priority problemsIn our mid-point, we see that the vast majority of high priority problems have been eliminated, leaving us to address our medium priority problemsFinally, our end point, we see almost all of our problems are low priority. We can choose to fix them if we’d like – and I believe we should.
35Managing Remediation Fully-Compliant System And here is our unicorn – a perfectly accessible system
36Managing Remediation Measuring improvement (concentrationStart - concentrationEnd) / timeOnly 1st order accurate. Perfect for snapshots.We can measure the improvement of our system by tracking it over time.Very simply, we can measure our starting concentration minus our ending concentration (in other words, our improvement) divided by time.There is an inherent flaw with such a simple approach in that it is most accurate when the time interval is a short one.A two point representation for a first derivative such as above is deemed 1st order accurate since this approximation has an error that scales with the time interval to the 1st power.Another key point to keep in mind is that the # of lines of code WILL change when bugs are fixed, so that you cannot just continually count bugs each time, you must also count lines each time as well.
38ConclusionIn the quest for #perfectA11y, prioritization helps us get closer quickerWe must maximize efficiency to have high positive impactMultiple factors exist that can be used to determine priorityIterate remediation efforts to progressively dilute themWe can measure success
39Connecting with Deque Twitter LinkedIn Web Email @dequesystems Deque Systems deque.com