Presentation is loading. Please wait.

Presentation is loading. Please wait.

Heuristic Test Strategy Model

Similar presentations


Presentation on theme: "Heuristic Test Strategy Model"— Presentation transcript:

1 Heuristic Test Strategy Model
A Context Driven Model for designing a test strategy

2 About me – John Kubiak Context Driven Tester, Guild Lead, and Coach at Serko Developer (C#, Java, ASP.net, PHP, Android) Automated Tester (Selenium, Redwood, JMeter, Perl) Member of AST Enterprise Service Planner with Kanban ICAgile Certified Professional Volunteer with ImpactNPO

3 Contact @johnnykubiak

4 About you Who has heard of the Heuristic Test Strategy Model?
Who has heard of Context Driven Testing? Who uses the HTSM? Who considers themselves to be Context Driven?

5 Overview Context Driven Testing The Heuristic Test Strategy Model
What is CDT? How do you know if you are a CDT? The Heuristic Test Strategy Model Purpose Deconstruction Creating a Context Practical applications of the HTSM and creating a test strategy Test Techniques Test Design Risk Analysis Discussions and more… Outro

6 Context Driven Testing
What is CDT? My definition: Discovery of information that could affect what I do as a tester Testing that doesn’t follow best practices but identifies good practices in different contexts based on the information discovered Testing that requires testers to keep their brains engaged at all times My definition: - Awareness and discovery of information that could have an impact on what I do as a tester. This could be: - deciding which techniques I may or may not use - influencing my test design - identifying oracles - deciding on documentation and artefacts, communication and media - ability to justify my decisions using the information that I discovered thanks to the HTSM - Testing that doesn’t follow best practices or standards but decides on which practices will be followed based on the context - Testing that requires people to keep their brains engaged at all times

7 Context Driven Testing
Who considers themselves a Context Driven Tester? What makes someone Context Driven? Intent – do you actively seek out context? Do you take the context and use it to drive test strategy and test related activities?

8 Heuristic Test Strategy Model
Heuristic: a fallible experienced based method for making a decision or solving a problem Test Strategy: the set of ideas that guide your test design Model: a composition of concepts Heuristic Not a guarantee Requires skill Test Strategy How you’ll cover the software and detect problems. Can also relate to how you will document your work, work with others, assign tasks. However these are also sometimes referred to as test logistics, and test products. Model A composition of concepts designed to help people know, understand, and simulate the subject which the model represents. Put it all together: A collection of concepts that may help you do better testing but will require skill and practice to develop that skill in order to apply it effectively

9 The Model http://www.satisfice.com/tools/htsm.pdf
An explanation of the HTSM As per the model, most testers do test within a context and will intuitively use some of these factors to drive their testing. However by only focusing on a few of these contextual factors, we’re likely to miss important bugs, run unnecessary tests, or misinterpret information. The model is primarily designed to facilitate self-learning for testers. It tries to cover most of the important factors that can have an impact on the way we test. In addition to discovering the information, once discovered I use this information to drive everything from test design, continue through test execution to identify new ideas, identify oracles, increase the probability that my tests add value, and eliminate irrelevant tests Concept is: -Three elements influence our test strategy, described here as test techniques -The three elements contain multiple heuristics for describing a variety of information Test Techniques - The info we discover helps us make better, more informed decisions as to what we test, how we test, when we, why we test, and where we test As we execute the strategy, we gain more information, this information may change our context, we adjust the strategy accordingly We start with rumours, we create our best attempt at strategy, we then gain info and refine our strategy and it improve it as the project progress. Important to have a flexible strategy, not be locked in by processes, or excessive test strategy documents that either make you not want to change strategy or become irrelevant as the project evolves The product of testing is perceived quality We gather information about how we have perceived the quality, another person may perceive it differently. We are not talking about the actual quality but our experience of quality

10 Deconstruction Product Elements are things that you intend to test. Software is complex and invisible. Take care to cover all of it that matters, not just the parts that are easy to see Project Environment includes resources, constraints, and other elements in the project that may enable or hobble our testing. Sometimes a tester must challenge constraints, and sometimes accept them Quality Criteria are the rules, values, and sources that allow you as a tester to determine if the product has problems. Quality criteria are multidimensional and often hidden or self-contradictory Test Techniques are heuristics for creating tests. All techniques involve some sort of analysis of project environment, product elements, and quality criteria Product Elements Most familiar to testers. It’s the things we interact with and the things we evaluate. We are trying to assess if the product elements meet the quality standards that are expected. Project Environment What is available to us that will help us test and what may present problems and hinder our testing. Will our environment allow us to do good testing or will it force us to apply bad work practice. Challenge those times when you are forced to do bad work. Never accept it. Only do work that you are proud of. Quality Criteria Why are we testing? How will we recognize problems that may not be as obvious as a function that doesn’t behave as expected? Test Techniques Using the information we gather about the product, the project environment, and quality criteria, we may notice an improvement in: - choosing the most effective techniques - minimizing the chances of overlooking some important tests - a more complete test strategy that enables us to test faster and find more important information for the people that we test on behalf of such as clients, developers, product owners.

11 Practical Application of the Model
Discover Information – Active Listening and Active Reading Reading documentation Examining comparable products Discussions Modelling the software Improve coverage Discover Information Most obvious Reading docs - seek out information related to the model – active reading - documentations could include requirements, s, old issues, artefacts from similar projects - Combine heuristics to identify new possibilities, e.g. input/output + data + transformations + performance + customer expectations Examining comparable products - Look for key elements on the model - Is our product as good as or better? Does it need to be? Discussions - With developers, BA’s, Product Owners, customers – actively listen and pick out information related to the model - Use the model to design more questions and facilitate conversation - Prepare talking points around the ideas before attending meetings - Research Modelling Software Create and improve models by getting more info Improve coverage Resolve ambiguity and identify gaps – e.g. have I checked for all the error messages and scenarios that can trigger these errors, are the error message informative to the user?

12 Use Information to Drive Testing
What types of test related activities are influenced by the HTSM? Specifics: Test Techniques Artefacts, Communication, and Media Risk Analysis Coverage Metrics Models Visualisation Bug prevention Challenging constraints Justifications Test Techniques: Gather info using HTSM, then identify techniques, then generate test ideas within each technique Artefacts Producing mind maps to share a model of context with others to comment on and create a consensus and clarity about what is factored into testing among everyone involved in the project. Can be expanded see Aaron Hodder’s link at end of slides. How will we communicate, media – , face2face, hip chat, skype, formal documentations What will we communicate – test plans (what type, level of detail), test cases, scripts, checklists, matrices etc. By gaining a better understanding of context we are able to make more informed decisions with respect to tools that we might use to support testing, artefacts that we will produce, and choose practices that appropriate based on the context. Use it to question the value of best practices and instead create your own practices that have worked in your experience or the experience of others. Risk based testing the quality criteria is good for this can be used with a technique known as outside-in where you identify the functions that will be tested, see which of the quality criteria apply to each function, then think of ways the program could fail within each category Coverage metrics in addition to covering risks, we need other ways as measuring coverage in case or risk analysis is wrong or incomplete Examples: have I covered all the functions (functional coverage), have I covered all the boundaries (boundary coverage), etc. Whatever needs to be tested, create a list of those items and then check have I covered each one, and I have covered them thoroughly enough, e.g. smoke case, normal use, extreme use, and disfavoured use. Still doesn’t guarantee we will cover everything but it increases our chances Models Creating models, e.g. functional diagrams, state transitions, data flow, anything that helps me understand more about what I’m going to test so I can design tests that are relevant. Visualisation Visualizing the mechanism behind functionality during testing, e.g. when I click this button what is really happening, what can I do that might make if fail? Bug Prevention Develop pre-dev testing notes for developers, including: acceptance criteria risks any other information that will help developers write better quality code and do more testing before handing it to a tester Goal is to reduce rework due to badly written code. Rework requires additional change, each change carries risk and create more bugs. Better to avoid rework. Challenge Constraints Identify constraints such as aspects of the project that you disagree with and believe to be inefficient, ineffective, and wasteful. Challenge the constraints. Justify Finally, justify your decisions if required by showing the information that influenced your decisions. Build credibility by showing you use skill to make informed decisions, rather than parking your brain and allowing your testing to be controlled by processes or best practices

13 Practice and Experimentation
Learn the data – keywords and questions (link to quiz at end of slides) Transform data into information – what do the keywords represent Develop skill and knowledge - apply the model in practice Practice, practice, practice Learn the data, transform into information. Then apply in practice to develop skill and knowledge. Pick any functions or piece of software and try using the factors in the HTSM See if you discover more information See if this information adds value to your testing e.g. find more bugs in less time or info that you can give to developers to help them produce better code If you can’t throw away practice then try experiment on the job but make sure you create an environment where it ok to fail in the pursuit of mastery

14 Outro HTSM is a powerful model for getting information and driving testing Flexible, adapts to different types of software, domains, SDLCs You own the model – change it, add new ideas It is heuristics, not a guarantee, can identify conflicting information Requires skill that is developed through practice and experimentation Preeti and Ram can share their experiences with you HTSM is a powerful model for getting information and driving testing Difficult to use at first, because of the possibilities and amount of information it can generate As you use it regularly it will become easier and faster, and feel more natural. Flexible Flexible and can be adapted to testing of all software, in all domains, in all SDLCs. Own the model You own the model, there is no one way for applying. Just be aware that it is a heuristic, it doesn’t guarantee it will solve your problems, nor does it guarantee it will provide the best solution. But it does provide you a framework for getting more information and with time and skill the decisions you make, the tools you choose, and way in which you apply those tools will improve. Heuristic Since you own your heuristics, change the model, and add to it when needed. E.g. hardware location. Practice Practice using the HTSM, making decisions based on the information you discover, ask yourself how can I tell if I’ve made the right or wrong decision. What are my alternatives and have I picked the best one? Share with other testers and developers, organise dojos for group practice and learn by observing and critiquing each other. Keep experimenting. Identify a problem, create a hypothesis, (in)validate the hypothesis. Take the information from the experiment to drive the next decision Talks to other testers Share stories about your journey to become context driven How is your testing improving How did you learn the model How do you apply it What differences are you noticing with your approach to testing, the value the generate, and its impact on others such as developers

15 Links Heuristic Test Strategy Model Quiz Context Driven Testing Test Techniques Lecture 1, Parts D, E, F -

16 Links Risk Based Testing Context Maps and Information Radiators mind-mapping-software-as-a-visual-test-management-tool/

17 Thanks! James Bach and Cem Kaner for most of the material in this presentation Vend for the venue Shirley and WeTest team for organising All of you for listening


Download ppt "Heuristic Test Strategy Model"

Similar presentations


Ads by Google