1. Heuristic Test Strategy Model
A Context Driven Model for designing a test strategy

2. About me – John Kubiak
Context Driven Tester, Guild Lead, and Coach at Serko
Developer (C#, Java, ASP.net, PHP, Android)
Automated Tester (Selenium, Redwood, JMeter, Perl)
Member of AST
Enterprise Service Planner with Kanban
ICAgile Certified Professional
Volunteer with ImpactNPO

3. Contact
@johnnykubiak

4. About you Who has heard of the Heuristic Test Strategy Model?
Who has heard of Context Driven Testing?
Who uses the HTSM?
Who considers themselves to be Context Driven?

5. Overview Context Driven Testing The Heuristic Test Strategy Model
What is CDT?
How do you know if you are a CDT?
The Heuristic Test Strategy Model
Purpose
Deconstruction
Creating a Context
Practical applications of the HTSM and creating a test strategy
Test Techniques
Test Design
Risk Analysis
Discussions and more…
Outro

6. Context Driven Testing
What is CDT?
My definition:
Discovery of information that could affect what I do as a tester
Testing that doesn’t follow best practices but identifies good practices in different contexts based on the information discovered
Testing that requires testers to keep their brains engaged at all times
My definition:
- Awareness and discovery of information that could have an impact on what I do as a tester. This could be:
- deciding which techniques I may or may not use
- influencing my test design
- identifying oracles
- deciding on documentation and artefacts, communication and media
- ability to justify my decisions using the information that I discovered thanks to the HTSM
- Testing that doesn’t follow best practices or standards but decides on which practices will be followed based on the context
- Testing that requires people to keep their brains engaged at all times

7. Context Driven Testing
Who considers themselves a Context Driven Tester?
What makes someone Context Driven?
Intent – do you actively seek out context?
Do you take the context and use it to drive test strategy and test related activities?

8. Heuristic Test Strategy Model
Heuristic: a fallible experienced based method for making a decision or solving a problem
Test Strategy: the set of ideas that guide your test design
Model: a composition of concepts
Heuristic
Not a guarantee
Requires skill
Test Strategy
How you’ll cover the software and detect problems.
Can also relate to how you will document your work, work with others, assign tasks.
However these are also sometimes referred to as test logistics, and test products.
Model
A composition of concepts designed to help people know, understand, and simulate the subject which the model represents.
Put it all together: A collection of concepts that may help you do better testing but will require skill and practice to develop that skill in order to apply it effectively

9. The Model http://www.satisfice.com/tools/htsm.pdf
An explanation of the HTSM
As per the model, most testers do test within a context and will intuitively use some of these factors to drive their testing. However by only focusing on a few of these contextual factors, we’re likely to miss important bugs, run unnecessary tests, or misinterpret information.
The model is primarily designed to facilitate self-learning for testers. It tries to cover most of the important factors that can have an impact on the way we test. In addition to discovering the information, once discovered I use this information to drive everything from test design, continue through test execution to identify new ideas, identify oracles, increase the probability that my tests add value, and eliminate irrelevant tests
Concept is:
-Three elements influence our test strategy, described here as test techniques
-The three elements contain multiple heuristics for describing a variety of information
Test Techniques - The info we discover helps us make better, more informed decisions as to what we test, how we test, when we, why we test, and where we test
As we execute the strategy, we gain more information, this information may change our context, we adjust the strategy accordingly
We start with rumours, we create our best attempt at strategy, we then gain info and refine our strategy and it improve it as the project progress.
Important to have a flexible strategy, not be locked in by processes, or excessive test strategy documents that either make you not want to change strategy or become irrelevant as the project evolves
The product of testing is perceived quality
We gather information about how we have perceived the quality, another person may perceive it differently. We are not talking about the actual quality but our experience of quality

10. Deconstruction
Product Elements are things that you intend to test. Software is complex and invisible. Take care to cover all of it that matters, not just the parts that are easy to see
Project Environment includes resources, constraints, and other elements in the project that may enable or hobble our testing. Sometimes a tester must challenge constraints, and sometimes accept them Quality Criteria are the rules, values, and sources that allow you as a tester to determine if the product has problems. Quality criteria are multidimensional and often hidden or self-contradictory Test Techniques are heuristics for creating tests. All techniques involve some sort of analysis of project environment, product elements, and quality criteria
Product Elements
Most familiar to testers. It’s the things we interact with and the things we evaluate.
We are trying to assess if the product elements meet the quality standards that are expected.
Project Environment
What is available to us that will help us test and what may present problems and hinder our testing.
Will our environment allow us to do good testing or will it force us to apply bad work practice. Challenge those times when you are forced to do bad work. Never accept it. Only do work that you are proud of.
Quality Criteria
Why are we testing? How will we recognize problems that may not be as obvious as a function that doesn’t behave as expected?
Test Techniques
Using the information we gather about the product, the project environment, and quality criteria, we may notice an improvement in:
- choosing the most effective techniques
- minimizing the chances of overlooking some important tests
- a more complete test strategy that enables us to test faster and find more important information for the people that we test on behalf of such as clients, developers, product owners.

11. Practical Application of the Model
Discover Information – Active Listening and Active Reading
Reading documentation
Examining comparable products
Discussions
Modelling the software
Improve coverage
Discover Information
Most obvious
Reading docs
- seek out information related to the model – active reading
- documentations could include requirements, s, old issues, artefacts from similar projects
- Combine heuristics to identify new possibilities, e.g. input/output + data + transformations + performance + customer expectations
Examining comparable products
- Look for key elements on the model
- Is our product as good as or better? Does it need to be?
Discussions
- With developers, BA’s, Product Owners, customers – actively listen and pick out information related to the model
- Use the model to design more questions and facilitate conversation
- Prepare talking points around the ideas before attending meetings
- Research
Modelling Software
Create and improve models by getting more info
Improve coverage
Resolve ambiguity and identify gaps – e.g. have I checked for all the error messages and scenarios that can trigger these errors, are the error message informative to the user?

12. Use Information to Drive Testing
What types of test related activities are influenced by the HTSM?
Specifics:
Test Techniques
Artefacts, Communication, and Media
Risk Analysis
Coverage Metrics
Models
Visualisation
Bug prevention
Challenging constraints
Justifications
Test Techniques:
Gather info using HTSM, then identify techniques, then generate test ideas within each technique
Artefacts
Producing mind maps to share a model of context with others to comment on and create a consensus and clarity about what is factored into testing among everyone involved in the project. Can be expanded see Aaron Hodder’s link at end of slides.
How will we communicate, media – , face2face, hip chat, skype, formal documentations
What will we communicate – test plans (what type, level of detail), test cases, scripts, checklists, matrices etc.
By gaining a better understanding of context we are able to make more informed decisions with respect to tools that we might use to support testing, artefacts that we will produce, and choose practices that appropriate based on the context. Use it to question the value of best practices and instead create your own practices that have worked in your experience or the experience of others.
Risk based testing
the quality criteria is good for this
can be used with a technique known as outside-in where you identify the functions that will be tested, see which of the quality criteria apply to each function, then think of ways the program could fail within each category
Coverage metrics
in addition to covering risks, we need other ways as measuring coverage in case or risk analysis is wrong or incomplete
Examples: have I covered all the functions (functional coverage), have I covered all the boundaries (boundary coverage), etc.
Whatever needs to be tested, create a list of those items and then check have I covered each one, and I have covered them thoroughly enough, e.g. smoke case, normal use, extreme use, and disfavoured use.
Still doesn’t guarantee we will cover everything but it increases our chances
Models
Creating models, e.g. functional diagrams, state transitions, data flow, anything that helps me understand more about what I’m going to test so I can design tests that are relevant.
Visualisation
Visualizing the mechanism behind functionality during testing, e.g. when I click this button what is really happening, what can I do that might make if fail?
Bug Prevention
Develop pre-dev testing notes for developers, including:
acceptance criteria
risks
any other information that will help developers write better quality code and do more testing before handing it to a tester
Goal is to reduce rework due to badly written code. Rework requires additional change, each change carries risk and create more bugs. Better to avoid rework.
Challenge Constraints
Identify constraints such as aspects of the project that you disagree with and believe to be inefficient, ineffective, and wasteful. Challenge the constraints.
Justify
Finally, justify your decisions if required by showing the information that influenced your decisions.
Build credibility by showing you use skill to make informed decisions, rather than parking your brain and allowing your testing to be controlled by processes or best practices

13. Practice and Experimentation
Learn the data – keywords and questions (link to quiz at end of slides)
Transform data into information – what do the keywords represent
Develop skill and knowledge - apply the model in practice
Practice, practice, practice
Learn the data, transform into information. Then apply in practice to develop skill and knowledge.
Pick any functions or piece of software and try using the factors in the HTSM
See if you discover more information
See if this information adds value to your testing
e.g. find more bugs in less time or info that you can give to developers to help them produce better code
If you can’t throw away practice then try experiment on the job but make sure you create an environment where it ok to fail in the pursuit of mastery

14. Outro
HTSM is a powerful model for getting information and driving testing
Flexible, adapts to different types of software, domains, SDLCs
You own the model – change it, add new ideas
It is heuristics, not a guarantee, can identify conflicting information
Requires skill that is developed through practice and experimentation
Preeti and Ram can share their experiences with you
HTSM is a powerful model for getting information and driving testing
Difficult to use at first, because of the possibilities and amount of information it can generate
As you use it regularly it will become easier and faster, and feel more natural.
Flexible
Flexible and can be adapted to testing of all software, in all domains, in all SDLCs.
Own the model
You own the model, there is no one way for applying. Just be aware that it is a heuristic, it doesn’t guarantee it will solve your problems, nor does it guarantee it will provide the best solution. But it does provide you a framework for getting more information and with time and skill the decisions you make, the tools you choose, and way in which you apply those tools will improve.
Heuristic
Since you own your heuristics, change the model, and add to it when needed. E.g. hardware location.
Practice
Practice using the HTSM, making decisions based on the information you discover, ask yourself how can I tell if I’ve made the right or wrong decision. What are my alternatives and have I picked the best one?
Share with other testers and developers, organise dojos for group practice and learn by observing and critiquing each other.
Keep experimenting. Identify a problem, create a hypothesis, (in)validate the hypothesis. Take the information from the experiment to drive the next decision
Talks to other testers
Share stories about your journey to become context driven
How is your testing improving
How did you learn the model
How do you apply it
What differences are you noticing with your approach to testing, the value the generate, and its impact on others such as developers

15. Links
Heuristic Test Strategy Model Quiz Context Driven Testing Test Techniques Lecture 1, Parts D, E, F -

16. Links
Risk Based Testing Context Maps and Information Radiators mind-mapping-software-as-a-visual-test-management-tool/

17. Thanks!
James Bach and Cem Kaner for most of the material in this presentation
Vend for the venue
Shirley and WeTest team for organising
All of you for listening