1. Introduction Outline
These slides are distributed under the Creative Commons License. In brief summary, you may make and distribute copies of these slides so long as you give the original author credit and, if you alter, transform or build upon this work, you distribute the resulting work only under a license identical to this one. For the rest of the details of the license, see
ET Workshop v Introduction
©2002 Amland Consulting

2. Test Management and Techniques ET Planning, Exec. and Documentation
Exploratory Testing
Introduction Outline
Introduction
Introduction to testing.
What is Exploratory Testing?
Where to use it?
When to use it?
Introduction to Risk 1. 2. 3. 4. 5.
Test Management and Techniques
ET Planning, Exec. and Documentation
Fasinerende klokke: den har tall fra 3, 6, 9 osv. Til 30??!?
Likner på den som Normannen solgte på postordre til Sverige; watch sold on mail order to Sweden with hands painted on the front.
ET Styles
ET Management
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

3. 1. Introduction 1.1 Introduction to testing – thinking like a tester
Exploratory Testing 1. 2. 3. 4. 5.
1. Introduction
1.1 Introduction to testing – thinking like a tester
1.2 Introduction to Exploratory Testing
1.3 Introduction to Risk and Risk-Based Testing
Introduction to
testing in GENERAL
Exploratory Testing
Risk and Risk-Based Testing
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

4. Different testing approaches
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Different testing approaches
Skeptical approaches
Analytical approaches
Information-driven approaches
Time-honored but less effective approaches
Experiential and intuitive approaches
And…?
Ross Collard (2002)
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

5. Test Management and Techniques ET Planning and Documentation
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Different testing approaches (1)
Skeptical approaches
“In God We Trust, Everything Else We Test!”
“The barbarians (software engineers) are at the gate”
“Let’s use a scatter gun to test with, and see what bugs we hit”
Ross Collard (2002)
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

6. Analytical approaches
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Different testing approaches (2)
Analytical approaches
Let’s analyze the functional specs. to understand the system’s expected behavior
“Let’s develop a model of the system, and then use this conceptual model as a basis for testing”
Let’s derive the test cases by analyzing the description logic, process flows, equivalence classes, changes of state, or input combinations, etc.”
Is this how we LIKE to test?
Ross Collard (2002)
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

7. Information-driven approaches
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Different testing approaches (3)
Information-driven approaches
“Let’s focus the depth and intensity of the testing in the high risk areas, based on the perceived threats and vulnerabilities of the system”
“Let’s follow this bug list or check list in our testing”
“Let’s go ask the software engineers what to test, because they know how the system works”
“Let’s look under the hood and read the code”
“Let’s follow the clients’ direction, because they have the final sign-off authority”
My FAVOURITE?
Ross Collard (2002)
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

8. Time-honored but less effective approaches
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Different testing approaches (4)
Time-honored but less effective approaches
“Let’s follow the book”
“We always do it this way”
“You shouldn’t change that features because it will screw up our testing.” The tail wags the dog.
“Testing is easy, or at least a lot easier than software design, programming and maintenance.”
“Anyone can do testing”
“Errors just happen. They are caused by bad luck.”
Ross Collard (2002)
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

9. Experiential and intuitive approaches
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Different testing approaches (5)
Experiential and intuitive approaches
“Let’s think blue-sky, speculate and follow our intuition.”
“We have good hunches about where the bugs are lurking.”
“Let’s jump in an explore the system’s behavior hands-on, so we can decide how to test it.”
“Let’s find the important bugs fast, and worry about the test paperwork later.”
This includes EXPLORATORY TESTING!?
Ross Collard (2002)
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

10. Test Management and Techniques ET Planning and Documentation
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Different testing approaches…
Exploratory Testing
Let’s explore, design the tests and test the system concurrently (James Bach)
Let’s learn about the system, test it and reports bugs as we go (Cem Kaner)
Let’s structure and document our creative testing so we know where we have been
Let’s apply everything we have learned about testing as we learn about the system, let’s do ”thinking-while-testing”!
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

11. What’s Special about a Tester’s Brain?
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
What’s Special about a Tester’s Brain?
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

12. Epistemology – the Study of Knowledge
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Epistemology – the Study of Knowledge
Epistemology is the study of how we know what we know. The philosophy of science belongs to Epistemology. All good testers practice Epistemology.
From Rapid Software Testing, copyright © James Bach
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

13. Basic Skills of Epistemology
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Basic Skills of Epistemology
Ability to pose useful questions.
Ability to observe what’s going on.
Ability to describe what you perceive.
Ability to think critically about what you know.
Ability to recognize and manage bias.
Ability to form and test conjectures.
Ability to keep thinking despite already knowing.
Ability to analyze someone else’s thinking.
(from Cambridge International Dictionary of English) conjecture ANTAKELSE / HYPOTESE verb, noun  (to form) a guess or judgment, based on the appearance of a situation rather than on proof  We'll never know exactly how she died. We can only conjecture. [I] He conjectured that the company would soon be in financial difficulties. [+ that clause] There's been a lot of conjecture in the papers recently about the royal marriage. [U] It's pure conjecture. Nobody knows the facts. [U] conjectural adjective 
refute “BEVIS” verb [T]  FORMAL OR LAW  to say or prove that (a person, statement, opinion, etc.) is wrong or false  to refute a person/theory/argument The barrister used new evidence to refute the charges and clear the defendant. refutation noun [C/U]  FORMAL OR LAW 
From Rapid Software Testing, copyright © James Bach
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

14. Tunnel-Vision is Our Great Occupational Hazard
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Tunnel-Vision is Our Great Occupational Hazard
invisible
problems
Problems you can find with your biases…
Major problem with ET?
invisible
problems
From Rapid Software Testing, copyright © James Bach
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

15. Test Management and Techniques ET Planning and Documentation
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Når man skal utforske
Funksjonalitet kan man
Ikke være for bastant!
Antakelse
motbevise
A Tester’s Attitude
Cautious
Jump to conjectures, not conclusions.
Practice admitting “I don’t know.”
Have someone check your work.
Curious
What would happen if…?
How does that work?
Why did that happen?
Critical
Proceed by conjecture and refutation.
Actively seek counter-evidence.
Good testers
are hard to fool.
(from Cambridge International Dictionary of English) conjecture verb, noun  (to form) a guess or judgment, based on the appearance of a situation rather than on proof  We'll never know exactly how she died. We can only conjecture. [I] He conjectured that the company would soon be in financial difficulties. [+ that clause] There's been a lot of conjecture in the papers recently about the royal marriage. [U] It's pure conjecture. Nobody knows the facts. [U] conjectural adjective 
refute verb [T]  FORMAL OR LAW  to say or prove that (a person, statement, opinion, etc.) is wrong or false  to refute a person/theory/argument The barrister used new evidence to refute the charges and clear the defendant. refutation noun [C/U]  FORMAL OR LAW 
From Rapid Software Testing, copyright © James Bach
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

16. To improve our ”judgement and skills”:
Exploratory Testing 1. 2. 3. 4. 5.
To improve our ”judgement and skills”:
Heuristics:
A heuristic is a fallible method for finding the solution to a problem. It's essentially a plausible guess, or a mechanism that helps generate plausible guesses.
Fra:"James Bach"
Dato:Sat, 17 Aug :27: Emne:RE: Context-sensitive skills and Heuristics
Hi Stale, I'm speaking of heuristic the way the term is used in AI, Psychology, and Philosophy. A heuristic is a fallible method for finding the solution to a problem. It's essentially a plausible guess, or a mechanism that helps generate plausible guesses.
"Avoid driving while intoxicated, because there is an elevated danger of an accident" is a heuristic. "Never drive when intoxicated" is, by contrast, a rule. The difference between them is that we relate to heuristics as a tool to apply; something that might help us do the right thing in a given situation, whereas we relate to a rule as something to comply with; something that defines right behavior. Using heuristics properly requires that you exercise discretion and judgment, on some level; whereas judgment may get in the way of rules. It's helpful to have contradictory heuristics, because that's like having a variety of advice available before making a decision; whereas contradictory rules make compliance impossible. -- James
James Bach,
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

17. Heuristics…continued:
Exploratory Testing 1. 2. 3. 4. 5.
Heuristics…continued:
"Avoid driving while intoxicated, because there is an elevated danger of an accident" is a heuristic.
"Never drive when intoxicated" is, by contrast, a rule.
See previous Note-page for reference to James Bach:
”The difference between them is that we relate to heuristics as a tool to apply; something that might help us do the right thing in a given situation, whereas we relate to a rule as something to comply with; something that defines right behavior.”
James Bach,
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

18. Testing is done in Context
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Testing is done in Context
The value of any practice depends on its context.
There are good practices in context, but there are no best practices.
People, working together, are the most important part of any project's context.
Projects unfold over time in ways that are often not predictable.
The product is a solution. If the problem isn't solved, the product doesn't work.
Good software testing is a challenging intellectual process.
Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.
Cem Kaner Black Box Testing:
Testing is done in context
Rather than looking for The One True Way to run a test group, we should recognize that the testing group is a service organization and that its work is done in a context. Life cycles, testing group missions, testing techniques must fit within the overall organization. We should look at / for relevant factors (perform a requirements analysis ) to guide our decisions.
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

19. Exercise: The Triangle Program
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Exercise: The Triangle Program
Specification:
This program takes three numbers as input.
The numbers represent the dimensions of a triangle.
When you click on the check button, the program tells you what kind of triangle the sides represent:
scalene (no side equal to any other)
isosceles (two sides are equal)
equilateral (all sides are equal)
Please test this program.
Scalene = skjev, isosceles = likebent, equilateral = likesidet
Several classes of issues were missed by most students. For example:
Few students checked whether they were producing valid triangles. (1,2,3) and (1,2,4) cannot be the lengths of any triangle.
Knowledge of the subject matter of the program under test will enable you t ocreate test cases that are not directlu suggested by the specification. If you lack that knowledge , you will miss key tests. (This knowledge is sometimes called “domain knowledge”, not to be confused with “domain testing”)
Few students checked non-numeric values, bad delimiters, or non-integers
The only boundaries tested were at MaxInt or 0
Myer’s Answer:
Test case for a valid scalene triangle
Test case for a valid equilateral triangle
Three test cases for valid isosceles triangles (a=b, b=c, a=c)
One, two or three sides has zero values (5 cases)
One side has a negative
Sum of two numbers equals the third (e.g. 1,2,3) is invalid b/c not a triangle (tried with 3 permutations a+b=c, a+c=b, b+c=a)
Sum of two numbers is less than the third (e.g. 1,2,3) (3 permutations)
Non-integer
Wrong number of values (too many, too few)
From Black Box Software Testing, copyright © 1996 – 2002 Cem Kaner
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

20. Example tests: The Triangle Program
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Example tests: The Triangle Program
Example of tests or groups of tests:
Test case for a valid equilateral triangle
At least three test cases that represent valid isosceles triangles (all permutations, e.g. 3,3,4; 3,4,3; 4,3,3)
Test case in which one side has a zero value
See Meyer’s Answer in his book
Myers 1979, page 3.
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

21. 1. Introduction 1.1 Introduction to testing – thinking like a tester
Exploratory Testing 1. 2. 3. 4. 5.
1. Introduction
1.1 Introduction to testing – thinking like a tester
1.2 Introduction to Exploratory Testing
1.3 Introduction to Risk and Risk-Based Testing
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

22. What is Exploratory Testing?
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
What is Exploratory Testing?
"Exploratory testing involves simultaneously learning, planning, running tests, and reporting / troubleshooting results."
Dr. Cem Kaner (2001)
"Exploratory testing is an interactive process of concurrent product exploration, test design and test execution.”
” To the extent that the next test we do is influenced by the result of the last test we did, we are doing exploratory testing.”
James Bach, Satisfice (2001)
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

23. Test Management and Techniques ET Planning and Documentation
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Said about eXtreme Programming
Agile software development is not conventional software development done more quickly or done on tippie-toe. Agile software development is software done differently.
Ron Jeffries, ( on agile-testing list, April 24, 2002)
proven (no single technique is new)
application oriented
planned and disciplined
controllable and reliable
risk minimizing
Two sides of extreme programming:
for the developer: freedom, flexibility, fun
for the manager: controllability, reliability, high quality
Martin Lippert (University of Hamburg), ICSTEST 2002
En ANALOGI for å forklare ET!!
Kjenner deltakerne til XP??
en “revolusjon” innen systemutvikling sammenliknet med tradisjonell “krav og spesifikasjons”-drevet (vannfalls)utvikling
med fokus på små grupper, intern kommunikasjon, mindre dokumentasjon
Analogi – likhet:
XP vs. WaterFall
ET vs. Scripted Testing
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

24. Test Management and Techniques ET Planning and Documentation
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
The eXtreme Programming and Exploratory Testing Analogy:
Agile software testing is not conventional (scripted) software testing done more quickly or done on tippie-toe.
Exploratory Testing:
proven (no single technique is new)
application oriented
planned and disciplined
controllable and reliable
risk minimizing
Two sides of Exploratory Testing:
for the tester: freedom, flexibility, fun
for the manager: controllability, reliability, high quality
En ANALOGI for å forklare ET!!
ET er IKKE tilfeldig
Planned and Disciplined
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

25. Test Management and Techniques ET Planning and Documentation
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
ET vs. Scripted Testing
Product Exploration
Test Design
Test Execution
Exploratory Testing
Fully
Scripted
Testing
Ad-hoc
Testing
Bug
Hunting
Automated
Tests
Jarle Våga (2002)
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

26. What is Scripted Testing?
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
What is Scripted Testing?
Small (but realistic) example:
How to script and test this login? (Functional tests only – not security!)
Version 1 – positive tests:
Input:
Enter a valid User-id
Enter av valid Password
Press Enter or click login
Expected Output:
The system will log you in and display main menu
Version 2 – posistive tests:
Enter User-id = AMLAND
Enter Password = OK123
Press Enter
Negative tests – even worse:
General version:
Enter combinations of negative user-id/ password with the other input as valid input.
Or extremely detailed input……(for automation).
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

27. Sample test scripts (4 of “many”):
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Sample test scripts (4 of “many”):
Sample test script 1:
Launch the Login screen
Enter User-id: “xyz”
Enter Password: “zyx”
Press <Enter>
Expected result: login ok
Sample test script 2:
Click the “Login” button
Sample test script 3:
Launch the Login screen
Enter User-id: “”
Enter Password: “zyx”
Press <Enter>
Expected: login rejected
Sample test script 4:
Click the “Login” button
Ask participants about how detailed their test scripts are?
How good are they?
What kind of errors do they NOT find?
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

28. Sample Generic Scripts (2 of “many”)
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Sample Generic Scripts (2 of “many”)
Sample generic test script 1:
Launch the Login screen
Enter valid User-id
Enter valid Password
Press <Enter> or click button
Expected result: login ok
Sample generic test script 2:
Launch the Login screen
Enter invalid User-id
Enter valid Password
Press <Enter> or click button
Expected result: login rejected
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

29. Sample test “Pattern” script (checklist)
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Sample test “Pattern” script (checklist)
Input fields:
Valid data
Invalid data
Length > max
Length = max +1
Length = max
Length = max –1
Combinations of above …
Actions:
Keyboard
Buttons …
Operations:
Add, Modify, Inquiry, Delete
What to test for each…
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

30. When to use Exploratory Testing? (1)
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
When to use Exploratory Testing? (1)
A common goal of exploration is to probe for weak areas of the program.
Test team’s resource consumption per week:
25% of the group’s time developing new tests
50% executing old tests (including bug regression)
25% on exploratory testing
Cem Kaner (2001a)
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

31. When to use Exploratory Testing? (2)
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
When to use Exploratory Testing? (2)
When there is little or no specifications and / or requirements
When you have little or no domain knowledge
When you don’t have time to specify, script and test
Uncertainty and Time Pressure!
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

32. When to use Exploratory Testing? (3)
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
When to use Exploratory Testing? (3)
Exploratory Testing is extremely useful when faced with software that is
Untested
Unknown or
Unstable
The tester must create a map of the application as he goes on testing it.
Just like ordinary Explorers; after a trip they prepare maps and detailed descriptions of the country they traveled through.
Harry Robinson,
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

33. When to use Exploratory Testing? (4)
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
When to use Exploratory Testing? (4)
Take a more scripted approach when:
There are little uncertainty about how to test
New tests are relatively unimportant
The need for efficiency and reliability in executing tests is worth the effort of scripting
We are prepared to pay the cost of documenting and maintaining tests
From Black Box Software Testing, copyright © 1996 – 2002 Cem Kaner
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

34. The Fallacy of Repeated Tests: Clearing Mines
Exploratory Testing
The Fallacy of Repeated Tests: Clearing Mines
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Posted to by James Bach, July 14, 2001.
Stale Amland saw me do the ”minefield” exercise in a testing class and asked me to write a concise description of the reasoning behind it. (I’m also looking for comments, because I want to get this right.)
Heuristic: It’s better to vary tests than to repeat the same tests.
Minefield analogy: Finding problems in a product is like looking for mines in a minefield. If you’re trying to clear a minefield, you can’t do it by traveling the same line through it over and over again. That’s how you avoid finding mines. Take different paths through the field and you’ll cover it better. Varying tests is better than repeating tests, because testing is a sampling process, and you get a larger sample of the product when you vary the tests. Since a test is a question that you’re asking the product, why ask the same questions over and over again?
Although varying tests is a powerful idea. Depending on the situation, there may be reasons to avoid variation. I know of six distinct reasons. A thoughtful tester is aware of these issues, and makes a reasoned judgment about when to vary tests and when to keep them the same.
Seven Caveats to Always-Vary-Tests
You might rationally repeat tests...
1. if there is a substantially greater probability of a problem happening in an area that is exercised by the tests, compared to other areas. The distribution of problems across a product space is not necessarily uniform.
2. if any problem that could be discovered by those tests is likely to have substantially more importance than problems in other areas. The distribution of the importance of product behavior is not necessarily uniform.
3. if they have some value and are sufficiently inexpensive compared to the cost of new and different tests. New tests may still be vitally important for the test effort, however.
4. if the tests you repeat represent the only tests that seem worth doing. This is the virus scanner argument: maybe a repeated virus scan is okay, instead of constantly changing virus tests. However, sometimes we introduce variation because we aren’t sure what tests truly are worth doing.
5. if variation of tests is specifically prohibited by contract or regulation. In other words, the point of your testing may not be to find problems.
6. if the repeated tests comprise a performance standard that gets its value by comparison with previous executions of the same exact tests. When historical test data is used as an oracle, then you must take care that the tests you perform are comparable to the historical data. Holding tests constant may not the only way to make results comparable, but it can be the best choice available.
7. when the discovery of bugs is probablistic, perhaps due to important variables involved that you can’t control in your tests. Performing a test that is, to you, exactly the same as a test you’ve performed before, may result in discovery of a bug that was always there but not revealed until the uncontrolled variables line up in a certain way. This is the same reason that a gambler at a slot machine has for playing again after losing the first time.
Non-uniform distribution
Non-uniform importance
Cheap
Virus scan
Contract
Standard
Gambling
mines
From Rapid Software Testing, copyright © James Bach
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

35. Totally Repeatable Tests Won’t Clear the Minefield
Exploratory Testing
Totally Repeatable Tests Won’t Clear the Minefield
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
mines
fixes
From Rapid Software Testing, copyright © James Bach
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

36. Variable Tests are Therefore More Effective
Exploratory Testing
Variable Tests are Therefore More Effective
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
mines
fixes
From Rapid Software Testing, copyright © James Bach
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

37. Sample Product Test Cycle
Exploratory Testing
Introduction
Test Management and Techniques
ET Planning and Documentation
ET Styles
ET Management
Sample Product Test Cycle
1. Receive the product.
Formal builds
Informal builds
Save old builds.
2. Clean your system.
Completely uninstall earlier builds.
3. Verify testability.
Smoke testing
Suspend test cycle if the product is untestable.
4. Determine what is new or changed.
Change log
5. Determine what has been fixed.
Bug tracking system
6. Test fixes.
Many fixes fail!
Also test nearby functionality.
7. Test new or changed areas.
Exploratory testing.
8. Perform regression testing.
Not performed for an incremental cycle.
Automated vs. manual
Important tests first!
9. Report results.
Coverage
Observations
Bug status (new, existing, reopened, closed)
Assessment of quality
Assessment of testability
Put ET into the BIG Picture!
The Product Testing will NOT be discussed in this workshop.
From Rapid Software Testing, copyright © James Bach
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

38. 1. Introduction 1.1 Introduction to testing – thinking like a tester
Exploratory Testing 1. 2. 3. 4. 5.
1. Introduction
1.1 Introduction to testing – thinking like a tester
1.2 Introduction to Exploratory Testing
1.3 Introduction to Risk and Risk-Based Testing
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

39. No Risk? No Test! Exploratory Testing
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

40. Typical Questions for Testers
Exploratory Testing 1. 2. 3. 4. 5.
Typical Questions for Testers
How much testing is enough?
When should we stop testing?
When is the product good enough for release?
How good is our testing?
 Managing RISK!
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

41. Product Quality and test coverage
Exploratory Testing 1. 2. 3. 4. 5.
Product Quality and test coverage
100%
Worst
Poor
Perfect!
Good
Quality Risk Coverage (potential faults)
100%
Customer use Coverage
Rex Black 1999
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting

42. Test Management and Techniques ET Planning, Exec. and Documentation
Exploratory Testing 1. 2. 3. 4. 5.
Introduction Summary
Introduction
Introduction to Testing.
What is Exploratory Testing?
Where to use it?
When to use it?
Introduction to Risk
Introduction to testing
Different testing approaches
Basic skills as a tester – a testers attitude
Context-driven testing
Objectives
What is Exploratory Testing?
Concurrent
Product Exploration
Test Design
Test Execution
Where to use it?
As part of product testing
When to use it?
When there is high uncertainty
When creating new tests is important
When high reliability and MTBF is not main focus
Test team’s resource consumption per week:
25% of the group’s time developing new tests
50% executing old tests (including bug regression)
25% on exploratory testing
Introduction to Risk
A risk is an unwanted event that has negative consequences.
2 types of risk:
Project / process risk
Business / Product risk
Test Management and Techniques
ET Planning, Exec. and Documentation
ET Styles
ET Management
ET Workshop v Introduction
©2002 Amland Consulting
©2002 Amland Consulting