1. End to end testing (E2E): a unique test method!
Rob Smit
Senior Test Manager
Consultant E2E
Sogeti’s Second Testing Academy 29 April 2009

2. Agenda Complex stuff, these chains Facts from the research in 2008
What is E2E? (the theoretical picture)
What’s so special about E2E?
The E2E project model
Let’s make an E2E test case!
Some highlights from the E2E test method
Reference material
If you need us, just call us!

3. How a storm in Germany caused
Complexity in chains
How a storm in Germany caused
darkness in Morocco
2006/11/06

4. Facts from the research in 2008
E2E tests exceeds planning and budget by 200 – 600%
No special test strategy for E2E
No risk analysis for E2E
No ownership of the chain of processes
No knowledge how to set up an E2E test environment
No special techniques available for E2E testing
Business processes keep growing, and as they grow, they get more complex

5. The theoretical picture of E2E testing
Interface test X A B Y Z A B
Interface test
Testing connectivity
between two systems
E2E test
Test type where one or more processes are tested on a uninterrupted chain
of systems and platforms.
E2E test:
Test a process from start to finish. Test purpose: system and process integration
Interface test:
Sending data from A to B, sending data from B to A

6. E2E testing is complex because the processes are complex
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
E2E testing is complex because the processes are complex
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Life insurance
insurance policy
Risk
Risk
BANK
ACCOUNT
FIOD
CONTROL
Risk
Risk
Risk
MyWeb
INTERNET DOMAIN
Risk 5 x b
Risk r
TRAVEL
insurance policy
Risk
Risk
Risk 9
Risk x
Risk
Risk x
Risk x
Risk
Risk
Risk
Risk u y
Risk
Risk
Risk
Risk
Risk
Risk
Risk 7
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
DAMAGE
insurance policy
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk x 11
CURRENT
ACCOUNT
FIN. ADM
Risk
Risk
Risk
Risk
FIN. ACCOUNTING
ACCOUNTING
FIN. ADM
GENERAL
LEDGER
Risk
Risk
Risk
Risk
Risk
Risk 6
Risk
Risk
Risk
data
Risk
STOCK
insurance policy
Risk
Risk
SYSTEM
STAND
ALONE
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
LIVING
insurance policy
Risk 1 2 3 4
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk z
Risk
Risk
Risk
Risk
Risk
Risk 10
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
CAR
insurance policy
Risk
Risk
Risk
DWH
MIS
Risk
Risk
Risk
Risk
Risk
Risk x p
Risk
Risk
Risk
Risk x
Risk 8
Risk
Risk
Risk
Risk
Risk
Testing software was easy in those good old days…
Testing a change was simply a matter of testing the system and maybe some regression in parts that didn’t change…
No more…
All systems are connected, sending, receiving, processing data! Where do we begin our E2E test and where do we stop? What and where are the risks??? x q x

7. Finding risks, failures and dependencies is sometimes very difficult1 2 V 3 W A 4 X B Y 5 C Z
Should the risks in the secondary process be considered ? If so, and suppose risks are found, what do we do? Is the secondary process in scope? Can it be tested anyhow (is the knowledge accessible, do we have the budget and mandate?)
Finding E2E risks is sometimes very difficult. What’s happening further up the process? And do we need the impact, or is the E2E chance sufficient?

8. End to end testing according TMap
Adaptive
Make an
inventory of
processes
and systems
Analyze E2E- risks and determine feasibility
Determine
-Scope
- Clustering
Strategy
Infra
Design
Integral E2E test plan
Planning
& Budgeting GO
NO GO
Determine limitations and measures
- Determine strategy
-Planning
Budget
Design
Partial E2E test plan
Detailing data flows
Setting up Infra-structure
Build up E2E test environment
Intake
Infra-structure
Round up
Execute
Specify logical test cases
Specify physical test cases and initial situations
Design
E2E script
Preparation
Third level: E2E test environment coordinator, responsible for the setting up and building the E2E test environment.
Fourth level: E2E test specialist, responsible for the E2E test script and the test execution
Second level: E2E test manager, responsible for the partial E2E test, with it’s own test strategy, planning and budget in a E2E test plan
Special attention for the three GO/ NO GO’s! In every phase of the E2E proces, full commitment of ALL stakeholders is mandatory. Remember the Chain principle (the weakest link…)
First level: E2E director, responsible for the integral E2E test plan, which also contains the business case for the E2E test

9. Example: an online store
Actor
(Customer)
Actor
(Customer)
Invoice in financial system
Actor
(Sales emp)
Actor
(Shipping agent)
Work backwards (against the process flow)
Order in ordering system
Letter of contents in logistics system
Determine the IT-products that supports the business process
A proces is triggered by an actor and finishes at an actor

10. Example: designing a E2E test case
Initial situation
Action
Input
Processing
Output
Predicted result
Finally a test case has a predicted result. In this example, the result would be a received order (in a order management system or some kind of database. Notice: predicted result = Order
The next thing you must describe are the actions (the process steps) that must be executed to get the predicted result (a received order).
This is the first test case. This test case describes the initial situation (in which circumstances can the customer places his order?)
For example: a business process with a customer and an online store. The process starts when the customer (= actor) places his order.

11. Example: designing E2E test cases
End to end test case
An E2E test case must test the process from start to finish, using data flows, products and actors. The next product is the invoice, followed by the shipment bill.
So all test cases (new designed and found in testware) must be connected like this example. The predicted result of test case 1 + primary data (payment) is the initial situation of test case 2.

12. The E2E test case matrix

13. Using partial E2E tests and connecting the tests
Logical starting point
Logical finishing point
Systems, processes or data flows A B C D E F G H I J K L M N O P
Proces flow X O O O O O X X O O O O O O O X
Availability for E2E test
X: not available
O: available
Partial E2E test 1
Partial E2E test 2
Partial E2E test 3
E2E test
Start with?
End with?
Partial E2E test 1
Result A
Result prediction F
Partial E2E test 3
Initial situation I
Result prediction H
Result P
Result prediction E Initial situation F
Result prediction I
E2E is almost impossible to execute as an integral test. Max. 5-8 systems are manageable, otherwise use partial E2E and connect!
Major findings in E2E: design errors. These kind of errors are very expensive, the sooner found, the better! So start your E2E partial tests. This would be your advise ofcourse…
Imagine, systems G + H are not available (in building phase for another 6 months). What to do with the E2E test? Wait until G and H are ready, or start with partial E2E testing?

14. knowledge accessibility
Feasibility model
Project
environment
Test
Knowledge &
knowledge accessibility
Organizing &
Management
2/2
Within reach of the organization
Out of reach of the organization 1
= number of processes/
systems per circle 1 2
Within reach of the department 1
1/1 2
In project 6 2
1/1
2/1 3 2
1/1
1/1 1
1/1
1/1 1
= number of processes/ systems with complexity increasing factors 2
2/2
= number of processes/ systems with number of blocking issues

15. Fortis Insurance-ASR (launching customer in 2008)
Reference material
Fortis Insurance-ASR (launching customer in 2008)
Elapse time User Acceptance Test: 40% less
No more incidents caused by design errors
E2E in critical path: not happened since
KPN Telecom (pilot started in march 2009)
Still in design phase of the E2E test cases, but already many possible risks and design issues are found
Achmea (pilot started in march 2009)
Clear view on risks in the chains
Project quality already much better

16. The business processes are increasingly complex
Conclusion
The business processes are increasingly complex
Supporting IT-processes evidently grow in complexity
The E2E test method provides the method, activities, tools and examples to handle that complexity

17. Thank you for your attention
E2E test method has many new techniques
White paper release May 2009
Book July 2009
English version August 2009
Course materials available May 2009
Commercial materials available May 2009
Support for any E2E issue: any time!
Contacts
Rob Smit
Rob Baarda

18. Sogeti’s 2nd Testing Academy 29 April 2009 www.sogeti.com

19. System integration test is executed in 3 steps
E2E in V-model
wish, law, policy
chance, problem
use&
maintenance
requirements
acceptance
tests
E2E test
functional
design
Interface
test
technical
design
system-
tests
Interface test
developers
tests
realisation
SIT
System integration test is executed in 3 steps
Interface test: by comparing the interface documentation.
Moment of execution: design phase
Interface test: dynamic testing the technical and functional interface behaviour.
Moment of execution: system test
End to end test: dynamic testing the business processes over the integrated systems and platforms. Moment of execution: in system test or in acceptance test. Preferably as soon as possible!