Presentation is loading. Please wait.

Presentation is loading. Please wait.

End to end testing (E2E): a unique test method! Rob Smit Senior Test Manager Consultant E2E Sogetis Second Testing Academy 29 April 2009.

Similar presentations

Presentation on theme: "End to end testing (E2E): a unique test method! Rob Smit Senior Test Manager Consultant E2E Sogetis Second Testing Academy 29 April 2009."— Presentation transcript:

1 End to end testing (E2E): a unique test method! Rob Smit Senior Test Manager Consultant E2E Sogetis Second Testing Academy 29 April 2009

2 Agenda 1.Complex stuff, these chains 2.Facts from the research in 2008 3.What is E2E? (the theoretical picture) 4.Whats so special about E2E? 5.The E2E project model 6.Lets make an E2E test case! 7.Some highlights from the E2E test method 8.Reference material 9.If you need us, just call us!

3 Complexity in chains 2006/11/06 How a storm in Germany caused darkness in Morocco

4 Facts from the research in 2008 E2E tests exceeds planning and budget by 200 – 600% No special test strategy for E2E No risk analysis for E2E No ownership of the chain of processes No knowledge how to set up an E2E test environment No special techniques available for E2E testing Business processes keep growing, and as they grow, they get more complex

5 The theoretical picture of E2E testing X A B Y Z AB Interface test E2E test E2E test: Test a process from start to finish. Test purpose: system and process integration Interface test: Sending data from A to B, sending data from B to A E2E test Test type where one or more processes are tested on a uninterrupted chain of systems and platforms. Interface test Testing connectivity between two systems

6 E2E testing is complex because the processes are complex x data x 3 2 14 7 6 5 8 10 9 11 xbr y x x u x z xp xq CAR insurance policy TRAVEL insurance policy LIVING insurance policySYSTEM STAND ALONE Life insurance insurance policy DAMAGE insurance policy STOCK insurance policyFIOD CONTROL BANK ACCOUNT FIN. ACCOUNTING ACCOUNTING CURRENT ACCOUNT FIN. ADMDWH MIS x x FIN. ADM GENERAL LEDGER MyWeb INTERNET DOMAIN Testing software was easy in those good old days… Testing a change was simply a matter of testing the system and maybe some regression in parts that didnt change… No more… All systems are connected, sending, receiving, processing data! Where do we begin our E2E test and where do we stop? What and where are the risks??? Risk

7 Finding risks, failures and dependencies is sometimes very difficult V A 1 2 X Y W B 3 5 Z 4 C Finding E2E risks is sometimes very difficult. Whats happening further up the process? And do we need the impact, or is the E2E chance sufficient? Should the risks in the secondary process be considered ? If so, and suppose risks are found, what do we do? Is the secondary process in scope? Can it be tested anyhow (is the knowledge accessible, do we have the budget and mandate?)

8 Special attention for the three GO/ NO GOs! In every phase of the E2E proces, full commitment of ALL stakeholders is mandatory. Remember the Chain principle (the weakest link…) End to end testing according TMap First level: E2E director, responsible for the integral E2E test plan, which also contains the business case for the E2E test Adaptive GO NO GO GO NO GO GO NO GO Make an inventory of processes and systems Analyze E2E- risks and determine feasibility Determine -Scope - Clustering -Strategy -Infra Design Integral E2E test plan Planning & Budgeting Determine limitations and measures - Determine strategy -Planning -Budget Design Partial E2E test plan Detailing data flows Second level: E2E test manager, responsible for the partial E2E test, with its own test strategy, planning and budget in a E2E test plan Setting up Infra- structure Build up E2E test environment Intake Infra- structure Third level: E2E test environment coordinator, responsible for the setting up and building the E2E test environment. Fourth level: E2E test specialist, responsible for the E2E test script and the test execution Round upExecute Specify logical test cases Specify physical test cases and initial situations Design E2E script Preparation

9 Example: an online store Order in ordering system Invoice in financial system Letter of contents in logistics system Work backwards (against the process flow) Determine the IT-products that supports the business process A proces is triggered by an actor and finishes at an actor Actor (Customer) Actor (Customer) Actor (Sales emp) Actor (Shipping agent)

10 Example: designing a E2E test case For example: a business process with a customer and an online store. The process starts when the customer (= actor) places his order. Initial situation Predicted result Action Input Processing Output This is the first test case. This test case describes the initial situation (in which circumstances can the customer places his order?) The next thing you must describe are the actions (the process steps) that must be executed to get the predicted result (a received order). Finally a test case has a predicted result. In this example, the result would be a received order (in a order management system or some kind of database. Notice: predicted result = Order

11 Example: designing E2E test cases An E2E test case must test the process from start to finish, using data flows, products and actors. The next product is the invoice, followed by the shipment bill. So all test cases (new designed and found in testware) must be connected like this example. The predicted result of test case 1 + primary data (payment) is the initial situation of test case 2. End to end test case

12 The E2E test case matrix

13 Major findings in E2E: design errors. These kind of errors are very expensive, the sooner found, the better! So start your E2E partial tests. This would be your advise ofcourse… Using partial E2E tests and connecting the tests Systems, processes or data flows ABCDEFGHIJKLMNOP XOOOOOXXOOOOOOOX Logical starting pointLogical finishing point Partial E2E test 1 Partial E2E test 3 Partial E2E test 2 Proces flow Availability for E2E test X: not available O: available E2E testStart with?End with? Partial E2E test 1 Result AResult prediction F Partial E2E test 3 Initial situation I Result prediction H Result P Partial E2E test 3 Result prediction E Initial situation F Result prediction I Imagine, systems G + H are not available (in building phase for another 6 months). What to do with the E2E test? Wait until G and H are ready, or start with partial E2E testing? E2E is almost impossible to execute as an integral test. Max. 5-8 systems are manageable, otherwise use partial E2E and connect!

14 0 0 1 1 1/1 0 0 2 2 3 3 2/1 1/1 2 2 1 1 2 2 2/2 Project environment Test environment Knowledge & knowledge accessibility Organizing & Management In project Within reach of the department Within reach of the organization Out of reach of the organization 1 1 = number of processes/ systems with complexity increasing factors 6 6 2 2 1 1 2 2 = number of processes/ systems per circle 1 1 Feasibility model 2/2 = number of processes/ systems with number of blocking issues

15 Reference material Fortis Insurance-ASR (launching customer in 2008) Elapse time User Acceptance Test: 40% less No more incidents caused by design errors E2E in critical path: not happened since KPN Telecom (pilot started in march 2009) Still in design phase of the E2E test cases, but already many possible risks and design issues are found Achmea (pilot started in march 2009) Clear view on risks in the chains Project quality already much better

16 Conclusion The business processes are increasingly complex Supporting IT-processes evidently grow in complexity The E2E test method provides the method, activities, tools and examples to handle that complexity

17 Thank you for your attention E2E test method has many new techniques White paper release May 2009 Book July 2009 English version August 2009 Course materials available May 2009 Commercial materials available May 2009 Support for any E2E issue: any time! Contacts Rob Smit ( Rob Baarda (

18 Sogetis 2 nd Testing Academy 29 April 2009

19 End to end test: dynamic testing the business processes over the integrated systems and platforms. Moment of execution: in system test or in acceptance test. Preferably as soon as possible! E2E in V-model developers tests acceptance tests system- tests functional design realisation use& maintenance wish, law, policy technical design chance, problem requirements SIT Interface test E2E test Interface test Interface test Interface test: dynamic testing the technical and functional interface behaviour. Moment of execution: system test System integration test is executed in 3 steps Interface test: by comparing the interface documentation. Moment of execution: design phase

Download ppt "End to end testing (E2E): a unique test method! Rob Smit Senior Test Manager Consultant E2E Sogetis Second Testing Academy 29 April 2009."

Similar presentations

Ads by Google