Presentation is loading. Please wait.

Presentation is loading. Please wait.

Test process analysis of Gateway GPRS Support Node Author:Jorma Tuominen Supervisor:Professor Sven-Gustav Häggman Nokia Networks.

Similar presentations


Presentation on theme: "Test process analysis of Gateway GPRS Support Node Author:Jorma Tuominen Supervisor:Professor Sven-Gustav Häggman Nokia Networks."— Presentation transcript:

1 Test process analysis of Gateway GPRS Support Node Author:Jorma Tuominen Supervisor:Professor Sven-Gustav Häggman Nokia Networks

2 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 2 (15) Agenda Introduction GGSN in GPRS and 3G packet core network GGSN in Nokia Intelligent Content Delivery system Software development process and testing phases Research problem Research method Results Conclusions

3 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 3 (15) GPRS and 3G packet core network BSC RNC 3G SGSN 2G SGSN NMS BG GGSN LIG Internet FW Corporate customer Mobile ISP Intranet services WAP MMS IP backbone CG Inter-PLMN network BTS BS News Sports Games Trading

4 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 4 (15) Nokia Intelligent Content Delivery System ICD GGSNTA CGOSCNSM user data CA GTP’RADIUS or DIAMETER LDAP user data + RADIUS

5 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 5 (15) SW development process and testing phases

6 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 6 (15) Research problem Testing requires tradeoffs between cost, time and quality. How available testing time and resources can be used as effectively as possible to find and correct the critical defects from the product before it is released?

7 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 7 (15) Research method Participating in GGSN system testing Literature study of GPRS and 3G packet core network elements and protocols Literature study of software process and software testing Collecting and analyzing testing metrics Defect analysis Risk analysis Planning risk based testing

8 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 8 (15) Testing metrics Testing effort metrics Distribution of working hours by test phases Defect metrics Number of discovered defects per week Percentage of “correction not needed” defect reports Test case metrics Number of test cases Percentage of not relevant test cases Testing effectiveness metrics Hours needed to discover one defect Number of test cases needed to discover one defect Defect Removal Effectiveness

9 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 9 (15) Testing metrics examples Distribution of working hours by test phases (lessons learned) Number of discovered defects per week (real-time metrics)

10 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 10 (15) Defect analysis Search for the error prone functionality areas Defects discovered in the earlier product releases Defects discovered in the earlier testing phases Defects discovered by the customers Can be done after product release or during testing Pareto phenomena Many software phenomena follow a Pareto distribution: 80% of contribution comes from 20% of the contributors [Boehm 1989] Examples: 20% of the software modules contribute 80% of the errors. 20% of the errors cause 80% of the down time

11 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 11 (15) Risk analysis System requirements must be prioritised What is the likelihood that this feature will fail to operate? What would be the impact on the user if this feature will fail to operate? The likelihood of failure depends on the following: error proneness of the requirement newness of the requirement complexity of the requirement Likelihood of failure: 1 = very high,... 5 = very low Impact on the user: 1 = severe,... 5 = negligible Risk priority = (Likelihood of failure) x (Impact on the user)

12 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 12 (15) Risk based testing Risk 100% 50% Testing effort E2E2 E1E1 Risk based testing Traditional testing

13 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 13 (15) Results Requirements have been divided into three groups: High, medium and low risk requirements Test case titles will be planned for each requirement Test cases will be designed and executed as follows: High risk requirements: 100% of test cases Medium risk requirements: 70% of test cases Low risk requirements: 30 % of test cases Test design and execution effort can be reduced about 30%. Effect on the quality of the product can not be estimated yet, because the product is not yet released.

14 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 14 (15) Conclusions and future work The availability of the real-time testing metrics should be improved. Co-operation between the product development, the testing and the customer support should be strengthened. The most important future work is to harmonize the usage of requirement management database, test case database and defect tracking database.

15 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 15 (15) Thank You! Questions?

16 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 16 (15) Additional slides

17 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 17 (15) Cost of defects Phase where defect is discovered Relative cost to correct a defect Requirements1 Design3-6 Coding10 Development testing15-40 Acceptance testing30-70 Production Phase where defect is discovered Relative cost to correct a defect Definition1 High-level design2 Low-level design5 Code10 Unit test15 Integration test22 System test50 Post delivery100+

18 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 18 (15) Early involvement of the testers Important aspects of early involvement of the testers are: “Testers need a solid understanding of the product so they can devise better and more complete test plans, designs, procedures, and cases. Early test- team involvement can eliminate confusion about functional behavior later in the project lifecycle. In addition, early involvement allows the test team to learn over time which aspects of the application are the most critical to the end user and which are the highest-risk elements. This knowledge enables testers to focus on the most important parts of the application first, avoiding over-testing rarely used areas and under-testing the more important ones.” [Dustin 2003].

19 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 19 (15) Thoughts about testing “Program testing can be used to show the presence of bugs, but never to show their absence!” [Dijkstra 1969]. Testing completion criteria: Risk of stopping testing = Risk of continuing testing “In most cases ‘what’ you test in a system is much more important than ‘how much’ you test” [Craig 2002]. “Prioritise tests so that, when ever you stop testing, you have done the best testing in the time available” [ISEB 2003]. No risk  No test

20 Jorma Tuominen Test process analysis of Gateway GPRS Support Node 20 (15) References Myers G. The art of software testing, USA, John Wiley & Sons, ISBN , 1979, 177 p. Hetzel B. Complete Guide to Software Testing, USA, John Wiley & Sons, ISBN , 1988, 284 p. Craig R., Jaskiel S. Systematic Software Testing, USA, Artech House Publishers, ISBN , 2002, 536 p. Dijkstra E. W. Structured programming. Proceedings of the Second NATO Conference on Software Engineering Techniques, Rome Italy, NATO, 1969, pp Dustin E. Effective Software Testing, USA, Addison-Wesley, ISBN , 2003, 271 p. Bender R. Requirements Based Testing presentation. Proceedings of StarEast 2005, Software testing analysis & review conference, USA Orlando, Bender RBT Inc. 2005, 232 p. Beizer B. Black-Box Testing: Techniques for Functional Testing of Software and Systems, USA, John Wiley & Sons, ISBN , 1995, 294 p. Koomen T., Pol M. Test Process Improvement: A practical step-by-step guide to structured testing, Great Britain, Addison-Wesley, ISBN , 1999, 218 p. Hutcheson M. Software Testing Fundamentals: Methods and Metrics, USA, John Wiley & Sons, ISBN X, 2003, 408 p. Boehm B. W. Software Risk Management, USA, IEEE Press, ISBN , 1989, 495 p. Grove Consultants, ISEB testing foundation course material, Great Britain, Grove Consultants, 2003, 98 p.


Download ppt "Test process analysis of Gateway GPRS Support Node Author:Jorma Tuominen Supervisor:Professor Sven-Gustav Häggman Nokia Networks."

Similar presentations


Ads by Google