1. Introduction to Test Automation Jerzy.Nawrocki@put.poznan.pl www.cs.put.poznan.pl/jnawrocki/models/ Models and Analysis of Software Lecture 8 Copyright, 2003  Jerzy R. Nawrocki

2. What is test automation? That’s simple! 1. Buy a test execution tool. 2. Record the manual tests. 3. Play the tests back whenever you want to. „Just as there is more to software design than knowing a programming language, there is more to automating testing than knowing a testing tool.” -- M. Fewster & D. Graham

3. Quality attributes of a test case How effective in detecting defects? How exemplary ? (the more exemplary, the less test cases needed) How economic ? How evolvable ? (maintenance effort)

4. Test automation effects Effective Exemplary Evolvable Economic

5. Test activities Identify test conditions (‘what’ to test) and prioritize Design test cases (‘how’ to test) Build test cases (scripts, data etc.) Execute test cases Compare test outcomes to expected outcomes

6. Example test conditions Order created for a single item (VB10) Order quantity of 100 (VB23) Order cancelled (V8) ItemPrice [zl] Quantity Rose25 Bag31

7. An example test case Prerequisites: logged into the purchase order system as a data entry clerk; database system must contain the standard Data Set; no other new purchase order activity on the system. StepInputExp.outCond. 1 Create a new order for any one standard order, setting quantity to 100 Order confirmation message displayed VB10 VB23 2 Confirm the orderPurchase order printed VB10 3 Print a new orders reportNew orders report printed showing just this one new order VB10 VB23 4 Cancel the orderPurchase order cancellation notice printed V8

8. The limitations of automating software testing Does not replace manual testing Manual tests find more defects than automated tests James Bach: automated tests= 15%, manual= 85% Test automation does not improve effectiveness Test automation may limit software evolution Tools have no imagination

9. Summary Test-first coding for GUI is a real problem. Test-first coding for GUI is a real problem. Rational Robot is for regression testing, not for test-first coding. Rational Robot is for regression testing, not for test-first coding. SQABasic is an interesting language but it is not clear how to use it for test-first coding of GUI. SQABasic is an interesting language but it is not clear how to use it for test-first coding of GUI. Further research is necessary to solve the problem. Further research is necessary to solve the problem.

10. Bibliography M. Fewster, D. Graham, Software Test Automation, Addison-Wesley, Harlow, 1999. M. Fewster, D. Graham, Software Test Automation, Addison-Wesley, Harlow, 1999.

11. Questions?