Capture-Replay and Test Automation Models and Analysis of Software Lecture 9 Copyright, 2003  Jerzy R. Nawrocki

The Scribble application 1 Belgium 2 Denmark 3 Netherlands New item added in order 1 Netherlands Initial state 1 Netherlands 2 Belgium New item added – end 1 Belgium 2 Netherlands Sorted 1 Denmark 2 Netherlands Old item deleted Enter: Belgium Enter: Denmark Enter: 1 Sort

Manual test for Scribble Belgium Netherlands UK Add 2 new items to the sorted list. Move an item (the list gets unsorted). Add an item to the unsorted list. Delete an item. Try to delete an invalid item (position) countries.dcm countries2.dcm

The screen shots – Starting the Scribble Scribble File List

The screen shots – Openning a file Scribble File List New Close Print Exit Open Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK

The screen shots – Adding an item (Sweden) Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK Add Item OK Cancel Sweden| Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK New List Sort List Del Item Mov Item Add Item Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands Sweden UK

The screen shots – Adding an item (USA) Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands Sweden UK Add Item OK Cancel USA| Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands Sweden UK New List Sort List Del Item Mov Item Add Item Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands Sweden UK USA

The screen shots – Moving an item Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands Sweden UK USA New List Sort List Add Item Del Item Mov Item Scribble File List File Name countries List Sorted Countries on our mailing list are: 1 Belgium 2 Netherlands 3 Sweden 4 UK 5 USA Move Item OK Cancel 4 From 1 To Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Belgium Netherlands Sweden USA

The screen shots – Adding an item (Norway) Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Belgium Netherlands Sweden USA Add Item OK Cancel Norway| Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Belgium Netherlands Sweden USA New List Sort List Del Item Mov Item Add Item Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Belgium Netherlands Sweden USA Norway

The screen shots – Deleting a valid item Scribble File List File Name countries List Unsorted Countries on our mailing list are: 1 UK 2 Belgium 3 Netherlands 4 Sweden 5 USA 6 Norway Delete Item From Position OK Cancel 2 Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Belgium Netherlands Sweden USA Norway New List Sort List Add Item Mov Item Del Item Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Netherlands Sweden USA Norway

The screen shots – Deleting an invalid item Scribble File List File Name countries List Unsorted Countries on our mailing list are: 1 UK 2 Netherlands 3 Sweden 4 USA 5 Norway Delete Item From Position OK Cancel 7 Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Netherlands Sweden USA Norway New List Sort List Add Item Mov Item Del Item Scribble File List File Name countries List Unsorted Countries on our mailing list are: 1 UK 2 Netherlands 3 Sweden 4 USA 5 Norway Delete Error Position Not Valid OK

The screen shots – Closing Scribble File List File Name countries List Unsorted Countries on our mailing list are: UK Netherlands Sweden USA Norway Open New Print Exit Close Scribble File List File Name countries List Unsorted Countries on our mailing list are: 1 UK 2 Netherlands 3 Sweden 4 USA 5 Norway Close Save the document? OK i i No

The screen shots – Exiting Scribble File List Scribble File List Open New Close Print Exit

What was the test input? 1 Move mouse to Scribble icon 2 Double click

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu Scribble File List

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click Scribble File List

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option Scribble File List New Close Print Exit Open

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click Scribble File List New Close Print Exit Open

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click 11 Move mouse to Add Item Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK New List Sort List Del Item Mov Item Add Item

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click 11 Move mouse to Add Item 12 Click Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK New List Sort List Del Item Mov Item Add Item

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click 11 Move mouse to Add Item 12 Click 13 Type Sweden Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK Add Item OK Cancel Sweden|

What was the test input? 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click 11 Move mouse to Add Item 12 Click 13 Type Sweden 14 Move mouse to OK button Scribble File List File Name countries List Sorted Countries on our mailing list are: Belgium Netherlands UK Add Item OK Cancel Sweden|

The manual test process 1 Move mouse to Scribble icon 2 Double click 3 Move mouse to File menu 4 Click 5 Move mouse to Open option 6 Click 7 Move mouse to countries.dcm 8 Double click 9 Move mouse to List menu 10 Click 11 Move mouse to Add Item 12 Click 13 Type Sweden 14 Move mouse to OK button The amount of effort to automate a test case: 2 – 10 times the effort required to run the test manually (!)

The manual test process Unscripted manual testing: 1.Identify conditions to test 2.Think up specific inputs 3.Enter the inputs 4.Checked it worked OK

The manual test process Unscripted manual testing Vague manual scripts: 1.Read what to do 2.Think up specific inputs 3.Enter the inputs 4.Checked it worked OK

Vague manual script 1 Run up Scribble File menu displayed 2 Open file with sorted ListFile contents displayed 3 Add some items to ListItems added in order 4 Move an itemItem moved, List now unsorted 5 Add an itemItem added at end of List 6 Delete item form listItem deleted 7 Use invalid position numberError message displayed to delete an item 8 Save changes in new fileEnd of test

The manual test process Unscripted manual testing Vague manual scripts Detailed manual scripts: 1.Read what to do 2.Enter the inputs 3.Checked it worked OK

Detailed manual script 1 Click on Scribble iconScribble opened, File menu enabled 2 Move mouse to File menu, clickOptions available: Open, New, Exit 3 Move mouse to Open option,Filenames displayed including clickcountries.dcm 4 Move mouse to countries.dcmText plus List of 3 countries: Belgium, Netherlands, UK, File and List menus displayed...

Automating test execution Automated scripts are not like manual scripts Don’t automate testing by simply recording Automated execution with manual verification? How much shoul be compared (whole, minimum, something between)? Dynamic comparison and post-execution comparison Automated comparison messages have to be manually checked While tests fail the second time?

Conclusion Automated is not automatic Automated is not automatic

Bibliography M. Fewster, D. Graham, Software Test Automation, Addison-Wesley, Harlow, M. Fewster, D. Graham, Software Test Automation, Addison-Wesley, Harlow, 1999.

Questions?