Automation Testing Tools UNIT 4

Agenda Overview Information Gathering Tools and Vendors Evaluation Criteria Tools Evaluation Matrix Conclusion

Overview “Automated Testing” means automating the manual testing process currently in use. This requires that a formalized "manual testing process" currently exists in the company or organization. Minimally, such a process includes: Detailed test cases, including predictable "expected results", which have been developed from Business Functional Specifications and Design documentation. A standalone Test Environment, including a Test Database that is restorable to a known constant, such that the test cases are able to be repeated each time there are modifications made to the application.

Information Gathering Testers working Motorola labs Siemens Medical Solutions Online resources IEEE and ACM papers Vendor’s web-site Forums

Information Gathering Following are sample questions asked to tester who have been using some the testing tools: How long have you been using this tool and are you basically happy with it? How many copies/licenses do you have and what hardware and software platforms are you using? How did you evaluate and decide on this tool and which other tools did you consider before purchasing this tool? How does the tool perform and are there any bottlenecks? What is your impression of the vendor (commercial professionalism, on-going level of support, documentation and training)?

Tools and Vendors Robot – Rational Software WinRunner 7 – Mercury QA Run 4.7 – Compuware Visual Test – Rational Software Silk Test – Segue QA Wizard – Seapine Software

Tools Overview Robot – Rational Software WinRunner 7 – Mercury IBM Rational Robot v2003 automates regression, functional and configuration testing for e-commerce, client/server and ERP applications. It's used to test applications constructed in a wide variety of IDEs and languages, and ships with IBM Rational TestManager. Rational TestManager provides desktop management of all testing activities for all types of testing. WinRunner 7 – Mercury Mercury WinRunner is a powerful tool for enterprise wide functional and regression testing. WinRunner captures, verifies, and replays user interactions automatically to identify defects and ensure that business processes work flawlessly upon deployment and remain reliable. WinRunner allows you to reduce testing time by automating repetitive tasks and optimize testing efforts by covering diverse environments with a single testing tool.

Tools Overview QA Run 4.7 – Compuware With QA Run, programmers get the automation capabilities they need to quickly and productively create and execute test scripts, verify tests and analyze test results. Uses an object-oriented approach to automate test script generation, which can significantly increase the accuracy of testing in the time you have available. Visual Test 6.5 – Rational Software Based on the BASIC language and used to simulate user actions on a User Interface.  Is a powerful language providing support for pointers, remote procedure calls, working with advanced data types such as linked lists, open-ended hash tables, callback functions, and much more. Is a host of utilities for querying an application to determine how to access it with Visual Test, screen capture/comparison, script executor, and scenario recorder.

Tools Overview Silk Test – Segue QA Wizard – Seapine Software Is an automated tool for testing the functionality of enterprise applications in any environment. Designed for ease of use, Silk Test includes a host of productivity-boosting features that let both novice and expert users create functional tests quickly, execute them automatically and analyze results accurately. In addition to validating the full functionality of an application prior to its initial release, users can easily evaluate the impact of new enhancements on existing functionality by simply reusing existing test casts. QA Wizard – Seapine Software Completely automates the functional regression testing of your applications and Web sites. It’s an intelligent object-based solution that provides data-driven testing support for multiple data sources. Uses scripting language that includes all of the features of a modern structured language, including flow control, subroutines, constants, conditionals, variables, assignment statements, functions, and more.

Evaluation Criteria Record and Playback l Object Mapping Web Testing l Object Identity Tool Environment Support l Extensible Language Cost l Integration Ease of Use l Image Testing Database Tests l Test/Error Recovery Data Functions l Object Tests Support

3 = Basic 2 = Good 1 = Excellent

Strength & Weakness of Tools

Tool Selection Recommendation Tool evaluation and selection is a project in its own right. It can take between 2 and 6 weeks. It will need team members, a budget, goals and timescales. There will also be people issues i.e. "politics". Start by looking at your current situation Identify your problems Explore alternative solutions Realistic expectations from tool solutions Are you ready for tools?

Tool Selection Recommendation Make a business case for the tool What are your current and future manual testing costs? What are initial and future automated testing costs? What return will you get on investment and when? Identify candidate tools Identify constraints (economic, environmental, commercial, quality, political) Classify tool features into mandatory & desirable Evaluate features by asking questions to tool vendors Investigate tool experience by asking questions to other tool users Plan and schedule in-house demonstration by vendors Make the decision

Tool Selection Recommendation Choose a test tool that best fits the testing requirements of your organization or company. An "Automated Testing Handbook" is available from the Software Testing Institute (www.ondaweb.com/sti), which covers all of the major considerations involved in choosing the right test tool for your purposes.

References www.rational.com www.mercury.com www.seque.com www.seapine.com www.compuware.com www.testing.com www.sqa-test.com

Automation Tools For Testing.

Overview Why to Use the Automation? What to Automate? How many types of testing can be done? Cost Factor? Conclusion

Manual Testing May Be Problematic Repetative Take Resources May Lack coverage Potentially Inconsistent.

Why Use Testing Tools? Manual Testing Automated Testing Time consuming Low reliability Human Resources Inconsistent. Speed Repeatability Programing Capabilities Coverage Reliability Reusability

Which Test Case To Automate? Tests That need to run for every build of the web site. Tests that use multiple data values for the same actions (Data Driven tests) Idetical tests that need to be executed using different browsers

Which Test Cases Not to Automate? Usability testing One-Time Testing “ASAP” Testing Tests Without Predictable Results

From Manual to Automated Manual Testing Plan the Manual Test Perform the user actions Verify site under test functions as expected. Automated Testing Plan the Automated test Record user actions to generate the baseline test Enhance for web testing and playback automated test

Types of Automated Testing Fuctional Testing Performance Testing Load Testing Stress Testing Regression Testing Resource Usage Testing Memory Analyzer Testing Data Driven Testing Web (Performance & Functional) Testing And others

Testing Tools Winrunner with Test Director Loadrunner Rational Robot with Test Manager Silk Test OpenSTA Test Complete 3.8 WAPT (Web Application Testing) Suite Deluge ApTest (Bug and DefectsTracking Tools) AdventNet QEngine And 1000 of more tools are available.

How The Automation Tools working? (Winrunner)

GUI Map File

Cost Factor ShareWare Softwares FreeWare Softwares Winrunner with Test Director Loadrunner Rational Robot with Test Manager Silk Test TestComplete WAPT AdventNet QEngine FreeWare Softwares OpenSTA Deluge ApTest (Bug and DefectsTracking Tools) TPTest Suite Test Maker Open Load

Conclusion Automation is worst in use. Inconsistent. ReliabilityAutomation Tools Provides the only scalable solution for testing enterprise application. Repetability. Time saving.

References www.Mercury.com www.aptest.com www.sqa.test.com www.Softwareqatest.com www.wintask.com www.opensourcetesting.org www.soft.com www.AutomatedQA.com www.iBeta.com www.vmware.com www.bitpipe.com www.test.com