Presentation is loading. Please wait.

Presentation is loading. Please wait.

Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, and Alessandro Orso Georgia Institute of Technology.

Similar presentations


Presentation on theme: "Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, and Alessandro Orso Georgia Institute of Technology."— Presentation transcript:

1 Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, and Alessandro Orso Georgia Institute of Technology

2 2 Test Machine Web Server Test Slave Web Application Tests Web Application Tests Browser Automation Framework Browser Automation Framework Web Application Web Application Tests Web Application Tests Web Application Tests Web Application Tests

3 3 Test Machine Web Server Test Slave Web Application Tests Web Application Tests Browser Automation Framework Browser Automation Framework Web Application* Changed Outdated tests Web Application Tests* Web Application Tests* Web Application Tests* Web Application Tests* Web Application Tests* Web Application Tests*

4 My Web Search Georgia Institute of Technology Includes offices, departments, news 13. room, professional education HTML Source of “My Web Search”

5 My Web Search Georgia Institute of Technology Includes offices, departments, news 13. room, professional education CommandArg 1Arg 2 open/search.php typeqGeorgia Tech clickAndWaitid=search assertTextdocument.links[0]Georgia Institute of Technology assertText//body/div[1]/spanwww.gatech.edu A Selenium Test Case for “My Web Search” HTML Source of “My Web Search”

6 My Web Search Georgia Institute of Technology Includes offices, departments, news 13. room, professional education CommandArg 1Arg 2 open/search.php typeqGeorgia Tech clickAndWaitid=search assertTextdocument.links[0]Georgia Institute of Technology assertText//body/div[1]/spanwww.gatech.edu HTML Source of “My Web Search” A Selenium Test Case for “My Web Search” Change 1

7 My Web Search Georgia Tech Includes offices, departments, news 12. room, professional education CommandArg 1Arg 2 open/search.php typeqGeorgia Tech clickAndWaitid=search assertTextdocument.links[0]Georgia Institute of Technology assertText//body/div[1]/spanwww.gatech.edu HTML Source of “My Web Search” A Selenium Test Case for “My Web Search” Change 2

8 My Web Search Advanced Search Georgia Institute of Technology Includes offices, departments, news 13. room, professional education CommandArg 1Arg 2 open/search.php typeqGeorgia Tech clickAndWaitid=search assertTextdocument.links[0]Georgia Institute of Technology assertText//body/div[1]/spanwww.gatech.edu HTML Source of “My Web Search” A Selenium Test Case for “My Web Search” Change 3

9 My Web Search Georgia Institute of Technology Includes offices, departments, news 13. room, professional education CommandArg 1Arg 2 open/search.php typeqGeorgia Tech clickAndWaitid=search assertTextdocument.links[0]Georgia Institute of Technology assertText//body/div[1]/spanwww.gatech.edu HTML Source of “My Web Search” A Selenium Test Case for “My Web Search” Change 4

10  Structural Changes leads to  locator errors  form-data population problem  Content Changes leads to  obsolete content problem  Blind Changes  do not reflect on the client side 10

11 11 Test Web-App old Web-App old Web-App new Web-App new Execute test DOM Tree + Failure Message (1) Tests with repairs Tests with repairs Tests with repairs Tests with repairs Tests with repairs Tests with repairs (2) Generate repair candidates Generate repair candidates Discard failed Suggest passed Execute candidates (3)

12  Error messages:  “Locator not found” – Moved Element  “Assertion failed” – Wrongly selected element 12 Old DOM Tree body div h1h1 h1h1 a a ul div ul div New DOM Tree body div h1h1 h1h1 a a ul div ul div DOM Node locating properties = { id, xpath, class, linkText, name }

13  Repairing failing asserts  Replace expected value with actual value  Negate assertion condition  Form data population  Find newly added elements in the DOM  Choose random values for the elements to generate repair candidates 13

14  RQ1: Can WATER suggest repairs for most broken test scripts in a web application?  RQ2: Can WATER do so without providing too many false positives? 14

15  Three real world web applications with test cases. 15 TypeContent Management System Business Process Management User Automation Scripts

16  Three real world web applications with test cases. 16 TypeContent Management System Business Process Management User Automation Scripts PlatformPHPJavaDSL

17  Three real world web applications with test cases. 17 TypeContent Management System Business Process Management User Automation Scripts PlatformPHPJavaDSL Test Scripts42 selenium tests16 selenium tests180 (out of 5123) scripts translated to selenium

18  Three real world web applications with test cases. 18 TypeContent Management System Business Process Management User Automation Scripts PlatformPHPJavaDSL Test Scripts42 selenium tests16 selenium tests180 (out of 5123) scripts translated to selenium Test scripts run onVersions from SVN Live site daily

19 VersionsT broken C fail S avg T fix V fix – (4,4)(3,3) – (1) – (1)(0) – (1,1) VersionsT broken C fail S avg T fix V fix – (1,1) 1v – (5)(1.4)1v1.29

20 20 ScriptDomainChangedFixedSuggestions 10043careers.yahoo.comFormNo(15, 15, 0) 10754www.icade.frForm, Reloc.No www.terra.com.brLink textYes www.terra.com.brLink textYes www.google.com.brLink textYes1

21  RQ1: Effectiveness  WATER could suggest repairs to broken test cases  Joomla & Ofbiz studies: Suggestions correspond to actual fixes made by developers.  RQ2: Efficiency (wrong suggestions)  Joomla & Ofbiz study: 1-3 suggestions  Coscripter study: suggestions 21

22  Repair of JUnit tests  Daniel et. al (ReAssert) [ASE’09][ISSTA’10]  Regression testing for Web applications  Alshahwan & Harman [ICST’08]  Dobolyi & Weimer [ISSRE’09]  Roest, Mesbah & Van Deursen [ICST’10]  GUI Test repair  Memon & Soffa [FSE’03]  Grechanik, Xie & Fu [ICSE’09] 22

23 23


Download ppt "Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, and Alessandro Orso Georgia Institute of Technology."

Similar presentations


Ads by Google