1. X-PERT: Accurate Identification of Cross-Browser Issues in Web Applications Shauvik Roy Choudhary, Mukul Prasad, Alessandro Orso Labs of America Partially supported by

2. Web Applications 2

3. 3

4. 4 Mozilla Firefox Internet Explorer

5. 5 Mozilla Firefox Internet Explorer

6. 6 Google Chrome

7. 7 Apple Safari

8. 8 Mozilla Firefox

9. 9

10. 10 Mozilla Firefox Internet Explorer

11. 11 Mozilla Firefox Internet Explorer

12. Web Developer

15. Browser versions last year Source: StatCounter.com

16. Browser versions last year with at least 1% market share 150mn visits/month ( StatCounter.com)

20. Challenges Manual inspection is expensive DOM differs between browsers Mimic end user’s perception Produce readable, effective & actionable reports Engineering Issues… Modern apps have many dynamic screens

21. Detection of Cross-Browser Issues (XBI) Web Application Model Generation Model Comparison Error Report

22. Model Definition and Comparison WebDiff [ICSM’10] Roy Choudhary and Orso CrossCheck [ICST’12] Roy Choudhary, Prasad and Orso CrossT [ICSE’11] Mesbah and Prasad Screen Transition Graph Screen Model Screen image + geometry DOM Tree

23. Study of Real World Cross-Browser Issues (XBIs) Examined 100 websites Found 23 with XBIs XBI Types o Behavioral XBIs o Content XBIs o Structural XBIs 9% 22% 30% 57% Text Visual Random URL Generator

24. Limitations of existing techniques Miss Structural XBIs Focus on symptoms rather than causes o Duplicate reports o Low Precision

25. Comprehensive XBI Detection Behavioral XBIs Content XBIs Structural XBIs Text Visual

26. Example

27. Example

28. Example Behavior XBI 9% Behavior XBI 9%

29. Example Content XBI (Visual) – 30% Content XBI (Visual) – 30% Content XBI (Text) – 22% Content XBI (Text) – 22% Structural XBI 57% Structural XBI 57%

30. Comprehensive XBI Detection Behavioral XBIs Content XBIs Structural XBIs Text Visual

31. Comprehensive XBI Detection Behavioral XBIs Content XBIs Structural XBIs Text Visual Behavior check using Graph Isomorphism CrossCheck [ICST’12] Roy Choudhary, Prasad and Orso CrossT [ICSE’11] Mesbah and Prasad

32. Comprehensive XBI Detection Behavioral XBIs Content XBIs Structural XBIs Text Visual Text Content check using String equality WebDiff [ICSM’10] Roy Choudhary and Orso CrossCheck [ICST’12] Roy Choudhary, Prasad and Orso CrossT [ICSE’11] Mesbah and Prasad

33. Comprehensive XBI Detection Behavioral XBIs Content XBIs Structural XBIs Text Visual Visual Content check using Image Matching Computer Vision WebDiff [ICSM’10] Roy Choudhary and Orso CrossCheck [ICST’12] Roy Choudhary, Prasad and Orso

34. Detecting Structural XBIs Intuition: Disruption in the relative arrangement of web page elements leads to Structural XBIs Idea: Abstraction to represent relative arrangement and compare it to expose errors

35. Alignment Graph (AG) A graph capturing structural relationships between elements (rectangles) on the webpage An AG has: o Containment relationships o Alignment relationships o Attributes above right-align left-align

36. Step 1: AG Construction 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

37. Step 1: AG Construction a. Containment Relationship e contains e’ iff ( x 1 ≤ x 1 ’ ∧ y 1 ≤ y 1 ’ ∧ x 2 ≥ x 2 ’ ∧ y 2 ≥ y 2 ’ ), and if same bounds and XPath(e) ≤ XPath(e’) e e’

38. Step 1: AG Construction a. Containment Relationship

39. Step 1: AG Construction b. Alignment Relationship

40. Step 1: AG Construction c. Attributes Button 1 Button 2 What alignment attributes can we infer? Button 1 is on the left of Button 2 Button 1 and Button 2 have their tops aligned Button 1 and Button 2 have their bottoms aligned leftOf top-align bottom-align y 1 = y 1 ’y 2 = y 2 ’ x 2 < x 1 ’

41. Step 1: AG Construction c. Attributes Button 1 Button 2 leftOf top-align bottom-align

42. Step 2: Matching AGs

44. Text Visual Comprehensive XBI Detection Behavioral XBIs Content XBIs Structural XBIs Text Visual Report Generation Model Generation Behavior Checker Element Matcher Structure Checker Content Checker Model Comparison ✔ ✔ ✔

45. Empirical Evaluation Tool: X-PERT (Cross-Platform Error ReporTer) Research Questions: RQ1: Can X-PERT find XBIs in real web applications? RQ2: How does X-PERT compare to the state-of-art?

46. Experimental Protocol v14.0.1v9.0.9 v/s Ran X-PERT on Subject Applications Subjects + = XBIs (Ground Truth) Manual Check Error Reports Manual Check = False Positives & Negatives +

47. Subjects NameTypeStatesTransitions DOM Nodes (per screen) maxminaverage OrganizerProductivity1399274821000113051 GrantaBooksPublisher98378001562525852 DesignTrustBusiness102026437777218694 DivineLifeSpiritual109140611908249886 SaiBabaReligious13204260652412162 BreakawaySport191845148819113059 ConferenceInformation312878817853 FishermanRestaurant1517391461572021336 ValleyforgeLodge412541647335046 UniMelbUniversity98151421213113792 KonquerorSoftware54175861546816187 UBCClub7720610783412094 BMVBSMinistry520194901254415695 StarWarsMovie109284521971922626 800 - 140K elements Prior Art (6) Example and Survey (4) Random (4)

48. Effectiveness NAMEXBI BEHAVIORSTRUCTURE CONTENT TOTAL TEXTIMAGE TPFPTPFPTPFPTPFPTPFP Organizer1010900000 0 GrantaBooks271601100000270 DesignTrust72053000073 DivineLife1170361000 6 SaiBaba52029000049 Breakaway13001020000 2 Conference72030101070 Fisherman51031011052 Valleyforge30022001032 UniMelb22000000121 Konqueror00000000606 UBC00000000000 BMVBS10000000000 StarWars1200 00000 0 TOTAL103330602321379831 RQ1: Can X-PERT find XBIs in real web applications? Answer: Yes RQ1: Can X-PERT find XBIs in real web applications? Answer: Yes

49. Improvement NAMEXBI X-PERTCROSSCHECK PrecFPReclDpPrecFPReclDp Organizer10100%0 080%2 13 GrantaBooks27100%0 096%1100%0 DesignTrust770%3 100 %05%12286%3 DivineLife1165%6 100 %029%2491%3 SaiBaba531%980%07%5380%10 Breakaway1383%277%113%4954%12 Conference7100%0 0 0 0 Fisherman571%2 100 %044%580%8 Valleyforge360%2 100 % 050%133%0 UniMelb267%1 100 % 07%27100%0 Konqueror0NA6 00%11NA0 UBC0NA0 00%1NA0 BMVBS1NA0 00%2 0 StarWars12100%0 010%9183%3 TOTAL10377% 313195%132%38974%52 X-PERT Precision = 77% (45%  ) Recall = 95% (14%  ) RQ2: How does X-PERT compare to the state-of-art? Answer: X-PERT has better precision and recall RQ2: How does X-PERT compare to the state-of-art? Answer: X-PERT has better precision and recall

50. Experimental Data & Tool Release o Experiments (Crawl Data + Reports) o Layout testing algorithm implementation http://gatech.github.io/xpert

51. Related Work o Industrial tools Test Suites for Web Browsers – Acid and test262 MS Expression Web, BrowserShots, BrowserStack Browsera, MogoTest, BrowserBite o Research techniques Eaton & Memon [ IJWET’07 ] Dallmeier, Burger, Orth & Zeller [ JSTools’12 ] 51

52. Contributions Empirical evaluation shows effectiveness New Structural XBI Detection Algorithm Comprehensive Technique to detect XBIs Study of Real World XBIs

53. Thank You Shauvik Roy Choudhary, Mukul Prasad, Alex Orso