1. Applying Mutation Testing to Web Applications Upsorn Praphamontripong and Jeff Offutt Software Engineering George Mason University Fairfax, VA USA www.cs.gmu.edu/~offutt/ uprapham@gmu.edu, offutt@gmu.edu

2. Web App Software Failures Huge losses due to web application failures –Financial services : $6.5 million per hour –Credit card sales applications : $2.4 million per hour Oct 2004: Paypal waived transaction fees for an entire day because of a service outage after an upgrade Dec 2006: amazon.coms BOGO offer turned into a double discount 2007 : Symantec said that most security vulnerabilities are due to faulty software July 2008: Amazon failure caused businesses to lose information and revenue Mutation 2010© Praphamontripong and Offutt2 World-wide monetary loss due to poor software is staggering Thanks in part to Dr. Sreedevi Sampath

3. This Research Where do faults reside in web applications ? –In the novel control connections among web components –In the novel state management facilities –In the interfaces between clients and servers Key research problems : 1.Model web application faults 2.Invent new mutation operators for these faults 3.Design a mutation analysis tool for web applications 4.Evaluate operators, applicability and practicality Mutation 2010© Praphamontripong and Offutt3 Improve our ability to test web applications by using mutation analysis

4. Modeling Web Applications Mutation 2010© Praphamontripong and Offutt4 Atomic section model –Offutt and Wu, July 2009 Atomic SectionHTML –Server-generated section: all or nothing property gradeServlet A.htmlB.html P1 P4 P6 P5 P2 P3 C.html get () get (param1, param2, param3) get (param1, param2, param3) get () component Atomic section form link transition simple link transition Component expressions –Basic (p) –Sequence (p p1 · p2) –Selection (p p1 | p2) –Iteration (p p1*) –Aggregation (p p1 { p2})

5. Mutation 2010© Praphamontripong and Offutt5 Control Connection Faults TransitionsPotential Faults Simple link transition … Incorrect URL specified in the href attribute of an tag Form link transition Incorrect URL specified in the action attribute of a tag Inappropriate transfer mode Omitted necessary information or inappropriate information submitted via hidden controls Parameter mismatches

6. Mutation 2010© Praphamontripong and Offutt6 Control Connections Faults (2) TransitionsPotential Faults Component expression transition Incorrect / non-existing URL specified in an include directive Operational transition back, forward, … Unintended transitions Intentionally bypass the validation Redirect transition redirect, jsp:forward Incorrect / inappropriate URL specified in Incorrect / inappropriate URL specified in res.SendRedirect

7. Web Mutation Operators HTML : 6 Mutation Operators 1.Simple link replacement (WLR) 2.Simple link deletion (WLD) 3.Form link replacement (WFR) 4.Transfer mode replacement (WTR) 5.Hidden form field replacement (WHR) 6.Hidden form field deletion (WHD) JSP : 5 Mutation Operators 1.Server-side-include replacement (WIR) 2.Server-side-include deletion (WID) 3.Redirect transition replacement (WRR) 4.Redirect transition deletion (WRD) 5.Get session replacement (WGR) Mutation 2010© Praphamontripong and Offutt7

8. Web Mut OperatorsHTML Mutation 2010© Praphamontripong and Offutt8 1. WLR – Simple Link Replacement … … … 2. WLD – Simple Link Deletion … …

9. Web Mut OperatorsHTML Mutation 2010© Praphamontripong and Offutt9 3. WFR – Form Link Replacement … … … 4. WTR – Transfer Mode Replacement … <form action = URL 1 method = get> <form action = URL 1 method = post> …

10. Web Mut OperatorsHTML Mutation 2010© Praphamontripong and Offutt10 5. WHR – Hidden Form Field Replacement … <input type = hidden name = n1 value = v1> <input type = hidden name = n1 value = null> … 6. WHD – Hidden Form Field Deletion … <input type = hidden name = n1 value = v1> <!-- input type = hidden name = n1 value = v1 --> …

11. Web Mut OperatorsJSP Mutation 2010© Praphamontripong and Offutt11 1. WIR – Server-Side- Include Replacement … … … 2. WID – Server-Side-Include Deletion … <!-- %@ include file = URL 1 % --> …

12. Web Mut OperatorsJSP Mutation 2010© Praphamontripong and Offutt12 3. WRR – Redirect Transition Replacement … … … 4. WRD – Redirect Transition Deletion … <!-- jsp:forward page = URL 1 / --> …

13. Web Mut OperatorsJSP Mutation 2010© Praphamontripong and Offutt13 5. WGR – Get Session Replacement … session = get.session (true); session = get.session (false); …

14. Mutation 2010© Praphamontripong and Offutt14 webMujava : Generation

15. Mutation 2010© Praphamontripong and Offutt15 webMuJava : Mutants

16. Mutation 2010© Praphamontripong and Offutt16 Case Study : STIS JSP fileJava lines HTML lines Java/HTML ratio Comment lines Blank lines Total about0970.00819124 browse62830.755241238 categories34490.693721141 category_edit14370.38221386 index0310.0013751 login19320.59222396 logout10210.4813953 navigation_bar3250.1213950 page_footer240.506315 page_header971.299833 record_add4450.09221586 record_delete350.608420 record_edit36550.653025146 record_insert12460.26231596 record_search7410.17141173 update_search933.006321 Total2245810.392982261329 Helps users store, access and category information

17. Mutation 2010© Praphamontripong and Offutt17 Mutants and Tests JSP fileTotalTestsLiveKilledScore about1670 1.00 browse551314410.75 categories39116330.85 category_edit1360 1.00 index84081.00 login1700 1.00 logout73250.71 navigation_bar1050 1.00 page_footer42041.00 page_header32120.67 record_add84081.00 record_delete42041.00 record_edit2166150.71 record_insert94091.00 record_search22021.00 update_search32031.00 Total21980291900.87

18. Analysis 60 mutants (38%) were WHR (Hidden Form Field Replacement) Equivalent mutants : Changes of values of non-keys of records to be updated to or deleted from the database This was appropriately checked on the server Mutants not killed Mutation 2010© Praphamontripong and Offutt18

19. Mutation 2010© Praphamontripong and Offutt19 Hand-Seeded Faults Detected JSP file# Faults# TestsFoundRatio about4741.00 browse2013160.80 categories2611210.81 category_edit176140.82 index4430.75 login197110.58 logout3320.67 navigation_bar2521.00 page_footer2221.00 page_header5251.00 record_add9491.00 record_delete0n/a record_edit216140.67 record_insert9491.00 record_search3231.00 update_search3231.00 Total147801180.80

20. Mutation 2010© Praphamontripong and Offutt20 Analysis of Missed Faults Changes of scope setting of jsp:useBean This is state management Changes between equals method and sign (==) if (request.getParameter (userid).equals() if (request.getParameter(userid) == ) Unit testing?

21. Summary Conclusions –Demonstrated feasibility : Mutation operators, a tool –Initial evaluation : Tests, found faults, no comparison Future Work –Additional Web mutation operators – state, scope –SQL mutation operators –Support servlets –Controlled experiments using large, more complex, and industrial web applications –Release webMuJava Mutation 2010© Praphamontripong and Offutt21

22. © Praphamontripong and Offutt22Contacts Upsorn Praphamontripong uprapham@gmu.edu Jeff Offutt offutt@gmu.eduhttp://cs.gmu.edu/~offutt/ Mutation 2010