Leveraging User Interactions for In-Depth Testing of Web Applications Sean McAllister, Engin Kirda, and Christopher Kruegel RAID ’08 1 Seoyeon Kang November 4, 2008

Introduction 2 Static  Dynamic Complexity ↑ Bugs ↑ Vulnerability ↑

Introduction 3 Web application vulnerabilities – Result from lack of input validation – Example: Cross site scripting(XSS), SQL injection bugs

Review (1) 4 Cross site scripting(XSS) – Malicious scripting code which transfers sensitive information to the attacker is injected into the output of an application Stored XSS: Attacker persistently stores the malicious code in a resource such as a database. Reflected XSS : Script is reflected to the victim.

Review (2) 5 SQL injection – Malicious input can reach a DB query and modify the intended semantics of this query

Background (1) 6 Testing tool – White box testing : the source code of the application is analyzed to indicate errors or vulnerabilities. – Black box testing : Special input test cases are generated and sent to the application. Then, the results are analyzed.

Background (2) 7 Testing tool White boxBlack box Pros. Large code coverage In theory, can analyze all possible execution paths No false positives No need knowledge about implementation-specific details Cons. False positives Support limited programming language Limited code coverage

Background (3) 8 Vulnerability scanner – An automated security program that searches for software vulnerabilities within web applications

Background (4) 9 Steps of vulnerability scanner – Web spider finds valid injection points – Provide a variety of specially-crafted, malformed input values that trigger certain classes of vulnerabilities – Analyze response that web application returns

Limitations 10 Fail case : To leave a comment in blogging site

Limitations 11 Code coverage – Scanner does not have any knowledge about use cases – Built-in web spider is insufficient

Goal 12 Improve the effectiveness of web vulnerability scanners Increase test coverage

Approach 13 Leverage input from real users as a starting point for testing – Generate test cases that can be replayed Interact with the web application in a more meaningful fashion – For detection of stored XSS vulnerability

Increasing Test Coverage 1)Increasing testing depth 2)Increasing testing breadth 3)Stateful Fuzzing 14

1) Increasing testing depth Leverage actual user input – Collecting input Record requests directly at the server – Replaying input Execute one step Fuzzer replace the valid inputs to malformed values in database Previously recorded input values are used for the next step. Guided fuzzing : process of using previously collected traces 15

2) Increasing testing breadth Use current step as a starting point for fuzzing – For each step, only a single entry point is analyzed – Fuzzer attempts to find additional entry points Extended, guided fuzzing 16

3) Stateful Fuzzing 17 Fail scenario : On the page of shopping cart – Delete all items in cart – The following steps of the use case fails Show cart’s inventory Checkout Delete all items

3) Stateful Fuzzing 18 Stateful fuzzing – Take a snapshot of the state of the application after each step that is replayed – Restored to the snapshot Application feedback – Use the mapping of URL to functions to reuse user input for undetected pages

Implementation details (1) Require to be instrumented – To capture and restore objects – To extract the mappings between URLs and functions Need MVC-based web development framework – Used Django based on python 19

Implementation details (2) Capturing web requests – Log all incoming requests and mappings b/w URL and functions through the URL dispatcher class Replaying use cases – Analyze a page and find the form elements Capturing object manipulations – Attach event listeners that wait for signals that are raised when an object is created, updated, or deleted Fuzzer component – Use existing fuzzer component 20

Evaluation 3 publicly available, real-world web applications based on Django – Blogging application – Forum software – Web shop simple web spider and 3 existing web vulnerability scanners – Burp Spider – W3af spider – Acunetix Web Vulnerability Scanner 21

Test methodology Way to assess the effectiveness or coverage – Number of vulnerabilities – Number of locations that a scanner visits  Number of vulnerable injection points  Number of created different database objects (for detecting stored XSS) Only tested for XSS vulnerabilities 22

Experimental results 23 1) Blogging application 2) Forum software 3) Web shop

1) Blogging application 24

1) Blogging application Scanner effectiveness for blog application 25

2) Forum application 26 Scanner effectiveness for forum application

3) Online shopping application 27 Scanner effectiveness for online shopping application

3) Online shopping application Object creation statistics (Acunetix vs. this ) 28

Experimental results 29 Proposed fuzzing techniques consistently find more of bugs than other scanners Carry out meaningful interactions with the web applications, visiting many locations and creating a large variety of database objects Stateful fuzzing becomes useful when tested app is more complex and sensitive to the fuzzing steps

Related work Black box scanner – SecuBat - coverage limitation General vulnerability detection tool – Nikto, Nessus - use known velnerability DB Automated test case generator – Random test case generation - depth limitation 30

Summary The core of system is a black-box vulnerability scanner Proposed approaches to allow the scanner to reach deeper into application – Guided fuzzing – Extended, guided fuzzing – Stateful fuzzing This approach can identify more vulnerabilities than several open-source and commercial web vulnerability scanners 31

Thank you 32