1. The OWASP Testing Framework
(Based on the “new OWASP Testing Guide” presentation by Matteo Meucci and Alberto Revelli)
Austin OWASP - 8/28/2007

2. Introduction Who is Josh Sokol?
On the Web Systems Team at National Instruments
UNIX/Linux System Administrator ~10 years
Cisco Certified Network Associate
SANS GIAC in Web Application Security (GWAS)
Working on an initiative to bring a more security oriented mindset to the developers at NI.
Austin OWASP - 8/28/2007

3. Agenda: The OWASP Testing Framework
The Testing Methodology: How to Test
Reporting: How to Evaluate the Risk and Write a Report
Time Pending: Q&A
Time Pending: Tools and Resources
Austin OWASP - 8/28/2007

4. The OWASP Testing Framework
The problem of insecure software: companies next challenge
Why OWASP?
“It's impossible to underestimate the importance of having this guide available in a completely free and open way”– Jeff Williams (OWASP Chair)
Principles of Testing: comparing the state of something against a set of criteria defined and complete.
We want security testing not be a black art
Testing Techniques:
Manual Inspections & Reviews
Threat Modeling
Code Review
Penetration Testing
Austin OWASP - 8/28/2007

5. The OWASP Testing Framework
Phase 1: Before Development Begins
Before application development has started:
Test to ensure that there is an adequate SDLC where security is inherent.
Test to ensure that the appropriate policy and standards are in place for the development team.
Develop Measurement and Metrics Criteria (Ensure Traceability)
Austin OWASP - 8/28/2007

6. The OWASP Testing Framework
Phase 2: During Definition and Design
Before application development has started:
Security Requirements Review:
User Management (password reset etc.), Authentication, Authorization, Data Confidentiality, Integrity, Accountability, Session Management,Transport Security, Privacy
Design an Architecture Review
Create and Review UML Models
How the application works
Create and Review Threat Models
Develop realistic threat scenarios
Austin OWASP - 8/28/2007

7. The OWASP Testing Framework
Phase 3: During Development
Code Walkthroughs:
high-level walkthrough of the code where the developers can explain the logic and flow.
Code Reviews:
Static code reviews validate the code against a set of checklists:
CIA Triad
OWASP Top10, OWASP Code Review
Sox, ISO 17799, etc…
Austin OWASP - 8/28/2007

8. The OWASP Testing Framework
Phase 4: During Deployment
Application Penetration Testing
Focus of the OWASP Testing Framework Guide
Configuration Management Testing
The application penetration test should include the checking of how the infrastructure was deployed and secured.
Phase 5: Maintenance and Operations
Conduct operational management reviews
Conduct periodic health checks
Ensure change verification
Austin OWASP - 8/28/2007

9. Web Application Penetration Testing
What is Web Application Penetration Testing?
The process involves an active analysis of the application for any weaknesses, technical flaws or vulnerabilities
Will never be an exact science where a complete list of all possible issues that should be tested can be defined.
What is a vulnerability?
A weakness on a asset that makes a threat possible
OWASP’s approach in writing the guide
Open
Collaborative
Based on a “Black Box” approach
Defined testing methodology
Consistent
Repeatable
Under quality
Austin OWASP - 8/28/2007

10. Testing Paragraph Template
Brief Summary
Describe in "natural language" what we want to test. The target of this section is non- technical people (e.g.: client executive)
Description of the Issue
Short Description of the Issue: Topic and Explanation
Black Box testing and example
How to test for vulnerabilities:
Result Expected:
...
Gray Box testing and example
References
Whitepapers
Tools
Austin OWASP - 8/28/2007

11. Black Box vs. Gray Box Black Box Gray Box
The penetration tester does not have any information about the structure of the application, its components and internals
Black Box
The penetration tester has partial information about the application internals. E.g.: platform vendor, sessionID generation algorithm
Gray Box
White box testing, defined as complete knowledge of the application internals, is beyond the scope of the Testing Guide and is covered by the OWASP Code Review Project
Austin OWASP - 8/28/2007

12. Penetration Testing Methodology
Austin OWASP - 8/28/2007

13. Testing Model The test is divided into 2 phases:
Passive mode: in the passive mode the tester tries to understand the application's logic, plays with the application; a tool can be used for information gathering such as an HTTP proxy to observe all the HTTP requests and responses. At the end of this phase the tester should understand all the access points (gates) of the application (e.g. Header HTTP, parameters, cookies). A spreadsheet with the directory tree of the application and all the access points is created for use with the second phase.
Active mode: in this phase the tester begins to test using eight distinct sub-phases of security assessment.
Austin OWASP - 8/28/2007

14. Passive Mode: Example Use an HTTP proxy to observe all the HTTP
requests and responses.
WebScarab (OWASP)
TamperData (Firefox Extension)
Austin OWASP - 8/28/2007

15. Active Mode
OWASP split the set of tests in 8 sub-categories (for a total amount of 48 controls):
Information Gathering
Business logic testing
Authentication Testing
Session Management Testing
Data Validation Testing
Denial of Service Testing
Web Services Testing
AJAX Testing
Austin OWASP - 8/28/2007

16. Information Gathering
The first phase in security assessment is of course focused on collecting all the information about a target application.
Using public tools it is possible to force the application to leak information by sending messages that reveal the versions and technologies used by the application
Available techniques include:
Raw HTTP Connections (netcat)
The good ol' tools: nmap, amap, ...
Web Spiders
Search engines (“Google Dorking”)
SSL fingerprinting
File extensions handling
Backups and unreferenced files
Austin OWASP - 8/28/2007

17. Information Gathering: Example
Application Fingerprint
Knowing the version and type of a running web server allows testers to determine known vulnerabilities and the appropriate exploits to use along the tests. Netcat is the tool of choice for this very well known technique
$ nc demo.testfire.net 80 HEAD / HTTP/1.0
HTTP/ OK Connection: close Date: Mon, 27 Aug :36:11 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET X-AspNet-Version: Set-Cookie: ASP.NET_SessionId=atu011455ailys3tuk2hasqh; path=/; HttpOnly Set-Cookie: amSessionId= ; path=/ Cache-Control: no-cache Pragma: no-cache Expires: -1 Content-Type: text/html; charset=utf-8 Content-Length: 9550
...But what if the “Server:” header is obfuscated ?
Austin OWASP - 8/28/2007

18. Information Gathering: Example
Other hints can be found by sending the server a malformed request, for instance a “GET / HTTP/3.0”
HTTP/ Bad Request
Date: Sun, 15 Jun :12: 37 GMT
Server: obfuscated :P
Connection: close
Transfer: chunked
Content-Type: text/HTML; charset=iso
Apache
HTTP/ HTTP Version Not Supported
Server: obfuscated :P
Date: Mon, 16 Jun :04: 04 GMT
Content-length: 140
Content-type: text/HTML
Connection: close
Netscape Enterprise 4.1
HTTP/ OK
Server: obfuscated :P
Content-Location:
Date: Fri, 01 Jan :14: 02 GMT
Content-Type: text/HTML
Accept-Ranges: bytes
Last-Modified: Fri, 01 Jan :14: 02 GMT
ETag: W/e0d362a4c335be1: ae1
Content-Length: 133
IIS 5.0
...But what if the application simply returns a generic error page ?
Austin OWASP - 8/28/2007

19. Information Gathering: Example
The good news is that each server has a favorite way to order headers !
Here are the results for some common web servers when responding to a “HEAD / HTTP/1.0” command:
Austin OWASP - 8/28/2007

20. Business Logic Testing
In this phase, we look for flaws in the application business logic rather than in the technical implementation. Areas of testing include:
Rules that express the business policy (such as channels, location, logistics, prices, and products)
Workflows that are the ordered tasks of passing documents or data from one participant (a person or a software system) to another
One of the most common results in this step of the analysis are flaws in the order of actions that a user has to follow: an attacker could perform them in a different order to get some sort of advantage
This step is the most difficult to perform with automated tools, as it requires the penetration tester to perfectly understand the business logic that is (or should be) implemented by the application
Austin OWASP - 8/28/2007

21. Business Logic Testing: Example
Summer 2005 Issue of 2600:
In this article he describes a flaw that became apparent to him within a newly released BlackJack game on the Paradise Poker website. In BlackJack, when the dealer is showing an ace, the dealer offers the players the option to purchase insurance. This is a way for the players to pay to cut their losses should the dealer have ten (10, Jack, Queen, or King) in the hole.
On this particular online game, he noticed that when the dealer did have a pocket ten, there would be a noticeable pause before he was prompted with the Insurance request. When there wasn’t a pocket ten, the prompt appeared immediately.
What do you think happened next?
What do you think
Austin OWASP - 8/28/2007

22. Business Logic Testing: Example
After doing some quick calculations, he realized this bit of information gave him an edge over the house. He ended up playing the next seven hours exploiting this bug and made a nice chunk of change during that time.
Obviously we don’t know what caused the flaw in the game, but a good guess is that there was some calculation the system needed to make to determine whether or not to offer insurance. That calculation may have taken more time to perform in the situation the dealer had a ten.
Let’s pretend that we’re right and think about that for a sec. The code itself may have been completely correct in the sense that it did what it was supposed to do. It was the amount of time the code needed to execute that ended up being the tell. No different than when a poker player twitches when holding a great hand.
The fix may have been to change the execution profile of the code so that it made the same pause no matter what was in the hole. Talk about a challenge for game developers. Not only does the code need to be bug free in syntax and semantics, but they now need to worry about the execution profile for their games.
Austin OWASP - 8/28/2007

23. Authentication Testing
Testing the authentication scheme means understanding how the application checks for users' identities and using that information to circumvent that mechanism and access the application without having the proper credentials
Tests include the following areas:
Default or Guessable Accounts
Brute-force
Bypassing Authentication
Directory Traversal / File Include
Vulnerable “Remember Password” and Password Reset
Logout and Browser Cache Management
Austin OWASP - 8/28/2007

24. Authentication Testing: Example
Using Brutus to brute force HTTP Basic Authentication on a demonstration website.
Austin OWASP - 8/28/2007

25. Session Management Testing
Session management is a critical part of a security test, as every application has to deal with the fact that HTTP is by it’s nature a stateless protocol. Session Management broadly covers all controls on a user from authentication to leaving the application
Tests include the following areas:
Analysis of the session management scheme
Cookie and session token manipulation
Exposed session variables
Cross Site Request Forgery (CSRF)
HTTP Exploiting
Austin OWASP - 8/28/2007

26. Data Validation Testing
In this phase we test that all input is properly sanitized before being processed by the application, in order to avoid several classes of attacks. This is the most common web application security weakness.
Cross site scripting (XSS)
Test that the application filters JavaScript code that might be executed by the victim in order to steal his/her cookier
HTTP Methods and XST
Test that the remote web server does not allow the TRACE HTTP method
SQL Injection
Test that the application properly filters SQL code embedded in the user input
Other attacks based of faulty input validation...
LDAP/XML/SMTP/OS injection
Buffer overflows
Austin OWASP - 8/28/2007

27. Data Validation Testing: Example
XSS Exploit on
Austin OWASP - 8/28/2007

28. Denial of Service Testing
DoS are types of vulnerabilities within applications that can allow a malicious user to make certain functionality or sometimes the entire website unavailable. These problems are caused by bugs in the application, often resulting from malicious or unexpected user input
Locking Customer Accounts
User Specified Object Allocation
User Input as a Loop Counter
Writing User Provided Data to Disk
Failure to Release Resources
Storing too Much Data in Session
Usually not performed in performed on production environments
Austin OWASP - 8/28/2007

29. Denial of Service: Example
At a former employer there was an application that queried a database for a few thousand
rows of data when a user clicked a “submit” button. This request could take several minutes
to process and dramatically increased CPU utilization. The application did not stop the user
from clicking “submit” again while it was processing and each time the user did that it
spawned another process which further drove up CPU utilization. An impatient user could
easily cause the server to lock up and cause a denial of service.
Austin OWASP - 8/28/2007

30. Web Services Testing
Webservice “clients” are generally not user web front-ends but
other backend servers. The vulnerabilities in web services are
similar to other vulnerabilities such as SQL injection, information
disclosure and leakage etc but web services also have unique
XML/parser related vulnerabilities.
XML Structural Testing
XML Content-Level Testing
HTTP GET parameters/REST Testing
Naughty SOAP attachments
Replay Testing
Austin OWASP - 8/28/2007

31. Web Services Testing
The vulnerabilities are similar to other “classical” vulnerabilities such as SQL injection, information disclosure and leakage etc but web services also have unique XML/parser related vulnerabilities.
WebScarab (available for free at provides a plug-in specifically targeted to Web Services. It can be used to craft SOAP messages that contains malicious elements in order to test how the remote system validates input
Austin OWASP - 8/28/2007

32. Web Services Testing: Example 1
XML Structural Testing
In this example, we see a snippet of XML code that violates the hierarchical structure of this language. A Web Service must be able to handle this kind of exceptions in a secure way
<?xml version="1.0" encoding="ISO "?>
<note id="666">
<to>OWASP
<from>EOIN</from>
<heading>I am Malformed </to>
</heading>
<body>Example of XML Structural Test</body>
</note>
Austin OWASP - 8/28/2007

33. Web Services Testing: Example 2
XML Large payload
Another possible attack consists in sending to a Web Service a very large payload in an XML message. Such a message might deplete the resource of a DOM parser
<Envelope>
<Header>
<wsse:Security>
<Hehehe>I am a Large String (1MB)</Hehehe>
<Hehehe>I am a Large String (1MB)</Hehehe>…
<Signature>…</Signature>
</wsse:Security>
</Header>
<Body>
<BuyCopy><ISBN> </ISBN></BuyCopy>
</Body></Envelope>
Austin OWASP - 8/28/2007

34. Web Services Testing : Example 3
Naughty SOAP attachments
Binary files, including executables and document types that can contain malware, can be posted using a web service in several ways
POST /Service/Service.asmx HTTP/1.1
Host: somehost
Content-Type: text/xml; charset=utf-8
Content-Length: length
SOAPAction:
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="
xmlns:xsd="
xmlns:soap="
<soap:Body>
<UploadFile xmlns="
<filename>eicar.pdf</filename>
<type>pdf</type>
<first>true</first>
</UploadFile>
</soap:Body>
</soap:Envelope>
Austin OWASP - 8/28/2007

35. AJAX Testing
AJAX (Asynchronous JavaScript and XML) is a web development technique used to create more interactive web applications.
XMLHttpRequest object and JavaScript to make asynchronous requests for all communication with the server-side application.
Main security issues:
AJAX applications have a greater attack surface because a big share of the application logic is moved on the client side
AJAX programmers seldom keep an eye on what is executed by the client and what is executed by the server
Exposed internal functions of the application
Client access to third-party resources with no built-in security and encoding mechanisms
Failure to protect authentication information and sessions
AJAX Bridging
Austin OWASP - 8/28/2007

36. AJAX Testing
While in traditional web applications it is very easy to enumerate the points of interaction between clients and servers, when testing AJAX pages things get a little bit more complicated, as server-side AJAX endpoints are not as easy or consistent to discover
To enumerate endpoints, two approaches must be combined:
Look through HTML and Javascript (e.g: look for XmlHttpRequest objects)
Use a proxy to monitor traffic
Tools: OWASP Sprajax or Firebug add-on for Firefox
Then you can test it as described before (SQL Inj, etc..)
...and don't forget AJAX potential in prototype hijacking and resident XSS !
Austin OWASP - 8/28/2007

37. With firebug it is possible to efficiently inspect AJAX apps
AJAX Testing
With firebug it is possible to efficiently inspect AJAX apps
Austin OWASP - 8/28/2007

38. AJAX Testing: Example
The Samy and Spaceflash worms both spread on MySpace,
changing profiles on the hugely popular social-networking Web site.
In Samy attack, the XSS Exploit allowed <SCRIPT> in
MySpace.com profile. AJAX was used to inject a virus into the
MySpace profile of any user viewing an infected page and forced
any user viewing the infected page to add the user “Samy” to his
friend list. It also appended the words “Samy is my hero” to the
victim’s profile.
Austin OWASP - 8/28/2007

39. Penetration Testing Reports
Austin OWASP - 8/28/2007

40. Testing Report: Model The OWASP Risk Rating Methodology
Estimate the severity of all of these risks to your business
This is not universal risk rating system: vulnerability that is critical to one organization may not be very important to another
Simple approach to be tailored for every case
standard risk model: Risk = Likelihood * Impact
Step 1: identifying a risk
You'll need to gather information about:
the vulnerability involved
the threat agent involved
the attack they're using
the impact of a successful exploit on your business.
Austin OWASP - 8/28/2007

41. Testing Report: Likelihood
Step 2: factors for estimating likelihood
Generally, identifying whether the likelihood is low, medium, or high is sufficient.
Threat Agent Factors:
Skill level (0-9)
Motive (0-9)
Opportunity (0-9)
Size (0-9)
Vulnerability Factors:
Ease of discovery (0-9)
Ease of exploit (0-9)
Awareness (0-9)
Intrusion detection (0-9)
Austin OWASP - 8/28/2007

42. Testing Report: Impact
Step 3: factors for estimating impact
Technical impact:
Loss of confidentiality (0-9)
Loss of integrity (0-9)
Loss of availability (0-9)
Loss of accountability (0-9)
Business impact:
Financial damage (0-9)
Reputation damage (0-9)
Non-compliance (0-9)
Privacy violation (0-9)
Austin OWASP - 8/28/2007

43. Testing Report: Value the Risk
Step 4: determining the severity of the risk
In the example above, the likelihood is MEDIUM, and the technical impact is HIGH, so from a technical standpoint, the overall severity is HIGH. But business impact is actually LOW, so the overall severity is best described as LOW as well.
Austin OWASP - 8/28/2007

44. Testing Report: Decide What to Fix
Step 5: Deciding What To Fix
As a general rule, you should fix the most severe risks first.
Some fix seems to be not justifiable based upon the cost of fixing the issue but may be reputation damage from the fraud that could cost the organization much more than implement a security control
Step 6: Customizing Your Risk Rating Model
Adding factors
Customizing options
Weighting factors
Austin OWASP - 8/28/2007

45. Writing the Report I. Executive Summary
II. Technical Management Overview
III Assessment Findings
IV Toolbox
Austin OWASP - 8/28/2007

46. OWASP Penetration Testing Guide: Summary
A structured approach to the testing activities
A checklist to be followed
A learning and training tool
Pen-testers
A tool to understand web vulnerabilities and their impact
A way to check the quality of the penetration tests they buy
Clients
More in general, the Guide aims to provide a pen-testing standard that creates a 'common ground' between the pen-testing industry and it’s clients.
This will raise the overall quality and understanding of this kind of activity and therefore the general level of security in our infrastructures.
Austin OWASP - 8/28/2007

47. Questions?
Austin OWASP - 8/28/2007

48. A list of tools which are free and/or Open Source.
Tools and Resources
A list of tools which are free and/or Open Source.
Austin OWASP - 8/28/2007

49. OWASP Resources
OWASP AppSec FAQ Project FAQ covering many application security topics
OWASP Guide Project a massive document covering all aspects of web application and web service security
OWASP Legal Project a project focused on contracting for secure software
OWASP Testing Guide a project focused on application security testing procedures and checklists
OWASP Top Ten Project an awareness document that describes the top ten web application security vulnerabilities =
Austin OWASP - 8/28/2007

50. Black Box Testing Tools
OWASP WebScarab –
OWASP CAL9000 –
OWASP Pantera –
SPIKE –
Paros –
Burp Proxy –
Achilles Proxy –
Odysseus Proxy –
Webstretch Proxy –
Firefox LiveHTTPHeaders, Tamper Data and Developer Tools –
Sensepost Wikto (Google cached fault-finding) -
Austin OWASP - 8/28/2007

51. Testing Ajax
OWASP SPRAJAX –
Austin OWASP - 8/28/2007

52. Testing for SQL Injection
OWASP SQLiX
Multiple DBMS Sql Injection tool – [SQL Power Injector]
MySql Blind Injection Bruteforcing, Reversing.org – [sqlbftools]
Antonio Parata: Dump Files by sql inference on Mysql – [SqlDumper]
Sqlninja: a SQL Server Injection & Takeover Tool –
Bernardo Damele and Daniele Bellucci: sqlmap, a blind SQL injection tool –
Absinthe 1.1 (formerly SQLSqueal) –
SQLInjector –
Bsqlbf-1.2-th –
Austin OWASP - 8/28/2007

53. Testing Oracle
TNS Listener tool (Perl) -
Toad for Oracle -
Austin OWASP - 8/28/2007

54. Testing SSL
Foundstone SSL Digger -
Austin OWASP - 8/28/2007

55. Testing for Brute Force Password
SensePost Crowbar –
THC Hydra –
John the Ripper -
Brutus –
Medusa -
Austin OWASP - 8/28/2007

56. Testing for HTTP Methods
NetCat -
Austin OWASP - 8/28/2007

57. Testing Buffer Overflow
OllyDbg: "A windows based debugger used for analyzing buffer overflow vulnerabilities" -
Spike, A fuzzer framework that can be used to explore vulnerabilities and perform length testing -
Brute Force Binary Tester (BFB), A proactive binary checker -
Metasploit, A rapid exploit development and Testing frame work -
Austin OWASP - 8/28/2007

58. Fuzzer
OWASP WSFuzzer -
Austin OWASP - 8/28/2007

59. Googling
Foundstone Sitedigger (Google cached fault-finding) -
Austin OWASP - 8/28/2007

60. Source Code Analyzers
OWASP LAPSE –
PMD –
FlawFinder –
Microsoft’s FXCop –
Split –
Boon –
Pscan –
FindBugs -
Austin OWASP - 8/28/2007

61. Acceptance Testing
WATIR – - A Ruby based web testing framework that provides an interface into Internet Explorer. Windows only.
HtmlUnit – - A Java and JUnit based framework that uses the Apache HttpClient as the transport. Very robust and configurable and is used as the engine for a number of other testing tools.
jWebUnit – - A Java based meta-framework that uses htmlunit or selenium as the testing engine.
Canoo Webtest – - An XML based testing tool that provides a facade on top of htmlunit. No coding is necessary as the tests are completely specified in XML. There is the option of scripting some elements in Groovy if XML does not suffice. Very actively maintained.
HttpUnit – - One of the first web testing frameworks, suffers from using the native JDK provided HTTP transport, which can be a bit limiting for security testing.
Watij – - A Java implementation of WATIR. Windows only because it uses IE for it's tests (Mozilla integration is in the works).
Solex – - An Eclipse plugin that provides a graphical tool to record HTTP sessions and make assertions based on the results.
Selenium JavaScript based testing framework, cross-platform and provides a GUI for creating tests. Mature and popular tool, but the use of JavaScript could hamper certain security tests.
Austin OWASP - 8/28/2007

62. Site Mirroring wget – http://www.gnu.org/software/wget
curl –
Sam Spade –
Xenu -
Austin OWASP - 8/28/2007