1. Secure Software Development Risk-Based Security Testing Chapter 7 Rasool Jalili & A. Boorghani Dept. of Computer Engineering Spring 2012

2. Software Security 2 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Risk-Based Security Testing

3. Software Security 3 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Security testing has recently moved beyond the realm of network port scanning to include probing software behavior as a critical aspect of system behavior. Security testing done properly goes much deeper than simple black box probing on the presentation layer and even beyond the functional testing of security mechanism. Testers must carry out a risk-based approach, grounded in both the system's architectural reality and the attacker's mindset. By identifying risks in the system and creating tests driven by those risks, a software security tester can properly focus on areas of code where an attack is likely to succeed. Security testing has recently moved beyond the realm of network port scanning to include probing software behavior as a critical aspect of system behavior. Security testing done properly goes much deeper than simple black box probing on the presentation layer and even beyond the functional testing of security mechanism. Testers must carry out a risk-based approach, grounded in both the system's architectural reality and the attacker's mindset. By identifying risks in the system and creating tests driven by those risks, a software security tester can properly focus on areas of code where an attack is likely to succeed. Security testing

4. Software Security 4 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 The main difference of security testing and pentest: –the level of approach and –the timing of the testing itself. –Pen-testing happens once software is complete and installed in its operational environment. Also, penetration testing is focused outside  in and is somewhat brief. –Security testing can be applied before the software is complete, at the unit level, in a testing environment with stubs and pre-integration. –Should start at the feature or component/unit level, prior to system integration. The main difference of security testing and pentest: –the level of approach and –the timing of the testing itself. –Pen-testing happens once software is complete and installed in its operational environment. Also, penetration testing is focused outside  in and is somewhat brief. –Security testing can be applied before the software is complete, at the unit level, in a testing environment with stubs and pre-integration. –Should start at the feature or component/unit level, prior to system integration.

5. Software Security 5 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Security unit testing carries the benefit of breaking system security down into a number of discrete parts. Theoretically, if each component is implemented safely and fulfills inter-component design criteria, the greater system should be in reasonable shape. Security testing should continue at the system level and should be directed at properties of the integrated software system. This is precisely where penetration testing meets security testing. Finally, abuse cases developed earlier in the lifecycle should be used to enhance a test plan with adversarial tests based on possible abuse scenarios. Security testing involves as much black hat thinking as white hat thinking. Security unit testing carries the benefit of breaking system security down into a number of discrete parts. Theoretically, if each component is implemented safely and fulfills inter-component design criteria, the greater system should be in reasonable shape. Security testing should continue at the system level and should be directed at properties of the integrated software system. This is precisely where penetration testing meets security testing. Finally, abuse cases developed earlier in the lifecycle should be used to enhance a test plan with adversarial tests based on possible abuse scenarios. Security testing involves as much black hat thinking as white hat thinking.

6. Software Security 6 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Traditional approaches to computer and network security testing focus on network infrastructure, firewalls, and port scanning. This is especially true of network and software penetration testing (application penetration testing). Better penetration testing approaches take architectural risks, code scanning results, and security requirements into account, but still focus on an outside  in perspective. A classic outside  in paradigm focusing on firewall placement. Traditional approaches to computer and network security testing focus on network infrastructure, firewalls, and port scanning. This is especially true of network and software penetration testing (application penetration testing). Better penetration testing approaches take architectural risks, code scanning results, and security requirements into account, but still focus on an outside  in perspective. A classic outside  in paradigm focusing on firewall placement. From Outside  In to Inside  Out

7. Software Security 7 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Problem: the perimeter is only apparent at the network/packet level. At the level of software applications, the perimeter has all but disappeared. That's because firewalls have been configured (or mis-configured, depending on your perspective) to allow advanced applications to tunnel right through them. A good example of this phenomenon is the SOAP protocol, which is designed (on purpose) to carry traffic through port 80 for various different applications. In some sense, SOAP is an anti-security device invented by software people. In the brave new world of Service Oriented Architecture (SOA) for applications, we should not be surprised that the firewall is quickly becoming irrelevant. By contrast, in an inside  out approach to security, whereby software is itself subjected to rigorous risk management and security testing. Problem: the perimeter is only apparent at the network/packet level. At the level of software applications, the perimeter has all but disappeared. That's because firewalls have been configured (or mis-configured, depending on your perspective) to allow advanced applications to tunnel right through them. A good example of this phenomenon is the SOAP protocol, which is designed (on purpose) to carry traffic through port 80 for various different applications. In some sense, SOAP is an anti-security device invented by software people. In the brave new world of Service Oriented Architecture (SOA) for applications, we should not be surprised that the firewall is quickly becoming irrelevant. By contrast, in an inside  out approach to security, whereby software is itself subjected to rigorous risk management and security testing.

8. Software Security 8 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Software security is about making software behave in the presence of a malicious attack. Standard software testing literature is only concerned with what happens when software fails, regardless of intent. Software security - Software safety =an intelligent adversary. Most safety-critical systems (and high-assurance systems) posit a white hat world. Fact is, we live in a world with plenty of black hats as well, and we need to address that (head on). Attackers generally don't care whether a vulnerability is due to a flaw or a bug, although bugs tend to be easier to exploit. Design-level vulnerabilities are the hardest defect category to handle. Software security is about making software behave in the presence of a malicious attack. Standard software testing literature is only concerned with what happens when software fails, regardless of intent. Software security - Software safety =an intelligent adversary. Most safety-critical systems (and high-assurance systems) posit a white hat world. Fact is, we live in a world with plenty of black hats as well, and we need to address that (head on). Attackers generally don't care whether a vulnerability is due to a flaw or a bug, although bugs tend to be easier to exploit. Design-level vulnerabilities are the hardest defect category to handle. What's So Different about Security?

9. Software Security 9 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Different tasks to manage software security risks, such as: –Creating security abuse/misuse cases –Listing normative security requirements (and security features and functions) –Performing architectural risk analysis –Building risk-based security test plans –Using static analysis tools –Performing security tests –Performing penetration testing in the final environment –Cleaning up security breaches Different tasks to manage software security risks, such as: –Creating security abuse/misuse cases –Listing normative security requirements (and security features and functions) –Performing architectural risk analysis –Building risk-based security test plans –Using static analysis tools –Performing security tests –Performing penetration testing in the final environment –Cleaning up security breaches Risk Management and Security Testing

10. Software Security 10 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Do not forget: Software security is not security software. So, SWS testing must necessarily involve two diverse approaches: –Functional security testing: testing security mechanisms –Adversarial security testing: performing risk-based security testing motivated by simulating the attacker's approach Together, these two distinct activities are a mix of white hat (security functionality) and black hat (security attack) philosophies. Wrong belief: liberal application and use of various security features  "adding SSL“ to securing an application. Software security practitioners regret the over-reliance on "magic crypto fairy dust“. Do not forget: Software security is not security software. So, SWS testing must necessarily involve two diverse approaches: –Functional security testing: testing security mechanisms –Adversarial security testing: performing risk-based security testing motivated by simulating the attacker's approach Together, these two distinct activities are a mix of white hat (security functionality) and black hat (security attack) philosophies. Wrong belief: liberal application and use of various security features  "adding SSL“ to securing an application. Software security practitioners regret the over-reliance on "magic crypto fairy dust“.

11. Software Security 11 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Security testing: determining who should do the testing and what activities they should undertake. Who? –Standard testing organizations using a traditional approach  functional security testing. –Traditional QA staff? They have more difficulty due to the lack of expertise. 1.Security tests are difficult to design, due to thinking like an attacker. 2.Security tests do not often cause direct security exploit. –Bottom line: Risk-based security testing relies more on expertise and security experience (not testing experience). Security testing: determining who should do the testing and what activities they should undertake. Who? –Standard testing organizations using a traditional approach  functional security testing. –Traditional QA staff? They have more difficulty due to the lack of expertise. 1.Security tests are difficult to design, due to thinking like an attacker. 2.Security tests do not often cause direct security exploit. –Bottom line: Risk-based security testing relies more on expertise and security experience (not testing experience). How to Approach Security Testing

12. Software Security 12 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 How: –Books; helps a lot … –White & Black box testing with/ without having access to source. In Black Box testing, malicious input is an effort to break the program; Any testing method can reveal possible software risks and potential exploits. However, One problem with security testing (black or white box) is the lack of it (Testing Method). There is no end-point for software security; even a reasonable security testing routine is just a start. Unfortunately, security is being sold as a product, and most defensive mechanisms on the market do little to address the heart of the problem, which is bad software. Any testing approach is deeply impacted by software process issues. How: –Books; helps a lot … –White & Black box testing with/ without having access to source. In Black Box testing, malicious input is an effort to break the program; Any testing method can reveal possible software risks and potential exploits. However, One problem with security testing (black or white box) is the lack of it (Testing Method). There is no end-point for software security; even a reasonable security testing routine is just a start. Unfortunately, security is being sold as a product, and most defensive mechanisms on the market do little to address the heart of the problem, which is bad software. Any testing approach is deeply impacted by software process issues.

13. Software Security 13 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 The biggest problems in software security AS software takes input. Trust the input is a critical question that all software builders must consider. From buffer overflow (which involves putting too much input in too small a place) to SQL injection attack and cross-site scripting (XSS) attacks, trusting input is the common root cause. Carefully handling input is dominant to software security. If your program consumes data from "out there," you need to think carefully about who can feed and the stuff your program eats. The biggest problems in software security AS software takes input. Trust the input is a critical question that all software builders must consider. From buffer overflow (which involves putting too much input in too small a place) to SQL injection attack and cross-site scripting (XSS) attacks, trusting input is the common root cause. Carefully handling input is dominant to software security. If your program consumes data from "out there," you need to think carefully about who can feed and the stuff your program eats. Thinking about (Malicious) Input

14. Software Security 14 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 STOMATH IS THE ROOT OF ALL DESEASES. Attacker toolkits focus plenty of attention on input, with a plenty of fault injection tools, re-players, and the like. By its very nature, penetration testing is focused on input as well If your program accepts input over the network, it needs to be very doubtful of what it is getting. Using a black-list approach (which tries to enumerate all possible bad input) will not work. Instead, software needs to defend its input space with a white-list approach. STOMATH IS THE ROOT OF ALL DESEASES. Attacker toolkits focus plenty of attention on input, with a plenty of fault injection tools, re-players, and the like. By its very nature, penetration testing is focused on input as well If your program accepts input over the network, it needs to be very doubtful of what it is getting. Using a black-list approach (which tries to enumerate all possible bad input) will not work. Instead, software needs to defend its input space with a white-list approach.

15. Software Security 15 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Testing around malicious input is a necessary but not sufficient condition. Security testing needs to focusing on data structures, components, APIs, program state, and so on. In addition to building tests around risks that remain in the system, testers should consider things like: –Sockets –Pipes –The Win32 Registry –Files –Remote procedure calls (RPCs) –Command-line arguments –…. Testing around malicious input is a necessary but not sufficient condition. Security testing needs to focusing on data structures, components, APIs, program state, and so on. In addition to building tests around risks that remain in the system, testers should consider things like: –Sockets –Pipes –The Win32 Registry –Files –Remote procedure calls (RPCs) –Command-line arguments –…. Getting Over Input

16. Software Security 16 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 Time is a critical issue; two major aspects of time to consider One problem is that most developers are unfamiliar with the effects of multithreading on their systems. That means they often overlook subtle time- based attacks. Unless a security tester thinks like a bad guy (black hat firmly on head), security testing will not be effective. Time is a critical issue; two major aspects of time to consider One problem is that most developers are unfamiliar with the effects of multithreading on their systems. That means they often overlook subtle time- based attacks. Unless a security tester thinks like a bad guy (black hat firmly on head), security testing will not be effective.

17. Software Security 17 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Spring 2012 پايان