Secure Software Development SW Penetration Testing Chapter 6 Rasool Jalili & M.S. Dousti Dept. of Computer Engineering Fall 2010

Software Security 2 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Software Penetration Testing

Software Security 3 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 QA & testing organizations are tasked with the broad objective of ensuring that a software application fulfills its functional business requirements. Security is not a feature! So, security testing (especially penetration testing) does not fit directly into QA. Security testing poses a unique problem: A majority of security defects and vulnerabilities in software are not directly related to security functionality. Instead, security issues involve often unexpected but intentional misuses of an application discovered by an attacker. QA & testing organizations are tasked with the broad objective of ensuring that a software application fulfills its functional business requirements. Security is not a feature! So, security testing (especially penetration testing) does not fit directly into QA. Security testing poses a unique problem: A majority of security defects and vulnerabilities in software are not directly related to security functionality. Instead, security issues involve often unexpected but intentional misuses of an application discovered by an attacker. Software Penetration Testing

Software Security 4 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 If we characterize functional testing as "testing for positives“ then penetration testing is in some sense "testing for negatives." At its heart, security testing needs to make use of both white hat and black hat concepts and approaches: –ensuring that the security features work as advertised (a white hat undertaking) and –intentional attacks can't easily compromise the system (a black hat undertaking). Thinking like a bad guy is so essential to good penetration testing. If we characterize functional testing as "testing for positives“ then penetration testing is in some sense "testing for negatives." At its heart, security testing needs to make use of both white hat and black hat concepts and approaches: –ensuring that the security features work as advertised (a white hat undertaking) and –intentional attacks can't easily compromise the system (a black hat undertaking). Thinking like a bad guy is so essential to good penetration testing.

Software Security 5 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 In any case, testing for a negative poses a much greater challenge than verifying a positive. It is very difficult to show whether or not a system is secure enough under malicious attack. Q: How many tests do you do before you give up and declare "secure enough"? If negative tests do not uncover any faults, this only offers proof that no faults occur under particular test; this by no means proves that no faults exist. Finding a security problem and fixing it is fine. But what about the rest of the system? In any case, testing for a negative poses a much greater challenge than verifying a positive. It is very difficult to show whether or not a system is secure enough under malicious attack. Q: How many tests do you do before you give up and declare "secure enough"? If negative tests do not uncover any faults, this only offers proof that no faults occur under particular test; this by no means proves that no faults exist. Finding a security problem and fixing it is fine. But what about the rest of the system?

Software Security 6 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Penetration testing is the most frequently and commonly applied of all SWS best practices. This is not necessarily a good thing. Often penetration testing is positioned on software development teams by security guys and everyone ends up angry! Too much driven by an outside  in approach. One major limitation of this approach is that it almost always represents a too-little-too-late attempt to tackle security at the end of the development cycle. Fixing things at this stage is, prohibitively expensive and almost always involves Band-Aids instead of cures. Post-penetration-test security fixes tend to be particularly reactive and defensive in nature. Software security penetration tests do not currently follow a standard process of any sort. Penetration testing is the most frequently and commonly applied of all SWS best practices. This is not necessarily a good thing. Often penetration testing is positioned on software development teams by security guys and everyone ends up angry! Too much driven by an outside  in approach. One major limitation of this approach is that it almost always represents a too-little-too-late attempt to tackle security at the end of the development cycle. Fixing things at this stage is, prohibitively expensive and almost always involves Band-Aids instead of cures. Post-penetration-test security fixes tend to be particularly reactive and defensive in nature. Software security penetration tests do not currently follow a standard process of any sort. Penetration Testing Today

Software Security 7 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Use of security requirements, abuse cases, security risk knowledge, and attack patterns in application design, analysis, and testing is rare in current practice. As a result, security findings are not repeatable across different teams and vary widely depending on the skill and experience of the tester(s). In practice, a pen-test can identify only a small representative sample of all of the possible security risks in a system. Pen-testing without any basis in security risk analysis leads to the "pretend security“. One big benefit of pen-testing is its compliance to a critical black hat view in its real production environment. Use of security requirements, abuse cases, security risk knowledge, and attack patterns in application design, analysis, and testing is rare in current practice. As a result, security findings are not repeatable across different teams and vary widely depending on the skill and experience of the tester(s). In practice, a pen-test can identify only a small representative sample of all of the possible security risks in a system. Pen-testing without any basis in security risk analysis leads to the "pretend security“. One big benefit of pen-testing is its compliance to a critical black hat view in its real production environment. ………………

Software Security 8 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Pen-testing can be used effectively. The best approach bases on security findings discovered from the beginning of the software lifecycle: –during requirements analysis, architectural risk analysis, and so on. Pen-testing is about testing a system in its final production environment. So, pen-testing is best suited to probing configuration problems and other environmental factors that deeply impact software security. Driving tests focusing on these factors with some knowledge of risk analysis results is the most effective approach. Make Use of Tools Tools (including the static analysis tools) should definitely be used in penetration testing. These tools can submit malformed, malicious, and random data to a system's entry points in an attempt to uncover faults; A tool-driven approach can't be used as a replacement for review by a skilled security analyst; but a tool-based approach does help reducing the cost. Pen-testing can be used effectively. The best approach bases on security findings discovered from the beginning of the software lifecycle: –during requirements analysis, architectural risk analysis, and so on. Pen-testing is about testing a system in its final production environment. So, pen-testing is best suited to probing configuration problems and other environmental factors that deeply impact software security. Driving tests focusing on these factors with some knowledge of risk analysis results is the most effective approach. Make Use of Tools Tools (including the static analysis tools) should definitely be used in penetration testing. These tools can submit malformed, malicious, and random data to a system's entry points in an attempt to uncover faults; A tool-driven approach can't be used as a replacement for review by a skilled security analyst; but a tool-based approach does help reducing the cost. Software Penetration Testing- a Better Approach

Software Security 9 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Fault Injection Tools One of the most interesting modern fault injection engines for security is the Cenzic tool. Any hacker (malicious or otherwise) uses these tools. Disassemblers and decompilers: Control flow and coverage tools Breakpoint setters and monitors Buffer overflow. Shell code. Rootkits. –Other tools include the following: Debuggers (user-mode) Kernel debuggers Fault Injection Tools One of the most interesting modern fault injection engines for security is the Cenzic tool. Any hacker (malicious or otherwise) uses these tools. Disassemblers and decompilers: Control flow and coverage tools Breakpoint setters and monitors Buffer overflow. Shell code. Rootkits. –Other tools include the following: Debuggers (user-mode) Kernel debuggers Tools for Penetration Testing

Software Security 10 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 CENZIC

Software Security 11 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Human review is necessary to reveal flaws in the design or more complicated implementation-level vulnerabilities (of the sort that attackers can and will exploit). However, review by an expert is costly, can be ineffective if the "expert" is not.  More structured and cost-effective solutions are needed. Penetration testing can benefit greatly from knowledge of the security risks built into a system. No design or implementation is perfect, and carrying risk is usually acceptable. Penetration testing can help finding what this means to your fielded system. Penetration testing should focus at the system level and should be directed at properties of the integrated SW system. The most common failure of the SW pen-testing is failure to identify lessons learned and propagate them back into the organization. Iterative pen-tests are coming to reveal fewer and less severe defects in the system. Don't forget that the real value of pen-testing comes from its central role in inspecting configuration and other essential environmental factors. Use pen-testing as a "last check" before code goes live instead of as a "first check" of security posture. Human review is necessary to reveal flaws in the design or more complicated implementation-level vulnerabilities (of the sort that attackers can and will exploit). However, review by an expert is costly, can be ineffective if the "expert" is not.  More structured and cost-effective solutions are needed. Penetration testing can benefit greatly from knowledge of the security risks built into a system. No design or implementation is perfect, and carrying risk is usually acceptable. Penetration testing can help finding what this means to your fielded system. Penetration testing should focus at the system level and should be directed at properties of the integrated SW system. The most common failure of the SW pen-testing is failure to identify lessons learned and propagate them back into the organization. Iterative pen-tests are coming to reveal fewer and less severe defects in the system. Don't forget that the real value of pen-testing comes from its central role in inspecting configuration and other essential environmental factors. Use pen-testing as a "last check" before code goes live instead of as a "first check" of security posture. Test More Than Once

Software Security 12 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 End of Chapter 5. End