1. Static Code Analysis and Governance Effectively Using Source Code Scanners

2. About Me Jonathan Carter – Principal Security Consultant @ Pure Hacking Governance Business Unit Application Security – Enterprise Security Architect and Designer – Security Researcher @ Fortify API’s, Frameworks, Threat Intelligence

3. Process Technology People Presentation Flow 1.What do scanners do? 2.How do they do it? 3.What do you need to worry about? 4.How do you address these concerns?

4. What do analyzers do? TranslationAnalysisReporting Source Code API Rules Security Intelligence Vulnerabilities 1 2 3 Process Technology People

5. Translation Mechanics Source Code 1 Translation builds a model of how data flows through various layers Allows full interoperability of languages Presentation Layer Business Layer Data Layer Model Process Technology People

6. Translation Example String URLparameter = Request[“URLElement”]; Object ‘URLParameter’ Declared of Type String; Temporary Object ‘t1’ Declared; ‘t1’ = Result of ‘Request’ object’s ‘GetElement’ Method Executed; ‘URLParameter’ = ‘t1’; 1. Engine Reads.NET Source Code and Encounters: 2. Engine Translates Statement into Intermediate Language: 3. Engine Adds New Content to Existing Translation of Code Process Technology People

7. Translation Pitfalls Potential False Negatives: – Language Versions Not Supported – Translation Incorrect Translation step is not easy – Does the Translator Support the Language? – Are there subtle differences between different versions of a particular language? – How will the user know when translation fails? Process Technology People

8. Translation Solutions Here’s What You Can Do: 1.Verify that scanner supports all languages involved in your scan 2.Ask vendors about roadmaps for languages 3.Ensure you know how to detect translation failures.

9. Scan Mechanics Analysis Intelligence ASP.NET Rules ADO.NET Rules T-SQL Rules Java Rules ModelVulnerabilities Process Technology People

10. Scan Example 1. Engine Translates.NET Source Code into Intermediate Language Model 2. Engine Recognizes That ‘Request’ Object is Dangerous Source 3. Engine Recognizes Dangerous Output and Declares XSS Presence Model Dangerous Source Rule XSS.NET XSS Rule Model

11. Scan Pitfalls Scan step is even trickier than translation – Do rules cover a particular library, API? – Are rules accurately describing the conditions for a vulnerability to exist? – Are the analyzers correctly applying a rule all the time? – Are the rules good at detecting the vulnerabilities you care about? – Are the rules being overly paranoid in describing risk?

12. Scan Pitfalls Potential False Positives: 1.Engine models data flow and control flow incorrectly Engine applies rules incorrectly 2.Rules identify data sources as untrustworthy and your organization disagrees 3.Rules don’t take into account dynamic nature of your code 4.Old Rules Process Technology People

13. Scan Pitfalls Potential False Negatives: 5.Code is simply missing and analyzer never applies rules to it 6.Rules Don’t Recognize New Methods, Classes Process Technology People

14. Scan Pitfall False Taint Promotion 1.Engine lacks enough computing resources to perform a full scan 2.To compensate, engine cuts corners during scan phase and makes broad generalizations about various data structures 3.Engine reports a large number of false positives

15. Scan Pitfall 1.Not Really Suited for Identifying Architectural Issues 2.Not Ideal for Finding Vulnerabilities in Dynamic Code Philosophical Limitations in Static Analysis

16. Scan Solutions Here’s What You Can Do: 1.Verify that the scanner uses the latest rules 2.Verify that rules adequately cover all of the libraries your code may use 3.Ensure that the engine provides detailed evidence of every vulnerability it reports. Process Technology People

17. Scan Solutions Here’s What You Can Do: 4.Contact product’s technical support when the evidence for a vulnerability is simply wrong 5.Ensure that the scanner’s rules identify any custom data sources and sinks 6.Examine Scan Logs to ensure scan failures are not occurring. Process Technology People

18. Scan Solutions Here’s What You Can Do: 7.Verify that the engine is including all of its rules when performing a scan 8.Exclude any data source rules for data sources your organization considers trustworthy 9.Gather feedback from developers about the accuracy of the results Process Technology People

19. Reporting Mechanics Vulnerabilities Report Project Preferences 3 Engine produces various reports Process Technology People

20. Reporting Example 1. Engine Identifies XSS Vulnerability in Scan XSS.NET XSS Rule Model 2. Previously, User Specifies Classification Scheme for Vulnerabilities Risk and Vulnerability Grouping Scheme 2. Engine Produces PDF XSS+Custom Vulnerability

21. Reporting Pitfalls Potential Problems: 1.Report does not take into account risk appetite of organization 2.Reports do not capture useful security metrics. 3.Vulnerability Description / Remediation advice not satisfactory Process Technology People

22. Reporting Solutions Here’s What You Can Do: 1.Demand to see sample reports from vendors before purchasing the scanner 2.Verify that the report’s risk assessment strategy is inline with your organization’s risk methodology 3.Inspect the engine’s capability to customize reports based on security metrics Process Technology People

23. Reporting Solutions Here’s What You Can Do: 4.Verify that you can produce reports that reflect your organization’s security metrics 5.Ask your software developers if they find the reports useful in identifying and fixing the issues Process Technology People

24. Process Impacts DesignBuildTestDeployMaintain Vendor Engagement Code Development Build Code Review QA Security Auditing Vulnerability Management Change Management Risk Assessment

25. Process Impacts Impacts to Processes Are Profound – Where should a scan occur in the SDLC? – How should the results be managed? – Should the organization refuse to release until scans are clean? – How does the organization aggregate the risks? – Does every project get a scan or just some? – How does the organization patch and maintain the scanner? Process Technology People

26. People Impacts Process Technology People DesignBuildTestDeployMaintain Vendors Software Developers Testers Security Auditors Release Engineers Project Managers Risk Analysts Operational Staff

27. People Impacts Process Technology People Impacts to People Are Profound – Who’s responsible for running the scan? – Who do we turn to when results look suspicious? – Who verifies that things are getting fixed? – Who agrees to audit the results? – Who accepts the risks of the associated vulnerabilities? – Who maintains the rules? – Who audits the quality of the scans?

28. Conclusions Source Code Analyzers are powerful and amazingly complex under the covers Anyone who tells you they are the complete solution is probably in sales ;-)

29. Conclusions Developers – Education about the scanner is critical to identifying false positives and negatives Risks Staff – Verify that scanner’s method of risk assessment is aligned with yours.

30. Conclusions Auditors – Don’t be overwhelmed by a lot of issues. Chances are good there are a lot of non-issues (risk appetite). Risk Owners – Insist that the results have been verified by someone who wrote the code

31. Contact Info