Copyright © The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License. The OWASP Foundation OWASP AppSec June 2004 NYC The Need for Metrics and Measurement in Application Security Jack Danahy OWASP Metrics and Measurement Standards Committee Project Lead

OWASP AppSec The Need for Metrics Identify critical areas of focus Set security investment priorities Track effectiveness of remediation and training Target critical remediation needs Evaluate ROI in security training investment Set and monitor security acceptance criteria Monitor compliance with established thresholds Publish trend analyses to document security efforts/progress Evaluate outsourcers’ compliance with contractual requirements Identify critical vulnerabilities early Learn how to fix the vulnerability Confirm vulnerability elimination Monitor performance of development teams and outsourcers Set critical priorities and security exit criteria Publish results Prexis Vulnerability Analysis Data Compliance/ Audit Managers Developers Program Managers Development Managers CSO/CISO

OWASP AppSec OWASP Metrics and Measurement Project Goals  Member survey and outreach to characterize significant and required metrics  Metrics gathering best practices framework  Recommendations for metrics gathering, tool analysis, metrics aggregation and weighting

OWASP AppSec The Case for Measurement The Need for Metrics:  Certification  Prioritization  Remediation  Tracking

OWASP AppSec Metrics for Certification  Governance  Credible, reliable metrics support compliance efforts by demonstrating pervasive security  Stability  Proof of security and lack of excessive patching increase customer confidence and reduce operational risk  Functionality  Validation of appropriate implementation of defined security components ensures that product meets baseline security requirements

OWASP AppSec Metrics for Prioritization  Determine application or project vulnerability  Determine severity of vulnerabilities  Prioritize remediation efforts low exposure Audience and Exposure high exposure Low Value High

OWASP AppSec Metrics for Remediation  Informed business-level decision support  Legacy applications: Wrap it, rewrite it, or replace it  Outsourced projects: Baselines and thresholds drive acceptance criteria and accountability  Resource allocation: focus investments and attention  Efficient workflow for developers  Specific identification of vulnerability  Explanation of vulnerability including potential impact  Conclusive remediation recommendations

OWASP AppSec Metrics for Tracking  Establish baseline and acceptable thresholds  Set accountability expectations with external vendors  Measure team performance  Provide reliable information to all areas of organization  Monitor progress over time requires:  Granularity of information  Periodicity of data (regulatory and public company requirements)

OWASP AppSec Sample Outsourcer Report Card

OWASP AppSec The Case for Measurement  Certification: Provide quantifiable measurement of security  Prioritization: Make informed resource allocation decisions  Remediation: Identify and eliminate risks caused by vulnerabilities  Tracking: Prove progress against reliable baselines and thresholds

OWASP AppSec Call for Participation  Active recruitment efforts underway   Questions? Comments?  Contact me at:

OWASP AppSec Thank you