1. Continuous and Visible Security Testing
with BDD-Security
Stephen de Vries
@stephendv

2. About me CTO Continuum Security 16 years in security
Specialised in application security
Author of BDD-Security framework
I’ve spent most of my career as a penetration tester and security consultant testing web and mobile applications. And let me say that as a security specialist it’s great to be at a conferences focussed on development and operations. Can I see with a show of hands who’s primary role is in security.

3. We’re not popular people
We’re not popular people. We don’t get invited to meetings because we’re the no guys, the guys who cause project delays because of our onorous security requirements. And this is particularly true for security testing which for many organisations is synonymous with penetration testing.

4. Security testing still stuck in a waterfall world
Feedback from security testing is too late
Rely on outside security “experts”
There are problems with relying on pentesting as a core security activity:
Which means it is a single control gate at the end of a development cycle.
The biggest problems is that it’s far too late in the development cycle.
and increasingly the interesting security vulnerabilities are in the business processes in the software itself.

5. Security is not something you add…
…it’s something that’s build in, just like quality,
scalability and performance
If we want to get security testing out of this model we don’t have to start from scratch, because we already handle other software properties in Agile and CD models with great success.
None of those properties can simply be added to an application, we have to build them in from the start. And security is no different.

6. Everyone is responsible for Move testing closer to the code
Continuous automated testing
security ^
quality
And looking at how testing has evolved over time two things stand out:
we’re doing more automation,
and we’re doing that automated testing closer to the code. Our unit and integration tests are a key part to making continuous delivery possible.
And I claim that that we can and should take the same approach with security testing: automated as much as possible, so that we can run them continuously and get feedback as quick as possible.
Not to replace manual security testing, but to complement it.

7. Difference of degree, not of kind
Quality testing
Security testing
And there are really a lot of similarities in the acts of quality and security testing.
the differences to security testing are of degree and not of kind.
We’re both testing functional and non-functional aspects of the software, we’re both testing boundary conditions. We’re both looking for defects. And we both find those defects by testing the boundaries of the system. Where the quality test is focused on functional defects, security testing is focused on defects that can be abused by a determined attacker in order to gain some benefit for themselves. So they’re looking for software failures that fail in a special way.
Although there are some differences between the two, those differences are of degree and not of kind. So instead of treating security as a special kind of activity, we can instead treat it as a specialised form of quality testing.

8. Non-Functional Security
Why
What
How
Business Context
Architecture
App Features
Threat Model
Non-Functional Security
Requirements
Functional Security
Requirements
Security Tests
Where should security tests come from?
The process differs from functional requirements and tests.
The central part of the process is the threat model: who is likely to attack me, what are they likely to attack and whether we care about those attacks or not?
We get to the threat model by analysing the business context defines the type of data you’re dealing with and also your apettite for risk.
The technical architecture of your application, web, mobile, is it internal only, deployed to cloud on hosted etc. Together with the features that you offer, e-commerce, or funds transfer or even just a simple search function in an app all contribute to defining the threat model.
At the end of the threat modeling step you also perform a simple risk analysis to find out which threats matter and should be addressed, and which don’t.
For this process to succeed should be visible. That is to say that the whole team, security operations and developers understand why we’re building specific features and how we’re testing that they work. If this is visible to the whole team then there’s less chance of misunderstanding requirements, or in fact ignoring security requirements.

9. Here’s a quick example of how the business context can influence the functional security requirements. Consider a typical password reset function.

10. This is a data leak.
An attacker somewhere on the Internet could determine if a given address is registered to the site. …and if the attacker had a database of addresses they could automated this.
But for online retailers this is not a serious risk.

11. But consider if you were running a dating site for spouses to have affairs. The threat model is different, because the potential attackers are different. In their threat model they assume that spouses checking up on each other are one of their potential attackers.

12. So they’re modified their password reset function accordingly.
So security testing is not just about avoiding implementation bugs, we need to have a good set of security requirements and from those build our security tests which should including functional aspects of the app.

13. Security Requirements
BDD-Specs (Given/When/Then)
Visible
Testable
Actionable
Up-to-date
Automated
Security Testing > Scanning
Requirements must meet the key criteria of being visible to the developers and operations team, actionable and up to date. The preferred tool of the security practioner here is the document or spreadsheet and I know what happens to those, they get filed in a drawer and never looked at again.
They must be detailed actionable requirements and not high level desires like: “Take adequate steps to secure personal data”. That’s not a requirement, that’s just a desire.
We should be able to test for security architecture flaws such as functional security in the app, as well as implementation bugs, XSS, SQLi
And by Using BDD style tools that use the given/when/then specifications. BDD specs are effectively automated tests written in a natural language so they do double duty as both a specification and a test.
This is the biggest win because we effectively have self-verifying specifications. This forces them to be up to date, because they’re not just serving as documentation but as tests as well.
GHERKIN

14. BDD-Security Testing Framework
BDD-Security = JBehave +
Selenium +
OWASP ZAP +
Nessus +
Internal security tools +
Pre-written baseline security specifications
----- Meeting Notes (09/11/14 12:49) ----- 20
Narrow domain

15. Examples: Infrastructure specifications

17. False positives

18. Security specifications for application itself
Authentication:
Passwords should be case sensitive
Present the login form itself over an HTTPS connection
Transmit authentication credentials over HTTPS
When authentication credentials are sent to the server, it should respond with a 3xx status code.
Disable browser auto-completion on the login form
Lock the user account out after <X> incorrect authentication attempts
Scanning the infrastructure is only part of the security story, we also want to test the functional aspects of the security of the app itself.

19. Manual Application Security Testing with OWASP ZAP
HTTP/S Proxy

20. BDD-Security Automated
Manual Application Security Testing with OWASP ZAP ^
Selenium
HTTP/S Proxy
ZAP API
BDD-Security

21. Configuring BDD-Security for in-depth testing
Edit config.xml with app specific values
Create Java class that defines Selenium methods for:
openLoginPage
Login
isLoggedIn
Logout

22. Demo
From bdd spec to here = 11m

23. Application Security Scanning with ZAP
From

27. Can Alice see Bob’s data?
Testing Access Control
Can Alice see Bob’s data?
From

28. Demo

29. Part of Continuous Integration process
Ant job in Jenkins
Run job after deploy to test environment
Fail the build if tests fail

30. Demo

31. Summary
Security testing doesn’t need special treatment: it differs from software testing in degree, not in kind
Automated Security tests can be integrated into a CI/CD model
Automated Security tests should include more than just scanning
BDD tools provide self-verifying specification
BDD-Security project to jump-start your own security specs

32. Similar tools
ZAP-JUnit (Java)
Guantlet (Ruby)
Mittn (Python + Burp Intruder)

33. I’ll be at Office Hours 13:45 Today Room: 211 @stephendv
Thank you
I’ll be at Office Hours 13:45 Today Room: