1. Educating System Testers in Vulnerability Analysis: Laboratory Development and Deployment Leonardo A. Martucci, Hans Hedbom, Stefan Lindskog, and Simone Fischer-Hübner Department of Computer Science Karlstad University SWEDEN

2. 2 WECS'7 Outline Introduction and background Course overview Course content Hands-on assignments Evaluation and lessons learned Conclusion Errata

3. 3 WECS'7 Introduction The constantly growing number of security vulnerabilities threats incidents has led to increased investments in the development of more secure systems The lack of security functionality and assurance may result in high costs Vulnerability analysis (VA) is an important means for improving security assurance of IT systems during test and integration phases

4. 4 WECS'7 Background A large telecom company decided to increase their efforts in VA by educating their software testers They decided to outsource the education and training of its testers A compact (3 days) VA course was developed at our department The course has been held 3 times during 2005 for a total of 45 participants

5. 5 WECS'7 Course Overview The emphasis of the course is on practical hands-on assignments The course is aimed for software testers with little or no security experience extensive knowledge in software testing The topics included in the course is based on a preliminary list of topics specified by the contractor A set of laboratory assignments were derived from this list Approximately 30-40% covers theoretical aspects and the rest is used for practical assignments

6. 6 WECS'7 Course Content The course content is divided into 4 blocks: Introduction to computer and network security Motivation, evaluation criteria, security standards, risk analysis, and ethics Computer and network security protocols and tools Cryptography, IPSec, SSH, SSL/TLS, PKI, VPNs, IDSs, firewalls, and a set of laboratory assignments Vulnerability analysis The four steps of VA: (1) reconnaissance, (2) research and planning, (3) attack mounting, and (4) assessment Known vulnerabilities, reconnaissance tools and information gathering Common host attacks, malicious code, node hardening, and several practical laboratory assignments

7. 7 WECS'7 Hands-on Assignments The following laboratory assignments are included: password cracking testing for randomness firewall black box testing network analyzing (and ARP spoofing) port scanning node hardening security scanning Final project Putting it all together (i.e., from grain to bread)

8. 8 WECS'7 Ethical Rules The participants were requested to follow the following ethical rules: Do not experiment with VA-tools without explicit permission of an authorized party Do not pass on/publish material, tools, and vulnerabilities to unauthorized parties Do not use your technical skills in criminal or ethically questionable activities Always report flaws to vendors/developers first Software tools provided in this course must only be used in a laboratory environment and on laboratory computers

9. 9 WECS'7 The Laboratory Environment The laboratory was prepared for 20 students working in pairs Each pair have their own workstation Each workstation Was dual boot – Windows XP and Feodora Core 3 Linux Equipped with an Ethernet NIC The laboratory was also configured with two servers One running Windows 2000 Server The other running Feodora Core 3 Linux The servers were in some assignments the target

10. 10 WECS'7 Password Cracking Goal To show that weak passwords could be a serious threat Running the assignment The password cracking tool John the Ripper was used to detect weak passwords on their own workstation running Linux Some easy to break passwords were introduced in the password file Knowledge obtained The participants have tested a password cracking tool to identify weak passwords

11. 11 WECS'7 Testing for Randomness Goal To educate the participants in how to identify non-random properties in sequences produced by a pseudo random number generator (PRNG) Running the assignment The NIST statistical test suite was used to evaluate outputs from different PRNGs A short introduction on hypothesis testing was needed in order for the participants to evaluate the output from the tool Knowledge obtained The participants have learned that: good PRNGs are a crucial cryptographic primitive automatic tools exist to validate PRNGs

12. 12 WECS'7 Firewall Goal To provide hands-on experience on how firewall rules in Linux using ipTables can be used Running the assignment The participants were asked to write firewall rules for the setup in the figure in order to implement a given policy Knowledge obtained The participants have the knowledge to write, read, understand, verify and evaluate firewall rules

13. 13 WECS'7 Black Box Testing Goal To learn how a protocol implementation can be evaluated using a black box testing method Running the assignment The PROTOS tool was used to evaluate the SNMP protocol in a CISCO 1005 router A ready-made test suite to perform a DoS attack was used Knowledge obtained The participants have learned that black box testing using automatic tools can be used to evaluate implementations of communication protocols

14. 14 WECS'7 Network Analyzing (and ARP Spoofing) Goal To show how easy it is to capture network traffic in a LAN using Ethereal Running the assignment Ethereal was used to capture a password sent over the network using TELNET Knowledge obtained The participants have learned how to manage a network analyzer to capture network traffic

15. 15 WECS'7 Port Scanning Goal To demonstrate how port scanners can be used to find open ports in a networked computer Running the assignment The participants were asked to gather information about open ports on the two servers using the Network MAPper (NMAP) in Linux Knowledge obtained The participants have learned how to use a port scanner to find unexpected open ports in a product before deployment

16. 16 WECS'7 Node Hardening Goal To educate the participants on how to increase the security of nodes by turning off unnecessary services restricting the rights of necessary services verifying that used software uses the latest patches Running the assignment The Bastille tool was used When running Bastille, a large set of questions are asked on how the user would like the node to be configured and after that automatically configure the system according to the answers Knowledge obtained The participants have learned the importance of correct configurations and to handle a node hardening tool

17. 17 WECS'7 Security Scanning Goal To show how to use security scanners in order to automatically scan the system for known vulnerabilities Running the assignment Two unpatched servers running Windows 2000 Server and Fedora Core 3 Linux were acting as targets Both the Internet Scanner (IS) and Nessus were used as scanners Neither the configuration nor the IP addresses of the servers were known to the students Knowledge obtained The participants have learned that security scanners are tools that can assist the testers in the verification process

18. 18 WECS'7 Putting it all Together Goal To let the participants conduct a full VA of a target with limited resources and time (<8 hours). Running the assignment The assignment was conducted in groups of 4 students Each group had two workstations and one server that was the target of evaluation The group was given a requirement specification describing the role of the server and its security requirements The exercise was to find out what has to be done to fulfill the requirements, perform the necessary changes and verify the result Knowledge obtained The participants have gained a better understanding on how to perform a full- scale VA

19. 19 WECS'7 Evaluation and Lessons Learned After each course instance, the participants have been asked to fill in a questionnaire used to evaluate the course Based on the answers, the following conclusion can be drawn The most popular assignments have been: Security scanning, port scanning, and node hardening The least interesting assignments have been: Testing for randomness and firewall Each participant has either been satisfied or very satisfied with the course We have also noticed that having a system administrator available during the course would greatly reduce the burden on the teachers

20. 20 WECS'7 Concluding Remarks A vulnerability analysis (VA) course aimed for software testers is described in the paper The focus is on the various laboratory assignments provided within the course All participants have either been satisfied or very satisfied with the course and we are convinced that the course has significantly raised their awareness concerning security and VA An investigation of how the participants use their knowledge in VA will be performed during spring 2006 Three new instances of the course are scheduled in 2006

21. 21 WECS'7 Errata Page 2, third sentence in second paragraph, i.e.: Students from an applied computer security course were engaged and trained to attack a target system and evaluate its security [2]. Delete and trained in the sentence.