1 Evaluating the Security Threat of Instruction Corruptions in Firewalls Shuo Chen, Jun Xu, Ravishankar K. Iyer, Keith Whisnant Center of Reliable and High Performance Computing Coordinated Science Laboratory University of Illinois at Urbana-Champaign June 24, 2002

2 Objectives Can transient errors cause security vulnerabilities in firewall software? Combine fault injection measurement with processor architecture details to develop a SAN model depicting the reliability, performance, and security of the firewall. Use the SAN model and publicly available security data to assess the relative significance of error-caused security violations.

3 Definitions of Terms Error-caused security vulnerability occurs when an error results in putting the software in a state where any packet can enter the system unchecked. Window of vulnerability is the time period during which such a vulnerability persists Security violation occurs when a number of malicious packets sufficient to launch an actual attack enter the system during a window of vulnerability

4 Errors, Vulnerabilities and Security Violations Temporary SV Erroneous instruction is evicted from cache Permanent SV Detected by intrusion detection systems, or system crash by new faults or latent faults Fault is not manifested Window of temporary security vulnerability Window of permanent security vulnerability Fault crashes the system Error Security vulnerability window System reboot Time t1t1 t2t2 t3t3 t4t4 t5t5 t6t6 t7t7 t8t8 Malicious packets

5 Fault Injection Experiment Address Pool Driver-based Linux Kernel Fault Injector Rule: Reject packet from attacker machine. Firewall Code Firewall machine Attacker Machine Firewall Log 5

6 Outcomes of Fault Injection Experiments Four categories of outcomes Not Activated or Not manifested: 78% CRASH + HANG: 20% Temporary security vulnerability: disappears when the erroneous location is overwritten, cached out or the system is re- booted. 2% Permanent security vulnerability: corrupts the semantic or structural integrity of the permanent data structures. Removing the errors does not eliminate the permanent security vulnerability. 0.05% Fault injection results used as parameters in the SAN model.

7 Error Sub- model Input Gates Workload Sub- model Overview of the SAN Model error error occurrence processor execution core cache cache replacement cache fetch maintenance reboot crash/hang P_SV T_SV reboot not manifested error CPU working packet firewall enable packet processing non-firewall workload idle non-firewall workload processing idle time job dispatch job non-firewall workload execution firewall execution non-firewall workload enable rp_out Error sub-model Workload sub-model flush all places task switch SAN Model: quantifies the relationship between processor architecture, workload, and errors characteristics

8 Error Sub-Model error error occurrence rate processor execution core cache cache replacement cache fetch Crash+Hang Perm. Security Vulnerability Temp. Security Vulnerability NA+NM non-firewall workload ex firewall ex Calculate the probability that a token arrives into Temporary Security Vulnerability or Permanent Security Vulnerability places Calculate the number of packets getting through the firewall in a single vulnerability window

9 Workload Sub-Model packet packet processing non-firewall workload idle non-firewall workload processing idle time job dispatch job

10 Rates of Security Vulnerabilities Rate of Temporary Security Vulnerability (TSV) with 0.1 Error/Day for 20 Firewall Machines Rate of Permanent Security Vulnerability (PSV) with 0.1 Error/Day for 20 Firewall Machines Average 14.9/yearAverage 0.37/year

11 Size of Vulnerability Windows Vulnerability window size links security vulnerabilities and security violations In order to calculate the rates of security violations, we need the distribution of the size of the security vulnerability window Assume 30% packets are malicious

12 Distribution of Number of Packets in a Vulnerability Window Probability Distribution: Processor Utilization by firewall = 50% non-firewall workload=10% malicious packet rate=30% Probability of Security Violation, given a security vulnerability P(security violation | security vulnerability)=0.197

13 Frequency of Security Violations Network protected by 20 firewalls Firewall Processor Util.: 50% Non-firewall workload: 10% Error rate: 0.1 error/day Malicious packet percentage Rate of error-cause violations per year 20% % %2.76 Operating System # kernel-related security vulnerabilities Time periodRate of software security bugs per year RedHat Linux1211/ / Solaris / / Windows / / Rate of Kernel-Related Software Security Bugs Rate of Error-Caused Security Violations

14 Conclusions There exist error-caused security vulnerabilities in firewall software. Transient errors can cause permanent security vulnerability. Errors propagate to permanent data structures. There is a non-negligible probability that error- caused security vulnerabilities become security violations.

15 Major References D. Stott. Automated Fault-Injection-Based Dependability Analysis of Distributed Computer Systems. Ph.D. Dissertation, UIUC, A. Ghosh et al. An Automated Approach for Identifying Potential Vulnerabilities in Software. IEEE Symp. on Security and Privacy, May J. Xu, S. Chen, Z. Kalbarczyk, R. Iyer. An Experimental Study of Security Vulnerabilities Caused by Errors. IEEE DSN01. July /30/2001