Program Security Richard Newman

What is Program Security? Security of executing software - Make software that functions properly (no bugs) – Good development practices/software engineering - Make sure you run good software – Known, trustworthy source - Make sure software that is run has not been changed – During distribution – While on host waiting for execution - Make sure processes can't do bad things – Restrict process access (usual protection mechanisms) – Sandbox - Monitor processes in case they do bad things anyway – Audit

What is a Programmed Threat? Potential source of harm from computer code May be in form of - Executable program - Executable code attached to another program - Executable code pushed onto stack of running process - Standalone script - Commands run on startup of program - Commands embedded in “non-executable” file – JPEG – Postscript - Macros

Examples of Programmed Threats 1. Trojan Horse – Program that purports to do one thing but (also) does another 2. Virus – Embedded in another program/file (becomes Trojan) – Must get user or system to run program/open file – Infects other files/drives – Hitchhikes to other file systems on host file via removable media or 3. Bacteria/Rabbits – Replicate so fast, use up all resources 4. Worm – Stand-alone program – Transfers itself to target system – Runs automatically on target system (generally)

More Programmed Threats 5. Buffer overflow attack – “Improper” parameters corrupts stack – Includes executable code – Return pointer in activation frame may be changed to point to code 6. SQL Injection – Interpretable commands included in SQL query – SQL engine executes malicious commands 7. Run command script – Malicious commands included in.rc (or similar) file – Commands executed when program is started

More Programmed Threats 8. Back Door/Trap Door – “Secret” way to get access to system – May be included for field technicians or administrators – See – Often first goal of intruders 9. Covert Channels – Violate information flow policy – Concern in MultiLevel Secure (MLS) systems – Type of Trojan Horse 10. Bugs – Most common :( – Traditionally, most costly

Exposures 1. Unmediated Access – Trap door/back door – Worm – Buffer overflow 2. Information Leaks – Covert Channel – Virus or worm activity – Trap door/back door 3. Logic & Time Bombs – Trojan Horse – Virus/Worm activity – Take special action when triggered by conditions – Time bomb a special case – condition is time 4. Unavailability – Rabbits, worms, botnets

Virus Desiderata 1. Detection Resistant – Evade detection by stealth measures 2. Robust – Hard to deactivate/remove/destroy 3. Infectious – Wide-ranging – Reinfection 4. Easy to create 5. Machine/OS/Application-independent – Able to infect wide variety of targets

Virus Dimensions 1. Lifetime – Transient – run once each time program launched – Resident – continue to run periodically or on events 2. Target – Boot sector – TSR code – Library code – Application – Document/image 3. Attachment method 4. Infection route – Removable media – diskettes, CD/DVD, thumb drives – /MIME – Downloaded files - FTP/HTTP

Viruses 1. History – Von Neumann's self-reproducing automata in 1960's – See – First seriously appeared in early 1980's – Elk Cloner, Brain – Big issue with PCs and floppy disks/bulletin boards 2. General MO – Infected program run – viral code runs first – Optionally takes measures to hide – Looks for new files/drives to infect, infects them – Does “other stuff” Logic Bomb Time Bomb Password cracking Install back door Wreak havoc – Returns control to original program

Viruses 3. Boot Sector Virus – Copies boot sector (small bootstrap program) to unused disk block – Overwrites boot sector with viral code – Intercepts calls to disk drive/TSR code – Redirects reads of boot sector to read copy in other location – Looks for new disk to infect whenever disk is accessed 4. Executable Virus – Adds viral code to executable program – May rewrite JUMP instruction to jump to viral code first, then issue JUMP to program code when done – May modify itself (code transformation) or modify where it is stored to evade detection (polymorphic virus)

Viruses 5. Macro Virus – Included in “non-executable” file with format supporting macros Spreadsheets Document preparation software Graphics editors – Copies macros into other files of same type – Modifies file contents to exercise macros Any format that has “active content” can provide a way for virus to take hold!

Basic Virus Figure 3-4 Virus Appended to a Program.

Virus attachment via GOTOs Figure 3-5 Virus Surrounding a Program.

Virus attachment inline Figure 3-6 Virus Integrated into a Program.

Virus replacement Figure 3-7 Virus Completely Replacing a Program.

Boot Sector Virus Figure 3-8 Boot Sector Virus Relocating Code.

Virus Detection 1. Recognize storage patterns – Modification date of file – Size of file 2. Recognize content change – Specific pattern in code – Checksum change – MAC change 3. Recognize viral programs – Static code analysis – limited by undecidability (halting problem) – Still, can do triage! 4. Detect bad execution patterns – Attempt to access inappropriate files – Attempt to open abnormal network connections – Abnormal system call sequences

Virus Controls Figure 3-9 Recognizable Patterns in Viruses.

Virus Stealth Methods 1. Modify system meta-information – Modification date – Access date – Process information – File control block/i-node table/SFT/etc. 2. Intercept system calls – Modify call/results (man-in-the-middle) 3. Compress target and itself – So file size does not change 4. Modify itself – Polymorphism (don't change functionality) – Evolution (change functionality) 5. Encryption – Viral code encrypted to hide purposes, methods – Also gives “free” polymorphism to some extent

Virus Controls 1. Back-ups/restore points 2. Buy COTS software from reliable vendors 3. Test new code on isolated system – Observe behavior – Fiddle with date/time 4. Run virus scanner – Keep up to date (always behind!) – Test outgoing as well as incoming files 5. Access control – Limit damage of infected programs to user running code 6. Hardware-based protection – Prevent damage to other processes, illegal execution – Protected instructions, mode bit(s), VM, write protection, etc. 7. File signatures – Audit files, system configuration, OS, applications, libraries, etc.

Worms 1. History – 1971 “Creeper virus” at BBN - “Reaper” to kill it – Name coined in Brunner's 1975 “The Shockwave Rider” – Enabled by network/LAN technology – Xerox PARC worm for using idle workstations (1982) – Morris worm 1987 – Code Red, etc. 2. General MO – Standalone program – Looks for target host – Transfers loader (micro-FTP) to target host See

PARC Worm 3. Xerox PARC worm Users ran server pgm on W/S when idle Worm “head” found idle workstations, sent work “Segments” did work, reported to head Head had backup segments also Had to shut down all stations to get to stop! See Shoch and Hupp, “The Worm Programs: Early Experience with a Distributed Computation,” Xerox Palo Alto Research Center,

Morris Worm 4. Morris worm – November 2, 1988 Experiment by grad student at Cornell University Looks for target host – random, /etc/hosts,.rhosts, hosts.equiv Tried to get access Sendmail “feature” - debug mode Symmetry of trust Finger flaw – buffer overflow Password guessing, common accounts/passwords Transferred “grappling hook” to target host (boot loader) Grappling hook got rest of worm, ran it (one-time password) Overwhelmed hosts with processes Overwhelmed networks with traffic

Morris Worm 4. Morris worm (con't) Stealth techniques “encrypted” code (flipped MSB in ASCII) Changed process name to innocuous pgm Changed process ID periodically – short life per proc Died completely after short time Sendmail access Back door, poor configuration, poor interface Symmetry of trust Remote login without password required Host lists trusted hosts If a host B is on list of A, likely host A is on list of B spaf.cerias.purdue.edu/tech-reps/823.pdf

Code Red Worm 5. Code Red Worm – July 2001 Attacked MS IIS Buffer overflow attack Patch had been available for a month Spread Only 1 st – 19 th of month – look for other IIS servers Did not determine if IIS server was vulnerable first Mischief Deface website - “Hacked by Chinese” Launch DoS attack 20 th -27 th of month vs. fixed IP addr

Code Red Worm 5. Code Red Worm IIS buffer overflow: GET /default.ida?NNNNNNNNNNNNNNNNNNNNNNNNN NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN NNNNNNNNNNNNNNNNNNN %u9090%u6858%ucbd3%u7801%u9090%u6858%ucbd3%u7801 %u9090%u6858%ucbd3%u7801%u9090%u9090%u8190%u00c3 %u0003%u8b00%u531b%u53ff%u0078%u0000%u00=a HTTP/1.0

Worm Controls 1. Prevent remote access – Only connect to network if necessary, when necessary – Shut down unneeded servers – Use firewall to limit access – Use VPNs 2. Protect remote access points – Dial-in callback – Proper configuration 3. Limit possible damage – Run servers at lowest possible privilege level – Run servers on special hosts with limited access – Remove general utilities/tools from server hosts 4. Monitor – Look for unusual access/behavior patterns, traffic

Trapdoors Allow unauthorized access Local – Magic password – Unauthorized user name/password Remote – Can be used for remote administration – Allows access over network

Trapdoor Controls 1. Testing for undocumented “features” 2. Code reviews 3. Maintenance – verify patches/updates 4. Monitor for logins/remote accesses 5. Check for input testing/bounds checking 6. Test for undefined machine opcodes

Buffer Overflow Figure 3-1 Places Where a Buffer Can Overflow.

More directed malicious threats Salami attack Collect small amounts of money/time/space Remain undetected (“in the noise”) Many drops of water make up the sea.... Remain because of rounding errors poor processes poor audit Covert channels Information leakage against system policy Generally modified utility program Privileged user runs Trojan horse Info transmitted in unusual way, “in the noise”

Covert Channels Form of Trojan Horse Modified program/driver Legitimate user (with access rights) runs code TH accesses other data, leaks it to low privilege process Figure 3-11 Covert Channel Leaking Information.

Covert Channel Types Storage Channel State of shared resource can be modified by service pgm State can be “read” by receiver Examples: File lock, or file presence Resource exhaustion (disk blocks, memory, i- nodes,...) Numbers handed out (process ID, etc.) In noise of accessible file (steganography) Timing Channel Rate or responsiveness of access to dynamic resource Examples: CPU slice access in timeshared system Network access time, ACK response time, etc. Timing channel can be converted to storage channel

Covert Channel Example Figure 3-13 File Lock Covert Channel.

Covert Channel Example Figure 3-14 File Existence Channel Used to Signal 100.

Covert Channel Example Figure 3-15 Covert Timing Channel.

Covert Channel Controls Prevent unauthorized flows to begin with Information flow policy and enforcement Standard methods for preventing unauthorized code changes Review code/system for possible flows Information flow and control flow analysis of code Shared Resource Matrix (SRM) method List shared resources Determine which modules modify them, read state Determine possible flows, then real flows Estimate maximum data rate for channels found Close channel if possible Make channel noisy or slow if can't close Audit for exercise of channel

Programmed Threat Controls Physical Access Access Control Process Isolation Virtualization Sand boxes Program verification Proof-carrying code Signed code Honey Pots Monitoring Development Controls Distribution and Deployment Controls

Development Controls Good design methodology Separation of Duty Version Management/Revision Control Configuration Management Verification Clean-room program development Design/Code Reviews Testing & Validation Proof-carrying Code

Good Development Practice Figure 3-19 Fault Discovery Rate Reported at Hewlett- Packard.

Configuration Management Change control Version control Configuration management – stable configs Backups, shadow copies, immutable versions Regression testing Audit trail Separation of duty

Good Design Practice Modularity Encapsulation – minimal coupling Information hiding Code reuse Design for testability/verification

Development Controls Figure 3-16 Modularity.

Good Design Practice Figure 3-17 Coupling.

Good Design Practice Figure 3-18 Information Hiding.

Best Design Practices Clean-room programming (Harlan Mills) – Design from requirements – Verify formally – Only then code and test – Testing is easy, fast – Premise is that understanding comes first – Results in better code, sooner – Proof carrying code (Necula) – Include computer readable proof – Recipient has mechanical verifier

Cost of Bugs Figure from Software Engineering Economics by Barry Boehm

Process Improvement TQM/CQI/CPI/SEI Capability Maturity Model/ ISO 9000/1, etc. Structured process so outcomes are Predictable Repeatable (not necessarily good!) Continuous improvement Feedback from results of product Feedback from results of process

SEI CMM Levels 1) Inital – chaotic 2) Repeatable 1) Planning 2) Islands of process 3) Configuration management 3) Defined 1) Management support, training 2) Standardization, communication 3) Peer reviews, documentation 4) Managed - Quantitative measures, analysis 5) Optimizing – use of feedback

NSA's SSE CMM NSA extended CMM for System Security Engineering Three areas: 1) Engineering (development) 1) SE development practices 2) Includes security analysis, vulnerability analysis 2) Project (management) 1) Quality assurance 3) Organizational 1) Training 2) Process improvement

Distribution & Deployment Controls Secure Path – From trusted source to current execution – Example: ctl-alt-del for login Signed Code – Code carries digital signature – Must check signature with verification key – Verification key must be secured and current – Certificate binding key must be trustworthy Proof-carrying Code – Does not rely on secure transmission – Does not rely on trustworthy source – Has proof of security properties attached – Simple theorem prover used to verify code