1. Operating System Security CS 136 Computer Security Peter Reiher February 4, 2010

2. Outline Interprocess communications protection
File protection and disk encryption
Protecting other OS resources
Logging and auditing

3. Protecting Interprocess Communications
Operating systems provide various kinds of interprocess communications
Messages
Semaphores
Shared memory
Sockets
How can we be sure they’re used properly?

4. IPC Protection Issues
How hard it is depends on what you’re worried about
For the moment, let’s say we’re worried about one process improperly using IPC to get info from another
Process A wants to steal information from process B
How would process A do that?

5. Message Security Process A Process B That’s probably not going to work
Gimme your
secret
That’s probably not going to work
Can process B use message-based IPC to steal the secret?

6. How Can B Get the Secret? He can convince the system he’s A
A problem for authentication
He can break into A’s memory
That doesn’t use message IPC
And is handled by page tables
He can forge a message from someone else to get the secret
But OS tags IPC messages with identities
He can “eavesdrop” on someone else who gets the secret

7. Can an Attacker Really Eavesdrop on IPC Message?
On a single machine, what is a message send, really?
A message is copied from a process buffer to an OS buffer
Then from the OS buffer to another process’ buffer
Sometimes optimizations skip some copies
If attacker can’t get at processes’ internal buffers and can’t get at OS buffers, he can’t “eavesdrop”
Need to handle page reuse (discussed earlier)

8. Other Forms of IPC Semaphores, sockets, shared memory, RPC
Pretty much all the same
Use system calls for access
Which belong to some process
Which belongs to some principal
OS can check principal against access control permissions at syscall time
Ultimately, data is held in some type of memory
Which shouldn’t be improperly accessible

9. So When Is It Hard?
Always possible that there’s a bug in the operating system
Allowing masquerading, eavesdropping, etc.
Or, if the OS itself is compromised, all bets are off
What if the OS has to prevent cooperating processes from sharing information?

10. The Hard Case Process A Process B
Process A wants to tell the secret to process B
But the OS has been instructed to prevent that
A necessary part of Bell-La Padula, e.g.
Can the OS prevent A and B from colluding to get the secret to B?

11. OS Control of Interactions
OS can “understand” the security policy
Can maintain labels on files, process, data pages, etc.
Can regard any IPC or I/O as a possible leak of information
To be prohibited if labels don’t allow it

12. Covert Channels Tricky ways to pass information
Requires cooperation of sender and receiver
Generally in active attempt to deceive system
Use something not ordinarily regarded as a communications mechanism

14. Covert Channels in Computers
Generally, one process “sends” a covert message to another
But could be computer to computer
How?
Disk activity
Page swapping
Time slice behavior
Use of a peripheral device
Limited only by imagination

15. Handling Covert Channels
Relatively easy if you know what the channel is
Put randomness/noise into channel to wash out message
Hard to impossible if you don’t know what the channel is
Not most people’s problem

16. File Protection
How do we apply these access protection mechanisms to a real system resource?
Files are a common example of a typically shared resource
If an OS supports multiple users, it needs to address the question of file protection

17. Unix File Protection
A model for protecting files developed in the 1970s
Still in very wide use today
With relatively few modifications
To review, three subjects
Owner, group, other
and three modes
Read, write, execute
Sometimes these have special meanings

18. Setuid/Setgid Programs
Unix mechanisms for changing your user identity and group identity
Either indefinitely or for the run of a single program
Created to deal with inflexibilities of the Unix access control model
But the source of endless security problems

19. Why Are Setuid Programs Necessary?
The print queue is essentially a file
Someone must own that file
Can’t make it world-writable
Or people will delete each other’s jobs
So how will people put stuff in the print queue?
Typical Unix answer is run the printing program setuid
To the owner of the print queue
Program only allows limited manipulation of the queue

20. Why Are Setuid Programs Dangerous?
Essentially, setuid programs expand a user’s security domain
In an encapsulated way
Abilities of the program limit the operations in that domain
Need to be damn sure that the program’s abilities are limited

21. Some Examples of Setuid Dangers
Setuid programs that allow forking of a new shell
Setuid programs with powerful debugging modes
Setuid programs with “interesting” side effects
E.g., lpr options that allow file deletion

22. Encrypted File Systems
Data stored on disk is subject to many risks
Improper access through OS flaws
But also somehow directly accessing the disk
If the OS protections are bypassed, how can we protect data?
How about if we store it in encrypted form?

23. An Example of an Encrypted File System
Issues for encrypted file systems:
When does the cryptography occur? Ks
Where does the key come from?
Transfer $100 to my savings account
Sqzmredq #099 sn lx rzuhmfr zbbntms
What is the granularity of cryptography?

24. When Does Cryptography Occur?
Transparently when a user opens a file?
In disk drive?
In OS?
In file system?
By explicit user command?
Or always, implicitly?
How long is the data decrypted?
Where does it exist in decrypted form?

25. Where Does the Key Come From?
Provided by human user?
Stored somewhere in file system?
Stored on a smart card?
Stored in the disk hardware?
Stored on another computer?
Where and for how long do we store the key?

26. What Is the Granularity of Cryptography?
An entire disk?
An entire file system?
Per file?
Per block?
Consider both in terms of:
How many keys?
When is a crypto operation applied?

27. What Are You Trying to Protect Against With Crypto File Systems?
Unauthorized access by improper users?
Why not just access control?
The operating system itself?
What protection are you really getting?
Data transfers across a network?
Why not just encrypt while in transit?
Someone who accesses the device not using the OS?
A realistic threat in your environment?

28. Full Disk Encryption All data on the disk is encrypted
Data is encrypted/decrypted as it enters/leaves disk
Primary purpose is to prevent improper access to stolen disks
Designed mostly for laptops

29. Hardware Vs. Software Full Disk Encryption
HW advantages:
Probably faster
Totally transparent, works for any OS
Setup probably easier
HW disadvantages:
Not ubiquitously available today
More expensive (not that much, though)
Might not fit into a particular machine
Backward compatibility

30. An Example of Hardware Full Disk Encryption
Seagate’s Momentus 7200 FDE product line
Hardware encryption for entire disk
Using AES
Key accessed via user password, smart card, or biometric authentication
Authentication information stored internally on disk
Check performed by the disk itself, pre-boot
300 Mbytes/sec sustained transfer rate
Primarily for laptops

31. Example of Software Full Disk Encryption
Vista BitLocker
Doesn’t encrypt quite the whole drive
Need unencrypted partition to hold bootstrap stuff
Uses AES for cryptography
Key stored either in special hardware or USB drive
Microsoft claims “single digit percentage” overhead
One independent study claims 12%

32. Protecting Other Resources
Devices
The processor itself

33. Protecting Devices
Some handled through file system protection/authentication
Others by only allowing kernel to access them
User access mediated by the kernel
Assumes kernel will be careful in what it allows users to do
Sometimes issue with alternate access
E.g., Firewire interfaces

34. Protecting the Processor
Mostly about protecting processes from each other
But also about ensuring user-level processes can’t execute privileged instructions
Most typically, instructions OS needs to do its work
Syscalls used to explicitly change mode

35. Presumptions for Processor Protection
Assume context switch code clears out all info from old process
Registers, etc.
Model is that processor is clean and devoted only to current user
Switch to privileged mode is explicit
Don’t allow arbitrary user code to run in this mode

36. Logging and Auditing An important part of a complete security solution
Practical security depends on knowing what is happening in your system
Logging and auditing is required for that purpose
Often (partially) built into OS

37. Logging No security system will stop all attacks
Logging what has happened is vital to dealing with the holes
Logging also tells you when someone is trying to break in
Perhaps giving you a chance to close possible holes

38. Access Logs One example of what might be logged for security purposes
Listing of which users accessed which objects
And when and for how long
Especially important to log failures

39. Other Typical Logging Actions
Logging failed login attempts
Can help detect intrusions or password crackers
Logging changes in program permissions
A common action by intruders
Logging scans of ports known to be dangerous

40. Problems With Logging Dealing with large volumes of data
Separating the wheat from the chaff
Unless the log is very short, auditing it can be laborious
System overheads and costs

41. Log Security
If you use logs to detect intruders, smart intruders will try to attack logs
Concealing their traces by erasing or modifying the log entries
Append-only access control helps a lot here
Or logging to hard copy
Or logging to a remote machine

42. Local Logging vs. Remote Logging
Should you log just on the machine where the event occurs?
Or log it just at a central site?
Or both?

43. Local Logging Only gives you the local picture
More likely to be compromised by attacker
Must share resources with everything else machine does
Inherently distributed
Which has its good points and bad points

44. Remote Logging
On centralized machine or through some hierarchical arrangement
Can give combined view of what’s happening in entire installation
Machine storing logs can be specialized for that purpose
But what if it’s down or unreachable?
A goldmine for an attacker, if he can break in

45. Desirable Characteristics of a Logging Machine
Devoted to that purpose
Don’t run anything else on it
Highly secure
Control logins
Limit all other forms of access
Reasonably well provisioned
Especially with disk

46. Auditing Security mechanisms are great`
Security mechanisms are great
If you have proper policies to use them
Security policies are great
If you follow them
For practical systems, proper policies and consistent use are a major security problem

47. Auditing
A formal (or semi-formal) process of verifying system security
“You may not do what I expect, but you will do what I inspect.”
A requirement if you really want your systems to run securely

48. Auditing Requirements
Knowledge
Of the installation and general security issues
Independence
Trustworthiness
Ideally, big organizations should have their own auditors

49. When Should You Audit? Periodically
Shortly after making major system changes
Especially those with security implications
When problems arise
Internally or externally

50. Auditing and Logs Logs are a major audit tool
Some examination can be done automatically
But part of the purpose is to detect things that automatic methods miss
So some logs should be audited by hand

51. A Typical Set of Audit Criteria
For a Unix system
Some sample criteria:
All accounts have passwords
Limited use of setuid root
Display last login date on login
Limited write access to system files
No “.” in PATH variables

52. What Does an Audit Cover?
Conformance to policy
Review of control structures
Examination of audit trail (logs)
User awareness of security
Physical controls
Software licensing and intellectual property issues

53. Does Auditing Really Occur?
To some extent, yes
2008 CSI/FBI report says more than 64% of responding organizations did audits
Doesn’t say much about the quality of the audits
It’s easy to do a bad audit