1. CENG334 Introduction to Operating Systems Erol Sahin Dept of Computer Eng. Middle East Technical University Ankara, TURKEY Protection Topics: Protection and security problems Object and domains of access Domain structure Domain implementations

2. 2 Sampiyon Fener!

3. 3 Protection and Security Protection and security are two related but different concepts of OSs. Protection: To control the access of programs, processes or users to the resources provided within a computer system. It’s about internal access. Security: To authenticate the system users to protect the integrity of the information stored in the system (both data and code) as well as the physical resources of the computer system. It’s about preventing the access from external agents.

4. 4 Goals of Protection Originally conceived as an adjunct to multiprogramming OS’s so that untrustworthy users can safely share common resources, such as memory, files residing on the disk. Protection ● Prevent violation of access restriction by a user ● Distinguish between authorized and unauthorized usage ● Provide a mechanism for the enforcement of the policies governing resource use ● Some are fixed during the design of the system ● Some are set by the management of the system ● Others are defined by the users of the system

5. 5 Principles of Protection ● Guiding principle – principle of least privilege ● Programs, users and systems should be given just enough privileges to perform their tasks

6. 6 Domain of Protection ● Operating system consists of a collection of objects, hardware or software ● Files, directories, hardware,.. ● A file can be readable but not writable.. ● Each object has a unique name and can be accessed through a well-defined set of operations. ● A CPU can only be executed on ● Memory can be read or written ● CD-ROM can only be read ● Protection problem - ensure that each object is accessed correctly and only by those processes that are allowed to do so.

7. 7 Domain Structure ● A process operates within a protection domain, which specifies the resources that the process may access. ● Each domain defines a set of objects and the types of operations that may be invoked on each object. ● Access-right = where rights-set is a subset of all valid operations that can be performed on the object. ● Domain = set of access-rights ● Right = read, write, execute …

8. 8 Domain Structure ● Domains need not be disjoint. ● For example ● A process in D 2 or D 3 can both execute print on O 4 ● Only a process in D 3 can execute O 1

9. 9 Domain Structure ● Associations between a process and a domain can be ● Static ● Fixed at the time of creation of the process ● May need to provide more rights than needed at the run time ● Dynamic ● A process can switch from one domain to another ● The content of the domain can also be changed

10. 10 Domain design ● Each user may be a domain. ● Access rights depends on the identity of the user. ● Domain switching occurs when the user is changed. ● Each process may be a domain. ● Access rights depends on the identity of the process. ● Domain switching corresponds to one process sending a message to another process, and then waiting for a response. ● Each procedure may be a domain. ● the set of objects that can be accessed corresponds to the local variables defined within the procedure. ● Domain switching occurs when a procedure call is made ● As an example, user and kernel modes define a dual domain system where the processes that run in kernel mode have the right to execute privileged instructions. ● But we also need to protect users from each other!

11. 11 Domain Implementation (UNIX) A domain is associated with the user through uid (user id) and guid (group id) ● Switching domain = changing user identification temporarily Domain switch accomplished via file system. ● Each file is associated with ● An owner identification ● a domain bit (known as the setuid bit). Consider the case where user (with user-id = A) starts executing a file owned by B. ● When the setuid bit is off ● the user-id of the process is set to A. ● When the setuid bit is on, ● the user-id of the process is set to B.

12. 12 setuid ● How does the passwd program work ● When executed by the user, the process runs in the user’s domain ● Cannot modify the /etc/passwd file! ● Solution: passwd program has its setuid bit set, that allows it to run with root access ● Modify /etc/passwd file

13. 13 Domain Implementation (Multics) ● Each ring corresponds to a single domain ● A process running in D i would have more privileges that a process running in D j, if i < j Multics Rings

14. 14 Model of Protection: Access Matrix ● View protection as a matrix (access matrix) ● Rows represent domains ● Columns represent objects ● Access(i, j) is the set of operations that a process executing in Domain i can invoke on Object j

15. 15 Use of Access Matrix (Cont.) ● Access matrix design separates mechanism from policy. ● Mechanism ● Operating system provides access-matrix + rules. ● If ensures that the matrix is only manipulated by authorized agents and that rules are strictly enforced. ● Policy ● User dictates policy. ● Who can access what object and in what mode.

16. 16 Use of Access Matrix ● If a process in Domain D i tries to do “op” on object O j, then “op” must be in the access matrix. ● Can be expanded to dynamic protection. ● Operations to add, delete access rights. ● Special access rights: ● owner of O i ● copy op from O i to O j ● control – D i can modify D j access rights ● transfer – switch from domain D i to D j

17. 17 Switching between domains ● A process executing in D 2 can switch to domain D 3 or D 4 ● A process executing in D 4 can switch to D 1 Allowing controlled change to the contents of the access-matrix entries requires three additional operations: copy, owner, and control.

18. 18 Access Matrix with Copy Rights The ability to copy an access right from one domain (or row) of the access matrix to another is denoted by an asterisk (*) appended to the access right. A process in D 2 can copy the read operation into any entry associated with file F 2

19. 19 Access Matrix with Copy Rights ● Two possible variants: ● A right is copied from access (I,j) to access (k,j); it is then removed from access (i,j); this action is a transfer of a right, rather than a copy. ● Propagation of the copy right may be limited. That is, when the right R* is copied from access (i,j) to access (k,j), only the right R (not R*) is created. A process executing in domain cannot further copy the right R.

20. 20 Access Matrix With Owner Rights Owner rights should allow the addition of new rights and removal of some rights.

21. 21 Access Matrix With Control Rights ● The copy and owner rights provide us with a mechanism to limit the propagation of access rights. ● However, they do not give us the appropriate tools for preventing the propagation (or disclosure) of information ● The problem of guaranteeing that no information initially held in an object can migrate outside of its execution environment is called ● the confinement problem.

22. 22 Implementation of Access Matrix How can the access matrix be implemented effectively? In general, the matrix will be sparse; that is, most of the entries will be empty. ● Options: ● Global table ● Access lists for Objects ● Capability lists for Domains ● A Lock-Key Mechanism

23. 23 Global Table ● Simplest implementation ● A global table consisting of ● triples ● Whenever an operation M is executed on an object O j within domain D i, the global table is searched for a triple with M E R k. ● If this triple is found, the operation is allowed to continue; otherwise, an exception (or error) condition is raised. Pros and Cons: ● The table is usually large and cannot be kept in memory ● Does not take into account special groupings of objects or domains ● If everyone can read a particular object, it must have a separate entry in every domain

24. 24 Access Lists for Objects Each column in the access matrix can be implemented as an access list for one object. The resulting list for each object consists of ordered pairs, which define all domains with a nonempty set of access rights for that object. It can be extended easily to define a list plus a default set of access rights.

25. 25 Capability Lists for Domains A capability list for a domain is a list of objects together with the operations allowed on those objects. ● The capability list is associated with a domain, but it is never directly accessible to a process executing in that domain. ● Rather, the capability list is itself a protected object, maintained by the operating system and accessed by the user only indirectly.

26. 26 Lock-Key Mechanism The lock–key scheme is a compromise between access lists and capability lists. ● Each object has a list of unique bit patterns, called locks. ● Similarly, each domain has a list of unique bit patterns, called keys. ● A process executing in a domain can access an object only if that domain has a key that matches one of the locks of the object. The list of keys for a domain must be managed by the OS on behalf of the domain. Users are not allowed to examine or modify the list of keys (or locks) directly.

27. 27 Comparison Access lists correspond directly to the needs of the users. ● When a user creates an object, she can specify which domains can access the object, as well as the operations allowed. Capability lists do not correspond directly to the needs of the users; they are useful, however, for localizing information for a given process. ● The process attempting access must present a capability for that access. The lock–key mechanism is a compromise between these two schemes. ● The mechanism can be both effective and flexible, depending on the length of the keys. ● The keys can be passed freely from domain to domain.

28. 28 CENG334 Introduction to Operating Systems Erol Sahin Dept of Computer Eng. Middle East Technical University Ankara, TURKEY URL: http://kovan.ceng.metu.edu.tr/ceng334 Security Topics: Goals of security

29. 29 Goals of security in computer systems Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 ● Data confidentiality ● Secret data should remain secret ● Your bank account details ● Data integrity ● Unauthorized user should not be able to modify and change your data ● Your bank account goes from 1,000,000 YTL -> 0 in a second ● System availability ● Nobody should be able to disturb the system to make it unusable ● DOS (Denial of Service) attacks ● Exclusion of outsiders ● Outsiders can take control of other people’s computers converting them into zombies and use them to coordinate their attacks ● Sending spam or coordinating DOS attacks

30. 30 Motivations of intruders and adversaries ● Casual prying by non-technical users ● Setting all files readable ● Snooping by insiders ● Students, systems programmers and other technical personnel consider it a personal challenge to break the security ● Determined attempts to make money ● Attempts to get info to access bank accounts ● Commercial and military espionage ● Serious and well-funded attempt to steal programs, trade secrets

31. 31 Security Measure Levels ● Security must occur at four levels to be effective: ● Physical ● The site containing the computer system should be secure. ● Human ● social engineering: “encourage” someone to share his access with others ● Phishing: Legitimate looking web site makes the victim to enter his confidential information ● dumpster diving: Checking out small papers, post-its notes, around the victim’s desk. ● Network ● The lines that carry information need to be secured. ● Wireless networks are especially vulnerable. ● Operating System ● Only another link within the whole chain. ● Security is as weak as its weakest chain

32. 32 User Authentication Basic Principles. Authentication must identify: Something the user knows Something the user has Something the user is This is done before user can use the system

33. 33 Passwords ● The most common approach to authentication ● When the user identifies herself by user ID or account name, she is asked for a password. ● If the user-supplied password matches the password stored in the system, the system assumes that the user is legitimate. ● Vulnerabilities ● To know the user or to have information about the user ● Social engineering ● Brute force guessing ● 4-decimal password: 10000 possible combinations ● Monitoring ● Expose through visual or electronic monitoring ● Shoulder surfing, keyboard watching ● Network sniffing ● Keylogging ● Exposure ● Passwords written down on paper ● Look for post-its on the victim’s desk, under the keyboard etc. ● Systems that force users to select hard-to-remember and long passwords make them less secure

34. 34 Encrypted Passwords How can the system store a password securely yet allow its use for authentication when the user presents her password? ● Keep a list of user:encrypted_password pairs ● Used to be visible ● There is a function that encrypts passwords ● Not even the root can see your passwords ● But open to cracking through guessing: ● Get the password file, ● Guess passwords using different combinations ● Find the matches A sample line from /etc/passwd owner:Ejrt3EJUnh5Ms:510:102:Some free text:/home/subdir/owner:/bin/bash

35. 35 One time passwords To avoid the problems of password sniffing and shoulder surfing, a system could use a set of paired passwords. ● When a session begins, the system randomly selects and presents one part of a password pair; the user must supply the other part. ● In this system, the user is challenged and must respond with the correct answer to that challenge. ● This approach can be generalized to the use of an algorithm as a password. ● The algorithm might be an integer function, for example. ● The system selects a random integer and presents it to the user. ● The user applies the function and replies with the correct result. ● The system also applies the function. If the two results match, access is allowed. ● Use of a hardware device to generate one-time passwords

36. 36 Authentication through Biometrics ● Measure individual specific features ● fingerprints ● Hands ● Eyes ● Face

37. 37 Authentication Using a Physical Object Magnetic cards ● magnetic stripe cards ● chip cards: stored value cards, smart cards

38. 38 Other Countermeasures The computer system can implement a number of countermeasures to strengthen the security ● Limiting times when someone can log in ● Automatic callback at number prespecified ● Limited number of login tries ● A database of all logins ● Simple login name/password as a trap ● security personnel notified when attacker bites

39. 39 Cryptography as a Security Tool ● Broadest security tool available ● Source and destination of messages cannot be trusted without cryptography ● Means to constrain potential senders (sources) and / or receivers (destinations) of messages ● Based on secrets (keys)

40. 40 Cryptography Kerckhoff’s principle: ● The algorithms should be public and the secrecy should reside exclusively in the keys

41. 41 Secret Key Cryptography ● One of the earliest types of cryptography ● Both encryption and decryption keys should remain secret ● Both the sender and the receiver should have possession of the keys ● The keys should be sent through a secure channel! ● Computation required for encryption and decryption are equal and manageable.

42. 42 Monoaphabetic substitution ABCDEFGHIJKLMNOPQRSTUVWXYZ QWERTYUIOPASDFGHJKLZXCVBNM Plaintext Cipher text Encryption key

43. 43 Monoalphabetic substitution QWERTYUIOPASDFGHJKLZXCVBNM KXVMCNOPHQRSZYIJADLEGWBUFT Plaintext Decryption key

44. 44 Monoalphabetic substitution ● One of the earliest type of cryptography ● There are 26! = 4 X 10^26 possible keys ● But statistical properties of natural languages can be used to crack. ● In English, e is the most common letter ● If in the ciphertext, v is the most common letter, then it is likely to encrypt e. ● two-letter combinations can be used (th, in) ● Suffers from the same vulnerability

45. 45 Public Key cryptography ● Distinct keys are used for encryption and decryption ● Encryption key is made public ● Everyone can see it and use it to encrypt their messages to send ● Decryption key is kept secret ● Only the receiver of the message can use this key to decrypt the message ● For a well-chosen encryption key, it is virtually impossible to discover the corresponding decryption key. ● The source of the symmetry: ● How much is: 213434545454545 X 213434545454545? ● Easy ● What is the square root of 45554305193388235720661157025? ● difficult

46. 46 Digital signatures ● Digital signatures used to sign e-mails or other documents ● To detect that they are not modified by the received ● A hashing function that computes a value for a given document ● MD5

47. 47 Watermarking ● Left: three zebras and a tree ● Right: three zebras and a tree and the complete text of five Shakespeare plays ● Use the 8 th LSB of the pixels to code the compressed text ● To the bare eye, it is invisible

48. 48 Program Threats ● Trojan Horse ● Code segment that misuses its environment ● Exploits mechanisms for allowing programs written by users to be executed by other users ● Examples: ● Spyware that accompanies a program that you’ve downloaded. The goal is to download ads to display on your system and creates pop-up browser windows. ● covert channels: A program may make your computer be a spam server. ● In UNIX, the search path can be long and may include “.” (the current directory). Imagine that you are in your friends home directory and type “ls”. Incidentally, your friend has a “fake ls” in his directory which is a script that deletes your own files in your home directory.. ;((

49. 49 Program Threats ● Trap Door ● The designer of a programmer or a system may leave a “hole” in the software such that it allows him to circumvent normal security procedures ● Could be included in a compiler ● Difficult/impossible to detect.

50. 50 Program Threats ● Logic Bomb ● Program that initiates a security incident under certain circumstances. ● The evil system admin writes a script which checks whether he is still on the payroll or not. If not, the script, deletes files, crashes the system, etcetera in an act of revenge.

51. 51 Program Threats ● Stack and Buffer Overflow ● Exploits a bug in a program (overflow either the stack or memory buffers). ● You should know this inside out from CENG331..

52. 52 Program Threats (Cont.) ● Source code. ● Looks for source code, and modifies it to include the virus. ● Polymorphic ● Virus changes each time it is installed to avoid detection. ● Usually virus cleaners look for “virus signatures” ( a sequence of bytes that characterizes the virus), and these viruses try to avoid them. ● Encrypted. ● The virus includes decryption code along with the encrypted virus, to avoid detection. ● The virus first decrypts and then executes. ● Stealth ● The virus attempts to avoid detection by modifying parts of the system that could be used to detect it. ● For instance, it can modify the read system call such that if the file it modified is read, the original file contents are returned.

53. 53 Program Threats (Cont.) ● Tunnelling ● The virus attempts to avoid detection by installing itself in the interrupt-handler chain or device drivers. ● Multipartite ● Infects multiple parts of a system, boot sector, files, etc.. ● Armoured ● The virus is coded to make itself hard for antivirus researchers to unravel and understand. ● Can also decompress itself to avoid detection.

54. 54 Login spoofing ● A user can write a program that looks exactly like the login screen ● The victim comes, enters his password ● The login spoofing program sends the password and quits ● The victim thinks it mistyped, and enters again to the actual login program

55. 55 Exploiting code bugs ● Buffer overflow attacks ● You know this well ;)

56. 56 Exploiting code bugs ● Format string attacks ● stems from the use of unfiltered user input as the format string parameter in certain C functions that perform formatting, such as printf(). ● A malicious user may use the %s and %x format tokens, among others, to print data from the stack or possibly other locations in memory. ● One may also write arbitrary data to arbitrary locations using the %n format token, which commands printf() and similar functions to write the number of bytes formatted to an address stored on the stack. ● Format string bugs most commonly appear when a programmer wishes to print a string containing user supplied data. ● The programmer may mistakenly write printf(buffer) instead of printf("%s", buffer). ● The first version interprets buffer as a format string, and parses any formatting instructions it may contain. ● The second version simply prints a string to the screen, as the programmer intended. Working example: http://julianor.tripod.com/bc/tn-usfs.pdf

57. 57 Other code bugs ● Return to libc attacks ● Pretty much all C programs link with libc ● Libc contains insecure functions such as strcpy() which copies an arbitrary string from any address to any other address ● Trick strcpy into copying the attackers program, called a shellcode, to the data segment and have it executed there ● Example: http://www.infosecwriters.com/text_resources/pdf/return-to-libc.pdf ● Integer overflow attacks ● For instance give large numbers a image size to a program ● Multiplication causes integer overflow ● Results in smaller buffer than needed ● Provides the ground for buffer overflow attack

58. 58 Other code bugs ● Code injection attacks ● Getting the target program to execute code without realizing it is doing so. > cp abc xyx > cp abc xyz; rm –rf /

59. 59 Other code bugs ● Privilege escalation attacks ● The attacker tricks the system into giving it more access rights that it is entitled to ● Typically he tricks it into doing something that only root can do ● A daemon that allows user to schedule work to be done periodically. ● run as root so that it can access files from the user’s directory ● Has a directory in which it stores commands that are scheduled to run ● Users cannot write to this directory ● The attacker’s program set its working directory to the daemon’s directory. ● It crashes and forces a core dump. ● Core dumps are written into the working directory. ● Dumps can be written there since it is made by the system.

60. 60 Malware Software that does bad things ● Trojan horses ● Hidden in a free program that is providing some useful function ● When called, it can do anything it wants ● Viruses ● Is a program that can reproduce itself by attaching its code to another program ● Can replicate through other programs ● Worms ● Replicates like viruses ● Can self-replicate

61. 61 Program Threats: Viruses ● Viruses ● Code fragment embedded in legitimate program ● Self-replicating and designed to “infect” other programs. ● Very specific to CPU architecture, operating system, applications ● Usually borne via email or as a macro. ● Usually a program called Virus dropper inserts virus onto the system

62. 62 Viruses ● The lifetime: ● Usually written in assembler ● Inserted in a program or a file and distributed through filesharing, or downloads ● Free programs, pirated versions of commercial software ● Once installe, the infected program is executed. ● Companion viruses ● Does not actually infect a program ● When user types prog, MS-DOS looks for a program called prog.com or prog.exe ● Prog.com does not exist, but prog.exe is a useful executable provided on all the systems

63. 63 Program Threats: Viruses ● Many categories of viruses, literally many thousands of viruses ● File. ● The virus appends itself to a file. ● It changes the start of the program so that the execution jumps to its code. ● After it executes, it returns the control to the program and goes unnoticed/ ● Boot. ● Infects the boot sector, executing every time the system is booted. ● Does not appear in the filesystem, and are difficult to detect and clean.

64. 64 Viruses ● Executable viruses ● Simplest version: overwrite an existing exeutable. ● Makes a readdir, and infects other executables ● Open the file and copy the virus code. ● Checks whether the file is already infected.

65. 65 Viruses ● Memory-resident viruses ● Remains active in memory at the very top or the bottom of the address space. ● Can even change the memory bitmap to remain unnoticed. ● Typically captures one of the traps or interrupt vectors, such as the system call trap, and get executed at every system call. ● Can also do lots of spying.

66. 66 Viruses ● Boot-sector viruses ● Overwrite the MBR such that it executes at every boot, before the booting of the OS. ● No OS, no protection: vulnerable state for the system ● Can also disguise itself as bad sectors. ● At boot time the virus copies itself to RAM, and directs the interrupt vectors to itself.

67. 67 Viruses ● Device driver viruses ● Infect a driver and get loaded automatically by the OS when it gets called. ● This is why device drivers should run as user processes. ● Macro viruses ● Macros are programs that can be attached to data files such as Word or Excel. ● Source code viruses ● Change the c files such that ● #include ● Run_virus().

68. 68 Program Threats: Macro Viruses Macro. ● Written in a high-level language, and are triggered when a program capable of running the macro is run. ● For instance a macro virus can be contained in a spreadsheet file. ● Visual Basic Macro to reformat hard drive Sub AutoOpen() Dim oFS Set oFS = CreateObject(’’Scripting.FileSystem Object’’) vs = Shell(’’c:command.com /k format c:’’,vbHide) End Sub

69. 69 Spreading of viruses ● Typically through shared files ● through network ● E-mails ● Downloads ● Browsing ● Sites that provide sexual content ● Sites that provide illegal download (warez’s) ● Sites that provide free programs ● Use checked software

70. 70 Antivirus and Anti-Antivirus Techniques Examples of a polymorphic virus All of these examples do the same thing

71. 71 Antivirus and Anti-Antivirus Techniques Integrity checkers Behavioral checkers Virus avoidance ● good OS ● install only shrink-wrapped software ● use antivirus software ● do not click on attachments to email ● frequent backups Recovery from virus attack ● halt computer, reboot from safe disk, run antivirus

72. 72 The Internet Worm Consisted of two programs ● bootstrap to upload worm ● the worm itself Worm first hid its existence Next replicated itself on new machines

73. 73 Mobile Code (1) Sandboxing (a) Memory divided into 1-MB sandboxes (b) One way of checking an instruction for validity

74. 74 Mobile Code (2) Applets can be interpreted by a Web browser

75. 75 Mobile Code (3) How code signing works

76. 76 Trusted Systems Trusted Computing Base A reference monitor

77. 77 Rootkits, and sony rootkit

78. 78 Multilevel Security (1) The Bell-La Padula multilevel security model

79. 79 Multilevel Security (2) The Biba Model Principles to guarantee integrity of data Simple integrity principle process can write only objects at its security level or lower The integrity * property process can read only objects at its security level or higher