1. Secure Systems Research Group - FAU Ideas for Operating Systems Patterns by Ed Fernandez and Tami Sorgente

2. Secure Systems Research Group - FAU Security and operating systems (OS) Intermediary between the user of a computer and its hardware Supports the execution of all the applications Architecture is fundamental in the organization its components and utilization of component services Most critical of the software layers Compromise can affect all applications and persistent data Most reported attacks occur through the OS Security Patterns for Operating Systems

3. Secure Systems Research Group - FAU OS Published Patterns File access control Controlled Virtual Address Space Reference Monitor Controlled Execution Environment Authenticator Controlled-Process Creator Controlled-Object Factory Controlled-Object Monitor OS Architecture Patterns

4. Secure Systems Research Group - FAU OS Pattern Ideas/ Future Work Process /Thread structure Administration roles Execution Rings Secure process communication /Protected entry points Distributed directory Isolated execution environments Virtual memory organization Virtual Address Space Structure

5. Secure Systems Research Group - FAU Process structure Process Control Block (PCB) –a data structure containing ID, a reference to its code, program counter, stack, registers, status, and references to its resources (domain). A process receives a separate address space for its execution. –Some architectures, e.g., the Intel X86 series, have hardware support for the type of PCB used by common operating systems (this accelerates context switching, changing execution from one process to another).

6. Secure Systems Research Group - FAU Process Control Block (PCB) Name (ID) identifier state if it is executing/ waiting, etc. PC (program Counter) registers pointer to code code pointer to devices pointer to stack memory limits...... The PCB defines the process context when going to another process need to save this context

7. Secure Systems Research Group - FAU Process States

8. Secure Systems Research Group - FAU Thread structure Lightweight process Faster context switching than a process Address space shared with peer threads. Program counter, a register set, and a stack. Thread Control Block (TCB) Shared address space –error or attack from other thread can corrupt its memory Thread stacks can be protected if they are kept in the system address space using separated segments or pages.

9. Secure Systems Research Group - FAU Thread structure Most modern operating systems, e.g., Solaris, –several threads are in one process – protects the thread group as a whole from other processes User processes - special thread packages – Posix in Unix, or through the language, as in Java or Ada OS defines kernel threads as units of concurrent execution Kernel threads usually don’t have any protection against each other

10. Secure Systems Research Group - FAU Thread Structure

11. Secure Systems Research Group - FAU Process/Thread Structure

12. Secure Systems Research Group - FAU Administration roles AIX reduces the privileges of the system administrator by defining five partially-ordered roles: Superuser, Security Administrator, Auditor, Resource Administrator, and Operator. Windows Windows NT uses four roles for administrative privileges: standard, administrator, guest, and operator. A User Manager has procedures for managing user accounts, groups, and authorization rules. Trusted Solaris ---This is an extension of Solaris 8. RBAC is used to restrict the rights of administrators, there is no superuser with all power. Argus Pitbull [Arg]—Least privilege applied to all processes, including the superuser. The superuser is implemented using three roles: Systems Security Officer, System Administrator, and System Operator. [Arg] Argus Systems Group, “Trusted OS security: Principles and practice”, http://www.argus- systems.com/products/white_paper/pitbull [Cam90] N.A.Camillone, D.H.Steves, and K.C.Witte, “AIX operating system: A trustworthy computing system”, in IBM RISC System/6000 Technology, SA23-2619, IBM Corp., 1990, 168-172.

13. Secure Systems Research Group - FAU Execution Rings Some hardware architectures define a set of protection rings (typically 4 to 32) with hierarchical levels of trust. Rings are a generalization of mode of operation and define finer domains of execution. Multics and the Intel processors among others. A combination (process, domain) corresponds to a row of the access matrix. Crossing of rings is done through gates that check the rights of the crossing process Process calling a segment in a higher ring must go through a gate.

14. Secure Systems Research Group - FAU Execution Rings

15. Secure Systems Research Group - FAU Execution Rings Processes are assigned to rings based on their level of trust; for example, if we had four rings we could assign them in decreasing order of privilege to: supervisor, utilities, trusted user programs, untrusted user programs The Intel X86 architecture applies two rules: –Calls are allowed only in a more privileged direction, with possible restriction of a minimum calling level. –Data in a segment at level p can be accessed only by a program executing at a more privileged level (<= p).

16. Secure Systems Research Group - FAU Execution Rings Program Status word of the process indicates its current ring and data descriptors also indicate their assigned rings. These values are compared to apply the transfer rules. Ring crossing applies protected entry points. A process calling a higher- privilege process can only enter this process at pre-designed entry points with controlled parameters. Jumping many rings is suspicious. Multics defined a call bracket, where calls are allowed only within rings in the bracket. More precisely, for a call from procedure i to a procedure with bracket (n1, n2, n3) the following rules apply: if n2 n3 the call is not allowed, if i < n1 any entry point is valid. This extension only makes sense for systems that have many rings. Rings don’t need to be strictly hierarchic, partial orders are possible and convenient for some applications. For example, a system including a secure database system could assign a level to this database equal but separated from system utilities; the highest level is for the kernel and the lowest level is for user programs. This was done in a design involving an IBM 370 [Fer78].

17. Secure Systems Research Group - FAU Secure process communication Process communication also has an effect on security Systems that use explicit message passing have the possibility of checking each message to see if it complies with system policies

18. Secure Systems Research Group - FAU Process communication Direct- indirect (mailbox) Symmetric (must know name)- asymmetric (does not need to know name) Synchronous (blocking)- asynchronous (non blocking) Automatic or explicit buffering Send by copy or send by reference Fixed size or variable size messages

19. Secure Systems Research Group - FAU Protected entry points Systems that use explicit message passing have the possibility of checking each message to see if it complies with system policies. For example, a security feature that can be applied when calling another process is protected entry points. A process calling another process can only enter this process at pre-designed entry points. This prevents bypassing entry checks. The number and size of arguments in a gate crossing can also be controlled (this may protect against some types of buffer overflow attacks).

20. Secure Systems Research Group - FAU Direct communication

21. Secure Systems Research Group - FAU Indirect Communication

22. Secure Systems Research Group - FAU Distributed directory Windows 2000 introduced the concept of Active Directory (AD) that provides an administrative structure for distributed domains. The AD is a tree that acts as a repository or registry for all resource information. A domain is a group of systems that share the same directory; in this sense the AD is a directory of directories (the Composite pattern again !). Each domain has a unique name and provides centralized administration for user and group accounts. Users can cross domains after being properly authenticated.

23. Secure Systems Research Group - FAU Distributed directory The AD introduces the concept of dynamic inheritance of rights along the tree. This is a direct application of the concept of implied authorization that we proposed in 1975 [Fer75]. Implied access rights are very convenient for system administration because the administrator can understand better the structure of rights in the system. W2000 also improved and expanded the use of groups for security and introduced the use of Kerberos for distributed authentication

24. Secure Systems Research Group - FAU Memory Protection Memory allocation most systems use paging, segmentation, or a combination of both Page table entries can be used to describe the permitted type of access to pages, such as read, write, execute Because pages do not correspond to logical units in a program this protection is rather imprecise

25. Secure Systems Research Group - FAU Page Addressing

26. Secure Systems Research Group - FAU Memory Protection Segmentation- Segments are variable-size units that correspond to logical program units Processes are given a descriptor segment that contains segment descriptors that indicate their type of access to a set of memory segments Can apply a more accurate protection Two approaches: capabilities and descriptors

27. Secure Systems Research Group - FAU Capabilities and descriptors Capabilities are a generalization of base/limit register pairs that include the type of permitted access and which are given to the process in a relatively persistent way Descriptors are similar to capabilities but instead of being carried with the process code they are loaded when the process is created

28. Secure Systems Research Group - FAU Capabilities and descriptors Descriptors and capabilities are usually combined with address translation and then their use adds very little overhead, although ring crossing is relatively slow The set of all the pages or segments (described by page entries or descriptors) assigned to a process represent its virtual address space (VAS); the set of all pages or segments that can be allocated to processes is the system virtual address space

29. Secure Systems Research Group - FAU Access control through segmentation

30. Secure Systems Research Group - FAU Virtual address space structure There are four Basic Approaches: 1.One address space per process 2.Two address spaces per process 3.One address space per user process, all of them with one address space for the OS 4.A single-level address space

31. Secure Systems Research Group - FAU Virtual address space structures

32. Secure Systems Research Group - FAU One address space per process supervisor gets its own address space NS32000, WE32100, and Clipper microprocessors good isolation but sharing is complex –(special instructions to cross spaces are needed)

33. Secure Systems Research Group - FAU Two address spaces per process Motorola 68000 series Data and instructions can be separated for better protection –(some attacks take advantage of execution of data or modification of code) problem of complex sharing plus a rather poor address space utilization

34. Secure Systems Research Group - FAU One address space per user process, all of them shared with one address space for the OS VAX series and in the Intel processors Not the best with respect to security –(the supervisor has complete access to the user processes and it must be trusted) Convenient for sharing utilities and other system programs Disadvantage is that the address space available to each user process has now been halved

35. Secure Systems Research Group - FAU A single-level address space Everything, including files, is mapped to this memory space Multics, IBM S/38, IBM S/6000, and HP’s PA-RISC Most elegant solution –(only one mechanism to protect memory and files) Potentially the most secure if capabilities are also used Hard to implement due to the large address space required

36. Secure Systems Research Group - FAU Isolated execution environments Virtual machines Tagged storage Capabilities Separation by encryption [Dal01] C. Dalton and T.H. Choo, “An operating system approach to securing services”, Comm. of the ACM, vol. 44, No. 2, February 2001, 58-64.

37. Secure Systems Research Group - FAU Virtual Machines VMM (virtual machine monitor) hardware Windows XP Linux VM1VM2

38. Secure Systems Research Group - FAU Virtual Machines

39. Secure Systems Research Group - FAU Summary Process /Thread structure Administration roles Execution Rings Secure process communication /Protected entry points Distributed directory Isolated execution environments Virtual memory organization Virtual Address Space Structure EuroPLoP deadline: January 28, 2005 Interested?