Virtual Machine Security Design of Secure Operating Systems Summer 2012 Presented By: Musaad Alzahrani

Outline Introduction Virtualization Benefits Virtualization Architectures Virtualization Techniques Security Benefits Security Vulnerabilities Conclusion

Virtualization is abstracting the hardware resources of a machine. It enables running multiple operating systems on virtual machines on the same processing hardware. Each virtual machine behaves like an independent machine. Virtualization reduces the total number of physical machines and consolidates several virtual machines on a single physical machine. Introduction

Virtualization Benefits Save hardware cost and footprint: virtualization provides the ability to take advantage of multiple operating systems on one physical PC. This allows us to buy less hardware and reduce overall system footprint. Take advantage of operating system services: with virtualization it is possible to take advantage of the capabilities offered by different operating systems on just one set of hardware. Make use of multicore processors: virtualization software can allow users to directly assign groups of processor cores to individual operating systems. For example, if a user wishes to use Linux and a real-time OS, more CPU and memory resources can be allocated to the real-time OS to optimize performance. Test beta software and maintain the legacy applications: programmers can test new releases of software without the need for dedicated test machines. If beta software corrupts a given operating system, a parallel operating system running on the same computer can still be used for development. Increase system security: virtualization reduces the need for multiple physical computers that operate at different security levels but are not fully utilized.

Virtualization Software To virtualize a given computer, a piece of software called a virtual machine monitor (hypervisor) must be installed on host OS or physical hardware. After this VMM software is installed, individual virtual machines VMs can be run on the same hardware. Each virtual machine can run its own operating system (guest OS). VMM manages guest OS and its interaction with host OS or physical hardware. It performs process scheduling, memory management, I/O management, and network management operations.

Virtualization Architectures There are two major virtualization architectures: hosted and bare-metal. Hosted virtualization: VMM is installed on top of a host operating system such as Windows Examples: Oracle VirtualBox, Microsoft Virtual PC and VMWare Workstation Bare-metal virtualization: VMM is installed directly on hardware for more low-level access. Examples: Microsoft Hyper-V, Oracle VM Server(Xen) and Amazon EC2(Xen)

Protection Rings x86 CPUs provide a range of protection rings in which code can execute. Ring 0 has the highest level privilege and is where the operating system kernel normally runs. The hypervisor runs directly on the hardware of the host system in ring 0.

Virtualization Techniques Traditional operating system sits directly above the hardware executing in the ring 0. In virtualization there are three of the underlying techniques: Paravirtualization, Full Virtualization without Hardware Assist, and Full Virtualization with Hardware Assist.

Paravirtualization Under paravirtualization, the kernel of the guest operating system is modified specifically to run on the hypervisor. This involves replacing any privileged operations that will only run in ring 0 of the CPU with calls to the hypervisor (known as hypercalls) and the hypervisor in turn performs the task on behalf of the guest kernel. This typically limits support to open source operating systems, such as Linux.

Full Virtualization without Hardware Assist It provides support for unmodified guest operating systems such as Windows. The term unmodified refers to operating system kernels which have not been modified to run on a hypervisor and, therefore, still execute privileged operations as though running in ring 0 of the CPU. The hypervisor provides CPU emulation to handle and modify privileged and protected CPU operations made by unmodified guest operating system kernels. This emulation process requires both time and system resources to operate, resulting in inferior performance levels when compared to those provided by paravirtualization.

Full Virtualization with Hardware Assist Hardware virtualization leverages virtualization features built into the latest generations of CPUs from both Intel and AMD. These technologies, known as Intel VT and AMD-V, provide extensions necessary to run unmodified guest virtual machines without the overheads inherent in full virtualization CPU emulation. These new processors provide an additional privilege mode below ring 0 in which the hypervisor can operate essentially, leaving ring 0 available for unmodified guest operating systems.

Security Benefits  Abstraction Hypervisor abstracts the hardware layer and each VM is allocated its own strictly bounded resources. This layer of abstraction provides additional security. Hypervisor is much simpler than traditional OS, So it is much easier to secure. Since the attacker does not know details of the host environment, manipulating and compromising the machine is much more difficult.

Security Benefits..  Isolation The hypervisors segment physical resources into isolated entities and allow each guest OS to run independently. Each VM encapsulates the guest OS and prevents a malicious guest OS from accessing resources it does not own. An attack on one VM should not affect any of the other VMs on the server or the host OS.

Security Benefits..  State restore VMs are able to restore to a previous state. The contents of the virtual disk for each VM are usually stored as a file on the host. Most VMs take a snapshot of the contents of the virtual disk when changes are made or on a time interval. When VM is compromised, the hypervisor can remove that VM or restore it to a state prior to attack.

Security Benefits..  Transience VMs can be started remotely. This allows them to be turned on and made available only when needed. Minimizing how much time a given computer is online is the best deterrent against malicious attacks, since an offline server cannot be accessed.

Security Benefits..  External monitoring Since VMs run on a subset of hardware resources, it is possible observe VM resource usage and detect malicious software from outside the VM. VMs can be monitored by an authorized dedicated VM that can view software activity. The hypervisor can give the dedicated VM permission to view resources allocated to the monitored VM.

Security Vulnerabilities  VM sprawl The biggest vulnerability of virtualization is due to the ease in which users can create many VMs in a short time. It becomes very difficult to secure, monitor, and maintain each VM. Traditional security methods need to be applied to each VM since the guest OS accesses the network directly. A compromised VM is a potential entry point for attackers to the hypervisor and host. VM sprawl wastes resources and creates more entry points for attackers.

Security Vulnerabilities..  State restore Even though the ability of a VM to restore to a previous state is often considered a security benefit to protect against data loss, returning to an unpatched or compromised state is a great danger. A VM may get a security patch, but if for some reason the user needs to rollback to a previous state, then the guest OS is no longer patched. Another concern is returning to a compromised state. A machine may detect a virus and remove it from the system. If a user returns to a state prior to virus removal, the virus may exist on the system.

Security Vulnerabilities..  Mobility Virtual machines are not physical, which means their theft can take place without physical theft of the host machine. The contents of the virtual disk for each VM are stored as a file by most hypervisors, which allows VMs to be copied and run from other physical machines. Attackers can copy the VM over the network or to a portable storage media and access data on their own machine without physically stealing a hard drive.

Security Vulnerabilities..  Hypervisor intrusion The hypervisor is a program, running on the host, so if it is compromised, all VMs it controls and the host itself are accessible to the attacker. If the host OS is not securely protected, the attacker could corrupt or externally modify guest OS while the VM is offline.  Hypervisor modification It does not matter how secure the original hypervisor is if it can be externally modified to use the attacker’s software. One attack of this form is known as Virtual Machine Based Root Kits (VMBR). In this attack, the hypervisor’s system calls to the host OS are changed to run malicious code.

Security Vulnerabilities..  Communication Attackers can use one VM to access or control other VMs on the same hypervisor. A malicious VM can potentially access other VMs through shared memory, network connections, and other shared resources. For example, if a malicious VM determines where another VM’s allocated memory lies, then it could read or write to that location and interfere with the other’s operation.

Security Vulnerabilities..  Denial of service An improperly configured hypervisor can allow a single VM to consume all resources, thus starving any other VM running on the same physical machine.

Conclusion Virtualization allows multiple OS installations to share the same hardware resources. The hypervisor manages these resources and to create the virtual environment for each guest OS. When virtualizing a machine, either hosted or bare-metal virtualization can be used. At a low level, these architectures depend on techniques such as paravirtualization, full virtualization without hardware assist, and full virtualization with hardware assist to accomplish virtualization.

Conclusion.. The hypervisor provides an additional layer of abstraction from physical hardware. This abstraction encapsulates malicious attacks and allows external monitoring for malicious attacks on a VM. Virtualization itself is not inherently unsecured; it is a new technology that potentially has new vulnerabilities and requires restructuring of manual security processes.

References On state of the art in virtual machine security: Qian Chen; Mehrotra, R.; Dubeyy, A.; Abdelwahed, S.; Rowland, K. Southeastcon, 2012 Proceedings of IEEE Digital Object Identifier: /SECon , Publication Year: 2012, Page(s):

Thank you for listening Questions?