824 scriptie definitief - vurore.nl · virtualization s.c. klaver 3 1 introduction in this chapter...

Virtualization

Challenges that virtualization bring to traditional IT security controls.

Coen Klaver Post graduate IT audit Vrije Universiteit Amsterdam University counselor: Kees van Hoof Company counselor: Lande Castberg

Virtualization S.C. Klaver

2

Table of Contents

1 INTRODUCTION............................................................................................................................... 3

1.1 REASON........................................................................................................................................ 3 1.2 SCOPE .......................................................................................................................................... 3 1.3 RESEARCH QUESTIONS ................................................................................................................. 4

2 VIRTUALIZATION ........................................................................................................................... 5

2.1 WHAT IS V IRTUALIZATION ? ......................................................................................................... 5 2.1.1 Traditional pillar .................................................................................................................... 5 2.1.2 Virtualization pillar ................................................................................................................ 6 2.1.3 Types of Virtualization ............................................................................................................ 6 2.1.4 Virtualization requirements .................................................................................................... 7

2.2 SUB-TYPES OF PLATFORM V IRTUALIZATION ............................................................................... 8 2.2.1 Full Virtualization .................................................................................................................. 8 2.2.2 Paravirtualization ................................................................................................................... 8 2.2.3 Hardware-Supported Virtualization ....................................................................................... 9 2.2.4 Single Kernel Image ..............................................................................................................10

2.3 TYPE I AND II VIRTUALIZATION ...................................................................................................10 2.4 BUSINESS DRIVERS ......................................................................................................................11 2.5 V IRTUALIZATION PRODUCTS .......................................................................................................11

2.5.1 IBM virtualization products ...................................................................................................12 2.5.2 VMware ESX server ...............................................................................................................16 2.5.3 Virtual networking .................................................................................................................20

3 RISK ASSESSMENT ........................................................................................................................22

3.1 HARDWARE .................................................................................................................................22 3.2 HYPERVISOR/KERNEL .................................................................................................................24 3.3 NETWORK ...................................................................................................................................25 3.4 V IRTUAL MACHINES ...................................................................................................................25 3.5 MANAGEMENT CONSOLES ..........................................................................................................27 3.6 STORAGE .....................................................................................................................................28

4 TRADITIONAL IT CONTROLS ....................................................................................................30

4.1 ASSET CLASSIFICATION ..............................................................................................................30 4.2 TYPE I OR TYPE II? ......................................................................................................................31 4.3 SEGREGATED ENVIRONMENTS ....................................................................................................32 4.4 INFORMATION LABELING AND HANDLING ...................................................................................35 4.5 MONITORING SYSTEM USE ..........................................................................................................35 4.6 CONTROL OF TECHNICAL VULNERABILITIES ................................................................................36

5 CONCLUSION ..................................................................................................................................38

6 APPENDIX A (USED SOURCES) ...................................................................................................40

7 DEFINITIONS ...................................................................................................................................42


3

1 Introduction In this chapter I will discuss the background and scope of this thesis.

1.1 Reason Virtualization is taking off in corporate environments. The focus is on the commercial benefits such as cost reduction through consolidation and maximizing the utilization of servers. The security impact from implementing virtualization is not often considered. One of the most commonly heard statements is that traditional security controls are sufficient and that virtualization does not need additional measures. Is that indeed the case? The introduction of virtualization within a Microsoft Windows-only environment will almost certainly bring a Unix-flavor operating system in the landscape. This will require additional hardening guides and modifications to the security monitoring software. Also, how does your patch management process handle copies of virtual machines that are offline? These are just simple examples that indicate that traditional security controls probably can not be copied one-to-one when deploying virtualization but will need customization. When performing IT audits and risk assessments in the past I have noticed that virtualization is already quite common but that the virtualization itself is rarely in scope of these audits or risk assessments. This is especially the case for low range products such as IBM p–series or VMware ESX on PC-architecture hardware. The audit takes into account the application, middleware and operating system and than jumps to the physical security whilst skipping the virtualization. An exception is the virtualization on the IBM mainframes.

1.2 Scope The scope of this thesis is the bare-bone platform virtualization. This is the virtualization of operating systems directly on the hardware. This is also called Type I virtualization. I will also touch upon virtual networks, as the virtualization products I look into also offer the ability to create virtual networks. In a traditional IT model there is a business process that relies on a certain application. This application is installed on an operating system on a server dedicated to that operating system and application. The IT control framework is build around this model. When virtualization is introduced, an additional layer is added. The application and operating system are now connected via the virtualization layer to the hardware. The direct connection with the underlying hardware is flexible. Based on this I will answer the question: what will change in the traditional IT control framework when the extra layer virtualization is introduced?


4

I will demonstrate that based on a risk assessment, additional controls must be implemented in the IT control framework to lower or remove any risk that virtualization introduces. This will allow organizations to manage the risk based on their risk appetite. It was my intention to base my paper on the NIST 800-53 framework. However along the way it turned out that the ISO 27001 standard gave me better support for this paper.

1.3 Research questions My main research question in this paper is: What will change to the IT control framework when the layer Virtualization is introduced? I formulated the following sub questions: Can virtual machines with different Confidentiality, Integrity and Availability ratings (CIA ratings) be combined on the same hardware and under what conditions? The CIA rating above relates to the data that is transported or processed by these virtual machines. Can a Development, Test, Acceptance and Production (DTAP) environment be combined on the same hardware and to what extent? With a DTAP I refer to the separated computing environments that are used for development, testing, acceptance testing and running production. I also formulated an optional question: Which virtualization specific commercial security products are there on the market? I decided not to follow up on this question for the following reasons:

• The maximum size of the paper was 30 pages. I am already exceeding this; • It is a paper subject on its own.


5

2 Virtualization

2.1 What is Virtualization? Virtualization is a very broad term for several techniques for abstracting software from the underlying architecture. Popek and Goldberg describe in their Architectural Principles for Virtual Computer Systems.PhD thesis, a Virtual Machine as an “efficient hardware-software duplicate of a real machine” [1]. During my research I found many and varied definitions of virtualization. See Appendix B. The definition of virtualization I defined for this paper is: “Virtualization is a technology that combines or divides IT hardware resources in logical objects by acting as an interface between the IT hardware resources and software. It abstracts the software from the underlying hardware”. I decided to create my own definition because some of the definitions only focused on the dividing of hardware resources, others were very complex. Also the whole scope of virtualization is very broad and it includes several different technologies. In my introduction I write that virtualization is taking off in corporate environments. However virtualization itself is not a new technology. It was developed in the 1960s by IBM to provide users concurrent access to their mainframes.

2.1.1 Traditional pillar Normally there is a one to one relationship between the software (application), the operating system, and the hardware. The operating system is specially built for and is installed on the hardware. The operating system runs in ring 0 and controls the hardware. For instance Windows XP must run on PC-architecture hardware and cannot be installed on a system with a Power-PC processor. The software is installed in the operating system and is specially built for a certain type of operating systems. Exceptions are cross platform applications such as those built in Java. The traditional x86 processors such as Intel and AMD use four privilege levels, called rings. Ring 0 is the most privileged ring and ring 3 is the least privileged. Normally an OS runs in ring 0 and user applications run in ring 3. Rings 1 and 2 are normally not used these days (see figure 1).


6

Figure 1, traditional pillar

2.1.2 Virtualization pillar With virtualization the direct relationship between the software and the underlying hardware is broken by an additional layer between the operating system and hardware. This new layer abstracts the operating system from the hardware. This makes it possible to have many types of operating systems on one type of hardware. It also allows an operating system to migrate from one physical machine to another very easily.

Figure 2, Virtualization pillar

2.1.3 Types of Virtualization There are several types of virtualization. With one technique the virtualization can be used to combine several IT resources (such as processor, memory, disk space or name space) which are then presented to the outside world as one logical object by the virtualization software.


7

The types of virtualization I refer to are:

• Resource virtualization; • Platform virtualization.

These IT resources can be fragmented or decentralized, but via virtualization they are presented to the outside world as one logical object. This virtualization technique is also known as “resource virtualization”. Examples are grid computing, virtual memory, logical volume management and channel bonding. Another technique is “platform virtualization”. This technique divides one IT resource (CPU, storage) into multiple logical objects. Platform virtualization can take place in different layers of the IT infrastructure:

• Between application and operating system (middleware clustering); • Between operating system and hardware (virtual machines).

Application virtualization is the complete abstraction and isolation of the application from the operating system and hardware. Application virtualization provides a container in which an application runs without directly interacting with the hardware resources. An example is Java Virtual Machine (JVM). The JVM is installed on a computer in the operating system. It provides a virtual machine that java applications run in. Once Java applications are compiled they can run on any architecture as long as the JVM is installed. My paper will focus on the platform virtualization between the operating system and the hardware. Some types of platform virtualization use an operating system between the hardware and the virtualization layer i.e. the virtualization software runs in the operating system. An example is VMware Workstation (this is different software from VMware ESX server). I will only focus on the so called “bare metal” platform virtualization: where the virtualization directly runs on the hardware. As stated above, platform virtualization allows multiple instances of operating system (OS) to run concurrently on the same hardware. These instances of operating systems are known as “guest OS” or “virtual machines”. The guest OS is separated from the hardware by a Virtual Machine Monitor (VMM), also known as a ‘hypervisor’. A single virtual machine can communicate with other virtual machines, as if each is a separate physical hardware (physical machine). Each virtual machine can be activated and restarted independently of the others. If the operating system fails in a virtual machine, there is no impact on the operating system running in other virtual machines. Similarly, applications running in one virtual machine have no impact on applications running in other virtual machines. The hypervisor sits between the guest OS and the hardware and controls the use of CPU, memory, and storage for the guest OS. The hypervisor is the interface between the hardware and the virtual machine.

2.1.4 Virtualization requirements Popek and Goldberg describe in their paper [1] the following three requirements for VMM’s:


8

1. Equivalence. The guest OS running in the virtual machine expects to execute on a physical machine. In order to execute properly the environment provided by the VMM needs to represent all aspects of a physical machine and must be indistinguishable.

2. Resource Control. The hypervisor must be in control of the physical system. (i) No virtual machine is allowed to directly access a physical resource not explicitly allocated to it. (ii) The VMM needs to be able to transparently regain control of a previously allocated resource.

3. Efficiency. A “statistically dominant subset of the virtual processor’s instructions needs to be executed directly on the real processor”. The VMM shouldn’t cause noticeable performance degradation when executing virtualized software.

2.2 Sub-Types of Platform Virtualization Within platform virtualization there are also different sub-types to be recognized. The difference between these flavors is in the way the guest OS is isolated from the underlying hardware. The following sub-types of platform virtualization can be distinguished:

• Full virtualization; • Paravirtualization; • Hardware supported virtualization.

2.2.1 Full Virtualization With full virtualization the hypervisor simulates a stand-alone instance of a computer with enough hardware (i.e. processor, hard disk, I/O ports, DMA channels, and Interrupts) to run a guest OS. The guest OS is not aware it is being virtualized and it requires no modification to run. This approach has some small overhead for each running virtual machine instance. The hypervisor intercepts all operating system instructions. For instance, CPU operations are processed by the virtual processor. Also, device I/O is enabled by emulating common devices within the VMM. The instructions for these virtual devices are translated and provided to the actual physical devices by the hypervisor. Full virtualization provides the best isolation between the guest OS and the hardware. This also makes the transport of fully virtualized machines between different physical platforms very simple.

2.2.2 Paravirtualization Paravirtualization requires modification of the guest OS to operate. It is also said that these guests are “aware” that they are virtualized. The differences are in the way the guest OS communicates with the hypervisor. Compared to full virtualization the system calls are provided to hypervisor via so-called hypercalls. The OS must be modified to be able to produce these hypercalls. The hypervisor also provides hypercall interfaces to the guest OS for kernel operations such as memory management. With paravirtualization there is less virtualization overhead, which should make is faster than full virtualization.


9

Standard operating systems such as Windows 2003 are not able to make use of paravirtualization out of the box. Modification of the operating system is needed or the hardware must support virtualization (see section 2.2.3.1). The open source Xen project is an example of paravirtualization that uses a modified Linux kernel and virtualizes the I/O using custom guest OS device drivers.

2.2.3 Hardware-Supported Virtualization Hardware-supported virtualization is available for PC-architecture processors and for IBM mainframes. With hardware-supported virtualization the support is already incorporated in the CPU design. Hardware virtualization does provide not provide a complete virtualization solution on its own. It is provides support for full virtualization by implementing the support already in the processor.

2.2.3.1 PC-architecture

The original x86 architecture did not meet the Popek and Goldberg virtualization requirements. Especially the Equivalence and Resource Control requirements were not met. For instance, privileged access failures were not intercepted and managed by the hypervisor but failed silently. This means that the hypervisor was not able to represent all aspects of the underlying hardware. With the traditional pillar the kernel of the operating system has direct CPU access as it is running in ring 0. With software virtualization techniques such as described in section 2.2.1 and 2.2.2 the hypervisor or VMM runs in ring 0. The guest OS runs in ring 1 or 3. This enables the isolation of the hypervisor from the guest OS (see figure 2). But the kernels of the operating system in the guest OS expect to run in ring 0 and a number of x86 instructions only work in ring 0. An option is to recompile the guest operating system (paravirtualization). However this is only possible if the source code is available. With full virtualization the hypervisor traps these ring 0 instructions and translates the instructions for the guest operating systems. This results in a decrease in performance. At this moment both Intel and AMD have released processors with specific hardware support for virtualization. Intel calls its virtualization technology VT and AMD’s technology goes by the name AMD-V, also known as Pacifica. Hardware-supported virtualization supports the use of two new CPU operations: a “root operation” and a “non-root mode”. Both operations have their own four privilege levels as used for normal x86 architecture. The hypervisor runs in root mode and all guest OSs run in non-root mode. As in paravirtualization, special hypercalls are used for the communication between the guest OSs and the hypervisor. These hypercalls take care of the resource allocation and device I/O. They also introduced a new ring, ring -1.

2.2.3.2 IBM mainframe architecture Virtualization is quite common on IBM mainframes. For the IBM Sytem/370 hardware (1977) the virtualization was intended to make the virtual environment appear as similar to the underlying hardware as possible. But the focus was on the correctness and not on the performance. The System/370 Extended Architecture (370-XA) incorporated several special virtualization “assists” to the processor architecture. These assists replaced


10

common functions and helped increase the performance of virtualization for these mainframes. The range of IBM mainframes assists was further increased in Enterprise Systems Architecture (ESA) which was introduced in 1988.

2.2.4 Single Kernel Image With Single Kernel Image (SKI) the host OS spawns multiple copies of itself. Therefore this type of virtualization can only run one type of OS. This is restricted even further because every instance must have the same kernel version and patch level. SKI is a very light form of virtualization. The main advantage is the better performance compared to the other type of virtualization but a strong disadvantage is the inflexibility. With SKI the guest OS’ security is at the same level as that of its host. SKI virtualization is used by Sun Solaris Zones.

2.3 Type I and II virtualization Another way to categorize virtualization types is by classifying them as a type I or II. With type I the hypervisor runs directly on the hardware. The micro operating system or kernel has been specially built for virtualization. The hypervisor directly accesses and controls the hardware. Type I hypervisors rely on a separate operating system for administration activities, the so called management console. The hypervisor runs in ring 0 or -1. With type II the hypervisor runs as an application in the operating system. The operating system is not specially built with virtualization in mind. The hypervisor relies on the host operating system to control access to the hardware. Here the host operating system runs in ring 0.

Figure 3, Type II and I hypervisor


11

This distinction between type I and II was made by John Robin in his paper: Analyzing the Intel Pentium's Capability to Support a Secure Virtual Machine Monitor [20 ]. The distinction does not mean that a type II hypervisor has a lower quality than a type I hypervisor. The reason that I focus in this paper on type I hypervisor only is that type II hypervisors will have different risks due to the fact that a type II runs as an application within an operating system.

2.4 Business drivers One of the biggest business drivers for implementing virtualization in corporate environments is the cost saving and cost avoidance. Implementing virtualization does not bring cost reduction by reducing the amount spent on software licenses. The number of used applications will not change and for every virtual machine instance an official operating system is still required. However, cost savings will be made in the following areas:

• A one time reduction in hardware acquisition costs. The capacity of hardware is utilized more efficiently and development testing and production can be hosted on the same hardware;

• An ongoing reduction in hardware maintenance costs; • Reducing costs for cooling, electricity and rack space in the data centre.

Cost avoidance will be achieved by:

• Faster deployment of new applications via virtual machines instead of having to purchase a new server;

• Reduction of planned downtime, since virtual machines can be easily moved to other hardware instead of being reinstalled and configured from scratch on new hardware;

• More affordable disaster recovery scenarios since virtual machines can easily be moved to other hardware.

2.5 Virtualization Products There is a wide range of platform virtualization products available on the market. This list contains exotic products such as Denali or Bochs. In this section I will give a more technical overview of a few of these products. I will focus on two products that are commonly encountered in large corporate environments, namely IBM virtualization and VMware ESX server. A more detailed description helps an auditor or risk assessor in identifying areas of investigations during an assessment. I will focus in the next paragraphs of this chapter on describing how the isolation between virtual machines and the abstraction from the hardware is being performed.


12

2.5.1 IBM virtualization products Currently IBM has virtualization software available for the following hardware environments:

• z-series (mainframe); • i-series (as400); • p-series (rs6000).

The z-series are IBM's mainframes, which run operating systems such as z/OS, z/VM, VSE/ESA and Linux. The i-series is the midrange product running operating systems such as OS/400 and Linux. The p-series runs AIX and Linux operating systems. The i-, p- and z-series all use Logical Partitioning (LPAR) technology while the z-series also has z/VM.

2.5.1.1 IBM LPAR for i and p-series Logical Partitioning (LPAR) is an IBM product. It was originally developed for mainframe computers. The LPAR technology is based on paravirtualization. Virtual machines are referred to as logical partitions. In this paragraph I will focus on the LPAR technology for i – and p series. In an LPAR configuration, individual processor(s) or parts of, memory, and I/O resources are assigned to one or more logical partitions. The more recent Dynamic LPAR (DLPAR) extends the capabilities by allowing resource allocation to take place while the logical partitions are running. Individual processors, memory and I/O resources can be:

• Released into a free pool; • Acquired from that free pool; • Moved directly between logical partitions.

Dynamic LPAR improves operational efficiency because computing resources can be moved between logical partitions at moments of peak usage. The hypervisor for LPAR resides in a firmware image and is stored in a system flash module in the server hardware. At boot up of the system the hypervisor is loaded in the first physical memory block and gathers information on the available processors, memory and I/O resources in the system. The hypervisor sets up the logical partitions and divides the resources between the logical systems based on boundary information stored in LPAR configuration tables that are stored in non-volatile RAM.


13

Figure 4, IBM LPAR overview The hypervisor program is the only program that can directly access processor registers and translation table entries. Programs running inside LPARs are not able to access the processor registers, other than through hypervisor service calls [5]: “When the operating system needs to create a page translation mapping, it must execute a specially architected hypervisor service call instruction on one of its processors, which transfer execution to a hypervisor program.” These hypervisor service calls indicate that the guest operating system is “virtualization aware” and that this is an example of paravirtualization. An attempt by LPAR programs to access a processor address directly results in an addressing exception interrupt for the operating system. The LPARs’ configuration tables are managed through the Hardware Management Console (HMC) graphical user interface software. HMC is also required for activating a powered down partition and for dynamic LPAR actions The HMC has several predefined roles:

• Super administrator; • Service representative; • Operator; • Product engineer; • Viewer.

Accounts and passwords are managed by the HMC super administrator role. Methods of access to the HMC are:

• Local login; • Remote login using the inbuilt web server (with optional SSL encryption); • Remote command line access via rexec or SSH.


14

The HMC is only able to access the LPAR configuration tables and does not have access to the LPARs themselves or data within them. For each logical partition there is also partition-specific firmware. This takes care of all firmware specific activities for that partition. It locates the operating system image on the logical partition and loads the boot image at start up. It also generates the device tree of specific hardware resources assigned to the partition. Each device tree only has devices assigned to that partition. The partition-specific firmware presents, via special services, the hardware resources to the operating system. These services are called Run-Time Abstraction Services (RTAS) in i and p – series. The result is that the hardware is presented in a uniform way to the operating system and is independent from the actual hardware. This allows changes to the hardware without the need to modify the operating system. LPAR allows the use of two types of partitions: dedicated or shared partitions. When one or more processors are dedicated to a single logical partition, this is called a dedicated partition. Shared partition is used when one or more processors are shared between multiple logical partitions*. These processors are also referred to as a virtual processor. The shared logical partitions can run in capped or uncapped mode. When a shared partition runs in uncapped mode and is at the maximum of its assigned processor capacity, then it can make use of idle processor capacity from other shared partitions. The unused capacity of these partitions is assigned to the shared pool. Capped partitions cannot make use of idle processor capacity from other partitions. Dedicated partitions are capped by default. The hypervisor controls all access to the real system memory. It translates the system calls from the logical partitions via the hypercalls of the operating system. In order for the hypervisor to manage this translation is uses several registers. These registers are extensions to the processor:

• Logical Partition Identity (LPI) register. This register contains a value that indicates to which partition the processor is assigned. All processors assigned to the same partition have the same value.

• Real mode offset (RMO) register. The RMO takes care of the mapping between the physical and logical memory of a partition. The entire physical memory is shared among all partitions and the hypervisor resides in physical address zero. The RMO translates the real physical address to correspond to the beginning of the logical memory address assigned to a logical partition (logical address 0). The RMO address is set when the partition is activated and the processor adds this address to each logical address interaction made by the programs in the logical partition.

• Real mode limit (RML) register. This register contains the size of the memory that a partition can access in real mode.

• Machine State Register. The Hypervisor State bit is set in the Machine State Register when the processor is executing Hypervisor instructions.

* Processor capacity can be assigned per 0.1 processor. It is possible to assign 1 or more virtual processors


15

2.5.1.2 IBM z-series and z/VM For z-series IBM provides the LPAR technology for logical partitioning and z/VM for software partitioning. The current version of z/VM was released in 2000. Z/VM can run the following operating systems: OS/90, Linux, OS/390, z/OS, and Transaction Processing Facility (TPF). An IBM z-series mainframe can run into in two architecture modes: “basic” mode and “LPAR” mode. In basic mode it runs as one physical machine and in LPAR mode virtualization is enabled. The z - series hypervisor uses processor architecture extensions to virtualize the hardware for each logical partition so that an operating system that runs in a logical partition without any changes. The z - series also introduced the concept of optional hypervisor calls that enable a hypervisor-aware version of an operating system to improve the utilization of the system resources by interacting directly with the hypervisor [39]. In LPAR mode the Processor Resource/Systems Manager (PR/SM) enables the dividing of the processor and I/O resources of the z-series. The PR/SM is the hypervisor and is a microcode that runs from firmware. At boot up of the system (Initial Micro Program Load the hypervisor is loaded in the first physical memory block and gathers information on the available processors, memory and I/O resources in the system. The hypervisor sets up the logical partitions and divides the resources between the logical systems based on boundary information stored in LPAR configuration tables (Input-Output Configuration Dataset.

Figure 5, IBM LPAR and z/VM The management of virtual machines within the PR/SM hypervisor is as described in the previous section performed through the Hardware Management Console (HMC). Part of the PR/SM is an interpretive execution facility which allows the virtual machines computing pages to be run on the processor. These pages are sent in a stream using a


16

single instruction called “Start Interpretive Execution” (SIE). The LPARs use the SIE instruction to divide a z-series processor complex into secure logical partitions. z/VM is the next layer on top of the LPAR virtualization. Z/VM allows the running of thousands of virtual machines on a z-series. The z/VM is comprised of several modules, the following two being the most important:

• The Control Program (CP). This is the hypervisor; • The Conversational Monitor System (CMS). The CMS is a single user

operating system.

The Control Program manages the resources of the underlying system and allocates the processors to the virtual machines. As the hypervisor it is in control of the underlying hardware and simulates the hardware for the virtual machines. When resources are shared the SIE instructions from one particular virtual machine are run until the machine’s time slot is consumed or until the virtual machine wants to perform a privileged operation. This privileged operation must be run by the CP who then takes control of the resources and puts the virtual machine in a wait state. After performing the privileged instruction the CP schedules the virtual machine’s next instruction. When a user logs on to z/VM the Control Program creates a separate guest virtual machine for each user. The Control Program then allows the user to start up an operating system within that virtual machine: Linux, CMS, z/OS, MVS or VSE. Access to z/VM is controlled via a user id and associated password. As stated above the CMS is a single user operating system. The system administrators use CMS to manage z/VM itself and the guest operating systems (i.e. the virtual machines). CMS runs on top of the CP and is terminated once another operating system like Linux or z/OS is started. Memory Management z/VM provides memory virtualization to the virtual machines via memory address translation. This translation is done through tables which maps the virtual memory pages to the real memory pages. The CP is responsible for maintaining these pages, so a virtual machine is unable to read tables belonging to other virtual machines or the CP. With z/VM two address translation tables are being used. Each virtual machine creates its own table which maps the virtual machine’s physical memory to its virtual memory. The virtual machine’s physical memory addresses relate to the host’s or hypervisor’s virtual addresses. Therefore the CP maps its virtual addresses to its physical addresses. This guarantees that even if a virtual machine manages to break out of its virtual addresses it cannot influence the total integrity of the CP and other virtual machines.

2.5.2 VMware ESX server VMware has a range of virtualization products. In this paper I will focus on the ESX server product as this is, based on my experience, the most commonly used VMware product in corporate datacenters.


17

ESX server is an operating system on its own as opposed to other Windows virtualization programs such as VMware Workstation or Microsoft Virtual Server. These products run as software package within the Windows operating system on the host. As a result these products don’t interact and don’t directly control the hardware resources. Based on the Popek and Goldberg paper these products cannot be considered as virtualization software since they don’t meet the criteria in rule 2: Resource Control. ESX is a full virtualization product but has some aspects of paravirtualization. ESX server makes use of software programs such as the VMware tools and virtual device drivers, such as Vmxnet. These tools provide better performance and do make a change to the guest operating system, but not to the guest operating system kernel [40]. ESX is made up of two core components:

• The ESX Server kernel, also known as the VMkernel; • The Console Operating System (COS), also known as VMnix.

VMkernel is the hypervisor or VMM software. It manages access to specific hardware resources on the hardware but allows direct hardware access to the processor, hyperthreading, symmetrical multi-processing and some part of the memory. These four resources are presented to the guest operating systems as is. The VMkernel schedules CPU time and access to memory, storage and virtual switches. The VMkernel sits on a Linux-based kernel and contains VMware proprietary code. VMnix is a customized Linux kernel-based operating system, optimized for running virtual machines. The computer boots from the VMnix and during the startup several scripts are initialized, preparing the system for virtualization. These scripts also load the VMkernel and contain configuration parameters. At the moment that VMnix is fully loaded it passes the resource control and the management of virtual machines over to the VMkernel. The VMkernel takes the hardware over via a soft reboot and becomes the primary operating system. The VMnix is then started as a virtual machine. ESX has role-based user access. Predefined roles are:

• No access user; • Read only user; • Administrator; • Virtual machine user; • Virtual machine power user; • Resource pool admin; • Datacenter admin.

The VMnix or COS provide the following management services:

• Authentication of ESX users; • Access via SSH, a web based management console, and FTP; • Configuration of the virtualization parameters and of the COS itself; • Support programs such as hardware agents and back-up programs.


18

ESX uses three types of hardware resource: virtual, console, or shared. Virtual resources can only be accessed by the VMkernel (on behalf of the guest OSs), console resources are limited to the COS, and shared resources can be accessed by the COS and the VMkernel.

2.5.2.1 Memory management As already noted before ESX does allow direct memory access, but that is within certain boundaries. The COS or VMnix is assigned a fixed amount of the available memory. This is used for running the COS but each virtual machine that is started also requires memory space within the COS to support the virtualization. The remaining memory is available for the virtual machines. ESX server manipulates this memory on different ways. The first way is the virtualization of the memory itself. The second way is to make efficient use of the memory.

Memory virtualization The VMkernel intercepts the memory pages from the virtual machines and translates them. Each virtual machine expects the memory address space to start at address zero. VMkernel gives this illusion by doing memory address translation. For this translation VMkernel makes use of several tables. The first is a data structure called a pmap. In the pmap the physical machine’s page numbers (MPNs) are mapped against the physical page numbers (PPNs). In this case the use of ‘physical’ does not relate to the real physical memory. This is because the MPNs are the added layer. This pmap table is used by the VMkernel. The VMkernel maps the PPNs to virtual page numbers (VPNs) via so-called guest tables. These VPNs are presented to the guest operating systems (i.e. the virtual machines). ESX also makes use of shadow page tables. These tables map the VPNs directly to the MPNs.


19

Memory optimalization ESX uses four ways to make efficient use of the available memory:

• Ballooning; • Transparent page sharing; • Swapping; • Idle memory tax.

These techniques make it possible to assign more memory to the total of virtual machines than is actually available on the physical machine. Via a special ballooning driver the guest operating system can dynamically increase or decrease its memory usage. The ballooning driver communicates with the VMkernel. When the memory pressure of the guest operating system increases the memory balloon is also increased. If the memory pressure decreases the memory balloon is decreased. The driver does this by forcing the guest operating system to place memory pages in the local swap file. The free memory can then be allocated to other virtual machines. All freed memory is zeroed out before it is being used by another virtual machine. Transparent page sharing allows memory sharing between virtual machines. The method used by ESX is transparent for the virtual machines. With transparent page sharing ESX server calculates the hash value of very memory page. When there is already a page with the same hash value resident in memory, VMkernel will map to that already existing page instead of loading that memory page. This is a very efficient way of sharing memory between virtual machines since the different operating systems and applications in the running virtual machines make use of static and common memory pages. Transparent page sharing even takes place when ESX is not running on maximum memory usage. It can be disabled per individual virtual machine. When one virtual machine modifies a shared memory page, a copy of the memory page is made and assigned to that virtual machine. ESX has an independent swap file which is not being used by the guest operating systems. This file is independent of the swap file in the virtual machines of the Console Operating System. When ESX is running with maximum memory usage it will determine which memory parts from the virtual machines are less used and page these to the swap file. This paging consumes CPU cycles and can be turned off. In ESX each virtual machine is allocated a certain amount of memory. Idle Memory Tax is a method to free unused memory from virtual machines and allocate it to machines in greater need of memory. Based on the priority of a virtual machine a cost is associated with the unused memory. The priority is set through the management console.


20

2.5.3 Virtual networking Virtual networking is not exactly platform virtualization. It is the virtualization of a network and its components. However both ESX server and the IBM virtualization products offer the functionality to set up virtualized network on top of the hypervisor.

2.5.3.1 ESX server (VMswitch) ESX also incorporates virtual switch technology in its products. A virtual switch is like a physical switch; it lets you connect other networking components together. Virtual switches can be bound to one or multiple physical network adapters at the same time. It is also possible to create a virtual network by not bonding a virtual switch to a physical network adapter. This technology is referred to as VMnet. This allows creation of a private virtual network that is visible only to other hosts configured on the same VMnet on the same physical host. The network traffic between virtual machines on the same virtual switch is transferred across the system bus instead of forwarded to the network infrastructure.

Figure 6, ESX virtual switch


21

2.5.3.2 IBM Hipersockets HiperSocket is a technology for providing in-memory TCP/IP connections between LPARs on a z-series mainframe. These connections will not travel on a network but will use connections in the memory i.e. they will not leave the mainframe environment. As a result these connections work at memory speed. This results in higher performance and higher availability. Also, it is not possible to intercept these connections since they are protected by the kernel. Virtual Ethernet For the i-series a comparable technology is available, called Virtual Ethernet. Virtual Ethernet also enables direct TCP/IP connections between logical partitions. LPARs assigned to the same virtual Ethernet can communicate through that link like with a physical VLAN. VIO server The VIO (Virtualised Input/Output) Server is functionality for IBM p-series. It works in conjunction with the hypervisor and provides virtual to physical mapping of SCSI and Ethernet devices. Enabling communication across a virtual internal Ethernet maintained by the hypervisor.


22

3 Risk Assessment As virtualization technology adds an extra layer between the operating system and the hardware, this will most definitely introduce new risks. A risk assessment helps to make a proper judgment as to whether virtualization is a suitable solution from a security point of view. The risk assessment is just a step in the risk management process. Based on the outcome of the risk assessment one can decide whether to accept or mitigate the identified risks or even decide against the implementation. The risk assessment should not only focus on the risks that can be identified for an operational device, but on all phases of a project, from the acquisition phase to implementation and ongoing support. Furthermore, not all applications are suitable for virtualization, especially applications that have a high I/O demand. The suitability of these applications for virtualization must be investigated. This process starts with investigating how much resources an application uses, and at which times. In this chapter I will investigate the risks that are most commonly associated with each layer of virtualization. The following layers are in scope for this risk assessment review:

• Hardware; • Hypervisor; • Network; • Virtual machines; • Management Consoles; • Storage.

I will not look in detail at risks that relate to guest operating system hardening.

3.1 Hardware The hardware is the actual physical component on which the virtualization software and virtual machines run. The hardware must be able to support the virtualization requirements; memory, storage, network and processor requirements for all applications and virtual machines must be known during the acquisition phase. These requirements must also constantly be monitored during the operational phase or during changes. Also, the availability requirements for the applications that run in the virtual machines greatly influence the choice of hardware. If an application with high availability requirements is virtualized these requirements must be taken into consideration and must be reflected. Running such an application from non redundant hardware will introduce a single point of failure. This can endanger the requirements to meet recovery time objectives during a hardware failure.


23

Risk Risk description

Capacity management is insufficient

The capacity requirements of a virtual machine are not considered when deploying a new virtual machine to an already-existing host system. The resources allocated to a newly deployed virtual machine exceed the available resources, either by itself or when running concurrently with other virtual machines. The resources include the following areas:

• Memory • Processor capacity • Disk space • Network throughput

Capacity shortages (in peak times) on already-existing hardware are not detected. This can result in a shortage of memory, disk space or processor requirements for running virtual machines. This can be especially the case when virtual machines are frequently moved between servers or if new virtual machines are added. As a result applications can crash and or suffer performance degradation.

Attacks on firmware

The code of the firmware-based hypervisor is the subject of a physical attack, resulting in the system not able to boot and start the hypervisor The firmware is replaced with a rogue or unofficial version. This rogue version can contain harmful software code. This can lead to a compromise of the separation of VMs provided by the hypervisor.

Single point of failure

Critical hardware components are not redundantly implemented. In the case of a hardware failure this could lead to the unavailability of all virtual machines hosted on the affected host.

Lack of support The hardware components are not supported by the virtualization software. As a result the virtualization software cannot be installed or not all expected functionality is present. Also this can lead to stability issues. With the chance that these only materialize during the production phase or if virtual machines are moved to other hardware.


24

3.2 Hypervisor/Kernel The hypervisor or virtual machine monitor is the layer between the virtual machines and the hardware. It controls the hardware resources such as memory, processor, network interfaces and storage. Breaking out of the hypervisor is considered the biggest risk for virtualization. If an application in a virtual machine breaks the boundaries of the hypervisor it can access and/or manipulate hardware resources belong to other virtual machines. At the moment of writing this paper this is still a theoretical risk. At the moment this risk materializes the impact will be enormous on the virtualization. On the other side it is to be expected that the virtualization vendors will provide patched hypervisor as soon as possible. The hypervisor itself can introduce a single point of failure. See also figure 2. The hardware underneath the hypervisor can be redundant and clustered. If one hypervisor is running and it fails; all virtual machines are affected.


Memory exploitation

Memory is shared between different virtual machines. If the memory is not protected very well it is possible that virtual machines can change data from other virtual machines. Memory in use or used by a virtual machine can be read by another virtual machine.

Denial of Service

Shared resources are completely claimed by one virtual machine or process and are not freed on time or on request. This can lead to unavailability issues for other virtual machines within the same hypervisor which make use of the same shared resources.

Buffer overflows

Virtual machines are able to break out of the control the hypervisor and can access resources allocated to other virtual machines or to the hypervisor.


The virtualization layer is a single point of failure for all virtual machines that run on it.


25

3.3 Network The network layer is the virtual network layer. On the virtual layer the virtual machines communicate to each other via virtual network adapter and virtual switches. These virtual networks must be treated as normal networks. Risks can be that network traffic on one physical network adapter can be read by all virtual machines. Also segmentation of the network must be taken into consideration.


Information leakage

Multiple virtual machines on the same hardware make use of the same physical network adapter and the network traffic is accessible by all virtual machines. As a result a privileged user on one of the virtual machines may have access to the network data belonging to another virtual machine. In that case it is possible to read potentially-confidential data.

Segmentation All the virtual machines belong to the same network segment and are attached to the same (virtual switch). The user from one virtual machine is able to connect to all virtual machines on the same host. Especially when one of the virtual machines is connected to the internet this is a big risk. The (virtual) networks are not labeled with a proper label indicating the sensitivity. As a result trusted virtual machines may be attached to untrusted or less trusted network segments.

No security monitoring

No intrusion detection or intrusion prevention is performed on inter-virtual machine traffic. This traffic never leaves the host system and therefore never flows through an IDS or firewall. Security incidents or malicious traffic may not be detected in this scenario.

3.4 Virtual Machines The virtual machines are the guest operating systems that run on top of the hypervisor. The different virtual machines are separated from each other by the hypervisor. The hypervisor controls access to system resources for these virtual machines. The virtual machines must be treated as normal operating systems. Traditional IT general controls such as logical access, physical access, patch management; security monitoring and vulnerability management must also be put in place for the virtual machines. The same IT controls that apply for a traditional operating system within a company also apply for the virtual machines. When implementing these controls special attention must be paid to the special characteristics of virtualization. For instance the patch management procedure must


26

take into account that there can be multiple copies of the same virtual machine on the network. Most probably one virtual machine is active while the others are halted. Also these halted virtual machines must be patched. Virtual machines are very flexible and can be moved around the datacenter to meet resource requirements. Moving the virtual machines can break interfaces to other systems. It can also be the case that virtual machines are deployed to hardware which does not meet the CIA requirements for the virtual machine i.e. a data classification scheme is not followed or not implemented. The same can happen on a traditional server. However I think that the likelihood of the happening in a virtualized environment is higher since if the infrastructure is there and it is easy to roll out a new virtual machine instance. The chance that for the existing hardware the CIA requirements are not reconsidered is higher. The management of virtual machines is crucial. It is very easy to roll out virtual machines in a network compared to the roll out of physical machines. This also means that it is also easier to accidentally implement rogue virtual machines on the network. Data storage can also be done on the virtual hard disk of the virtual machine. In case of a disaster this data is not accessible for a copy of the virtual machine that is started up on the disaster recovery site. This risk also exists for traditional computers and is not very specific for virtual machines. However I think that with virtualization this risk is more present since the virtual machine is a container and people are probably more inclined to store the data on the same machine. This allows easy moving of the virtual machine.


System security is insufficient

The deployed virtual machines are not considered and managed as normally managed operating systems and as a result traditional IT controls are not implemented. Examples but not limited to:

• Hardening • Anti virus software.

Data classification not followed

Virtual machines with a higher or more critical classification are deployed to hypervisor or hardware with a lower classification. As a result requirements for availability, confidentiality and integrity are not met. Applications with a higher classification are being run from a virtual machine that is configured according lower classification guidelines.

Management complexity

There is no clear overview on which hardware devices the virtual machines are running. Therefore troubleshooting of incidents is more difficult or making decisions about capacity are not based on the right requirements. There is no clear overview which virtual machines are inter-connected i.e. share virtual networks. Interfaces are not documented. Moving a virtual machine to other hardware or even another data centre can break these


27

connections. Information disclosure

In the virtual machine unneeded devices, such as USB, drives are activated. This allows USB media to be attached and classified information to be copied off the system or malicious software to be copied on the system.

Segregation of environments is not in place

Virtual machines that are used for development, testing, acceptance and production are run on the same hardware. Users on development can have privileged rights and therefore run programs that can break the virtualization isolation.

Insufficient vulnerability management

Multiple copies of virtual machines can exist on the network and in different locations. Only one copy is running while the others are halted. The halted virtual machines do not get patched.

Data storage on system

Data storage is done on the virtual hard disk of the virtual machine. In case of a disaster this data is not accessible for a copy of the virtual machine that is started up on the disaster recovery site.

Buggy support software

Special virtual machine support software (e.g. VMware tools) is installed in the guest operating system and there being bugs in it that allow virtualization controls to be bypassed.

Security policies do not meet requirements

Virtual machines are moved to another host system or to another physical location. This new location does provide a lower level of security.

3.5 Management Consoles The Management Consoles are the software programs that are used to configure the virtual machines of the host system. Via the Management Consoles it is possible to create, delete, start, stop and change configuration settings. Configuration settings can influence the allocated memory, processor capacity which can influence the availability of virtual machines. It is also possible to change settings in the hypervisor that enable sniffing of traffic on network interfaces. This can lead to confidentiality risks. Management console access can take place via the console command-line, remote console, web server user interface or management clients. As with any remote management of a system, no vulnerable protocols should be used to access the management console. By preference the management activities should be done out of band. The connection from the workstation to the management console can be intercepted when this connection is not encrypted and routed over the normal network. The logical and physical access to the management console is crucial and should be protected well. User-id’s and passwords should be used, and role-based access should be in place when available. The use of generic privileged accounts should be restricted and controlled.


28

System administrators should receive proper training for their daily work with management console. Since the hypervisor is administered via the management console all parameters are stored here. To enable a swift recovery in case of a disaster all these settings must be backed up regularly or the management console should be redundant.


System security is insufficient

The management consoles are not considered and managed as normally managed operating systems and as a result traditional IT controls are not implemented. Examples, but not limited to:

• Hardening; • Generic accounts; • Monitoring; • User and access administration; • Training for administrators; • Redundancy • Backup.

Unauthorized changes to configuration

Key configuration files are not protected with sufficient access rights. Changes are being made by development staff and resources needed for production are consumed by development or testing. Rogue virtual machines are created and started. Resources needed for production are consumed by development or testing.

External system attacks

The management console is connected to an untrusted network segment. This allows malicious users to exploit vulnerabilities against the management console and thus gaining access.

Physical access

The management console is run from a separate server or personal computer. This machine is located outside a datacenter. Attackers can get physical access to this machine and as a result the whole virtualized environment is compromised.

3.6 Storage The storage for platform virtualization is also something that must be considered when performing a risk assessment of a virtualized environment. The virtual machines themselves are contained in image files which are stored on the host system. But it is also very common that the virtual machines mount partitions from a SAN environment. While it is hard to steal a physical machine from a data center unnoticed it is easier to steal a virtual machine. First of all, how are you going to notice that a virtual machine is


29

stolen? Most probably the original image is still there and a copy has been made. The likelihood of this risk increases when there are multiple copies from a virtual machine. With virtual machines being containers it is possible to look from the outside into the container. The virtual machine is not started but the container file is just opened via an external program. The logical access controls in the virtual machine are circumvented and sensitive information can be exposed to unauthorized personnel. The impact of this risk depends on the fact if application data is stored within the virtual machine. As a best practice, see also section 3.4, this should not be the case. Proper access controls on the file system level must be in place to protect the virtual machine’s underlying image file against theft or unauthorized access.


Access control is insufficient

The virtual machine image files are not protected with correct access rights. As a result unauthorized users can see or change sensitive data within the virtual machine container file. Theft of a complete virtual machine. A complete virtual machine container file can be copied. The thief not only has possible sensitive data on the virtual machine but also the needed operating system and software to run the application.


The connection to the storage devices for all virtual machines is done via one connection.


30

4 Traditional IT controls In this chapter I will investigate if traditional IT controls can be copied one on one from a traditional pillar environment to a virtualization environment. In the situation that this is not the case I will provide recommendation to mitigate these risks. The central question for this chapter is: What will change to the IT control framework when the layer Virtualization is introduced? Since the size of this paper is limited I selected from the ISO 27001 standard the following IT controls: ISO ref ISO section name Objective

A.10.1.4 Separation of development, test and operational facilities

Development, test and production facilities shall be separated to reduce the risks of unauthorized access or changes to the operational system.

A.7.2.2 Information labeling and handling

An appropriate set of procedures for information labeling and handling shall be developed and implemented in accordance with the classification scheme adopted by the organization.

A.10.10.2 Monitoring system use Procedures for monitoring use of information processing facilities shall be established and the results of the monitoring activities reviewed regularly.

A.12.6 Technical Vulnerability Management

Timely information about technical vulnerabilities of information systems being used shall be obtained, the organization's exposure to such vulnerabilities evaluated, and appropriate measures taken to address the associated risk.

4.1 Asset Classification The asset classification or CIA rating is important in the information security. With the asset classification I refer to the criticality label that is assigned to a program of device by the owner. The asset classification influences the impact of a risk. For an application where the confidentiality is rated as high or critical the rating of a risk associated with a breach in the confidentiality will be higher than for an application that is rated as non sensitive. Deciding on which IT controls to implement to mitigate risks identified during a risk assessment is also very hard. Without an asset classification the impact of a risk is not known and therefore the cost of the IT controls can not be measured against the potential financial damages.


31

The asset classification or CIA rating for an application must be decided upon by the business owner. The business owner is the only one who can decide on the impact of breaches against the confidentiality, integrity or availability and who knows exactly what kind of data is used in the application. Based on the asset classification the proper IT controls can be implemented. Compared to the traditional pillar with virtualization there is no direct link between the application/operating system and the underlying hardware.

Figure 7, CIA ratings

4.2 Type I or type II? The first choice that must be made is between a type I or a type II hypervisor. One of the most heard comments on virtualization from a security point of view is that an extra layer is an added to the system. Therefore the chance of an incident happening is increased. By adding the extra layer for virtualization one also adds an extra layer that can be attacked [41]. A type II hypervisor runs on top of a guest operating system. As a result the hypervisor depends on this operating system for security.


32

Also, a type II hypervisor runs in ring 3. This increases the chance that an attacker is able to break out of the hypervisor. An example of a type II hypervisor is VMware Workstation. Recently a vulnerability was found in VMware Workstation (CVE-2008-0923). This vulnerability allowed attackers to escape the guest operating system and modify files to the underlying host operating system. For production environments I recommend only the use of a type I hypervisor. The risks associated with a type II hypervisor are likely to be too high. A type II hypervisor runs as an application on the host operating system (ring 3). A risk assessment is needed to decide between the implementation of a type I or II hypervisor. See also the next section.

4.3 Segregated Environments Based on ISO 27001 control A.10.1.4 a separation of environments must be in place between development, test and production environments. When separation is not in place there is the risk of unauthorized changes to production data. In such a scenario it is possible that a developer promotes an unapproved version to production. This can lead to financial losses due to fraud or unavailability. The control speaks of a separation between development, test and production. In a lot of large corporate environments an additional acceptance environment is added to this environment. The whole of environments is also known as a DTAP (development, test, acceptance and production). One of the business drivers for implementing virtualization is to efficiently make use of the hardware resources by combining the development, test and production environment on the same hardware. The hypervisor provides isolation between the virtual machines through the second formal requirement from Popek and Goldberg [1]. Running different virtual machines for development, test and production on the same hypervisor and on the same physical hardware would mean that the ISO control for segregated environments is met. The virtual machines are isolated from each other via the hypervisor. However developers and testers often have administrative rights on the systems they use for their activities or they can gain access by using programming facilities. They can run developments programs on a system and also perform debugging activities. This also gives them the opportunity to run harmful software. Due to bugs in the program, system resources can be consumed or changed accidentally or on purpose. Within a virtualized environment they very well can also have the ability to access the management console of the hypervisor, so that they can add, start, and stop virtual machines or change resources. These possibilities pose risk against the production being run on the same system.


33

Based on the risk assessment the following risks were identified that are related to the segregated environments: Layer Risk Risk description

Management console

Unauthorized changes to configuration

Key configuration files are not protected with sufficient access rights. Changes are being made by development staff and resources needed for production are consumed by development or testing. Rogue virtual machines are created and started. Resources needed for production are consumed by development or testing.

Virtual machines

Segregation of environments is not in place

Virtual machines that are used for development, testing, acceptance and production are run on the same hardware.

Hypervisor Memory exploitation Memory is used by different virtual machines. If not protected well virtual machines can change data from other virtual machines. Memory in use or used by a virtual machine can be read by another virtual machine.

Denial of service Shared resources are completely claimed by one virtual machine or process and not freed on time or on request. This can lead to unavailability issues for other virtual machines within the same hypervisor of which make use of the same shared resources.

Buffer overflows Virtual machines are able to break out of the control of the hypervisor and can access resources allocated to other virtual machines or to the hypervisor.

Based on the above it can be concluded that having different virtual machines for development, test and production on same physical hardware doesn’t mean that the ISO control for segregated environments is met. Additional controls must be put in place to mitigate the risks further. There is a difference between the DTAP environment for applications/database management systems and the infrastructure layer (operating system/hardware). Development, testing and acceptance for applications or database management systems should ideally only be performed on production infrastructure. If these activities are performed on hardware or operating systems that are not fully accepted there are too much dependencies for the application development live cycle. Bugs on the infrastructure layer can influence the application development. For the infrastructure layer the most secure solution is also to implement a DTAP environment.


34

The DTAP environment in a traditional environment would ideally look like this:

Fig 8 DTAP traditional .

Based on the risk assessment, the business value of the applications, and a cost benefit analysis it must be decided whether it is acceptable to have complete separate hardware for all phases or whether phases can be combined on the same hardware. Still it remains very questionable if development, testing, acceptance and production can be combined on the same host for critical systems. For development and testing it can be necessary to turn on memory settings such as transparent page sharing or memory swapping. These settings can influence the availability of the production. This may be acceptable for non sensitive systems. To meet the ISO control for segregated environments the hardware should be split between the different phases. On the hardware layer new hardware could step through the phases and promote from development, via test and acceptance to production. Once the hardware is labeled as production it can be used for development and testing on the application layer. For the acceptance and production on the application layer separate hardware should be available. Based on this scenario the absolute minimum is two separate hardware environments. Ideally in a virtualized environment the DTAP would look like this†:

Fig 9 DTAP virtualization † In the most ideal situation three separate physical environments should be used for virtualization. However given the mainframes this can be a costly choice.


35

If a type II hypervisor is available for the hardware and operating system that is in use in the company, it can also be considered to deploy the development and test environments via a type II hypervisor. In that case it is possible to do development from the workstation of the developer or on less costly hardware.

4.4 Information labeling and handling Based on ISO 27001 control A.7.2.2, an appropriate set of procedures for information labeling and handling should be developed and implemented in accordance with the classification scheme adopted by the organization. When such a procedure is not in place or not followed there is the chance that confidential information is transported via an insecure channel resulting in information disclosure. When deploying a new application, the application will receive the asset classification made by the business owner and based on the information processed by the system. In the traditional pillar it is quite common that new hardware is bought to deploy the new application. The asset classification gives guidance in choosing the right equipment. For an application with high availability requirements it is very obvious that the equipment will have redundancy features. When deploying through virtualization there is a good chance that the hardware is already available within the company so the application can easily be deployed via starting a new virtual machine. This is especially if the resources on the host system are sufficient for running the new virtual machine. With virtualization the direct link between the application and the hardware is less visible as a result of the added layer. As a result there is the chance that this will happen without considering if the supporting host system meets the requirements related to the CIA rating. Deploying a new virtual machine to a host system that is labeled as suitable for a higher classification is no issue from a security point of view. But deploying a virtual machine to a host that has a lower CIA rating is a risk. Technically this is a security issue because you are overprotecting something which doesn’t need it. To mitigate this risk a procedure must be developed and implemented which enforces that for virtualized applications the classification of the application and the host system are aligned. Virtual machines must only be allowed to be deployed on a host system that has the same or higher CIA rating.

4.5 Monitoring system use Based on ISO 27001 control A10.10.2, procedures for monitoring use of information processing facilities should be established and the results of the monitoring activities reviewed regularly. In this paragraph I want to focus on monitoring of inter virtual machine traffic. The virtualization products from VMware and IBM allow the creation of a virtual network. Within these virtual networks the virtual machines on the same host can communicate without the need for the data packets to travel on the physical network.


36

In this situation intrusion detection sensors of firewall on the physical network won’t pick up on an attack that is staged between two virtual machines on the same host, since the data doesn’t travel on the physical network. Therefore the attack can go by unnoticed. In the risk assessment the following associated risk was identified: Layer Risk Risk description

Network No security monitoring

No intrusion detection or intrusion prevention is performed on inter-virtual machine traffic. This traffic never leaves the host system and therefore never flows through an IDS or firewall. Security incidents or malicious traffic may not be detected in this scenario.

The following controls can be put into place to mitigate this risk:

• Install host-based intrusion detection software on every virtual machine. Let all the sensors report into a centralized logging facility;

• Virtualize a security device such as a firewall or an inline intrusion detection system on the same host platform. Force all inter virtual machine traffic to flow through this virtualized appliance;

• Enable firewalls on all virtual machine in order to reduce the attack window on certain host. Also, deny all traffic between virtual machines that have no need to communicate to each other;

• If available install specialized 3rd party security software that installs between the hypervisor and the virtual machines and intercepts all security incidents.

• An assessment should be made which virtual machines are allowed to communicate with each other. Virtual machines that are not allowed can be moved to a different host system or host firewall activated to block this traffic.

Based on the asset classification of the applications on the host system a choice of one or more of the above mitigating controls can be made. For a non-sensitive application it might be considered sufficient to enable the operating system’s built-in firewall, while one might consider acquiring special 3rd party software for mission critical applications. With VMware opening the virtual switch technology and allowing third party vendors such as Cisco do develop virtual switches/routers I expect new virtual security devices the market in the near future.

4.6 Control of technical vulnerabilities Based on ISO 27001 control A.12.6.1, timely information about technical vulnerabilities of information systems being used should be obtained, the organization's exposure to such vulnerabilities evaluated, and appropriate measures taken to address the associated risk. When there is no vulnerability management in place the organization will not be aware that there are vulnerabilities with application or operating systems. As a result patches preventing the exploitation of these vulnerabilities will not be deployed.


37

In a traditional environment a regular vulnerability scan and an inventory of servers with detailed information about the operating system and installed software is sufficient to identify vulnerable servers and apply the patches to the right machines. In a virtualized environment this will be much harder. There can be multiple copies of the same virtual machine at different places in the network. It could be very well that one copy is running and the others are halted. When applying patches according to the traditional way, only the active virtual machine will be patched. In the risk assessment the following associated risk was identified: Layer Risk Risk description

Virtual machines

Insufficient vulnerability management

Multiple copies of virtual machines can exist on the network and in different locations. Only one copy is running while the others are halted. The halted virtual machines do not get patched.

To mitigate this risk the patching procedure must be adapted to the following extent:

• The inventory must also indicate how many copies there are of a virtual machine and in what location these copies are stored. This also means that the duplication of virtual machines must be done along recorded guidelines.

• Each virtual machine must be started so that patches can be applied. This must be done in such a manner that running multiple copies of the same application of virtual machine doesn’t interfere with the business. Or copy the master copy of a patched virtual machine to the required locations.

• After patching the additional started copies must be halted. Another solution is to make use of 3rd party software that on start up verifies if the virtual machine is fully patched. If not patches will be applied. An example of such software is Virtualshield from Bluelane. VirtualShield runs on VMware ESX server and sits between the hypervisor and the virtual machines. The patching procedure should not only consider the software within the virtual machine, but also the management console and hypervisor that can contain vulnerabilities for which the vendor patches has issued. The hypervisor and management console must also be part of the vulnerability and patch management process.


38

5 Conclusion In this paper I investigated the challenge that virtualization posed to the traditional IT controls. My question was: What will change to the IT control framework when the layer Virtualization is introduced? I selected 4 ISO 27001 controls for more detailed investigation. The controls can be very well used, but adoption is needed to tailor them for virtualization. Separation of development, test and operational facilities Contrary to one of the business drivers for virtualization: “The ability for a company to combine all environments via virtualization on one piece of hardware”. I came to the conclusion that at least physical separation must exist between development/test and acceptance/production. One sub research question was: Can a Development, Test, Acceptance and Production (DTAP) environment be combined on the same hardware and to what extent? Not all Development, Testing, Acceptance and Production environments can be combined on the same hardware. Information labeling and handling When deploying virtualization the CIA rating for the application should be considered in order to prevent deploying virtual machines to hardware with a lower rating. One sub research question was: Can virtual machines with different Confidentiality, Integrity and Availability ratings (CIA ratings) be combined on the same hardware and under what conditions? Virtual machines with different CIA ratings can be combined on the same host system but only if the host system meets the CIA requirements of the most critical virtual machine. Monitoring system use Virtualization has the ability of inter virtual machine traffic. This traffic is not monitored via intrusion detection systems or firewall. One of the recommendations is to deploy virtualized security appliances, deploy host based firewall, or separate virtual machines that are not allowed to communicate. Technical Vulnerability Management The patch management must take into consideration that virtual machines are halted and therefore will not be patched.


39

Overall I think in general that with virtualization we are probably talking about the re-introduction of security techniques already well understood by the mainframe crowd (but never learned by the PC generation). Virtualization on itself can be used for all environments including production. However a risk assessment should be conducted when defining the requirements. Additional controls should be implemented to mitigate the identified risks. The decision to accept residual risks depends on the risk appetite of the business associated with the asset classification of the application involved. The decision for a company to mitigate the risks is part of the old battle between cost versus efficiency and profit versus security. Still I recommend to keep I mind that IT and security are a business enabler.


40

6 Appendix A (used sources) 1. Formal requirements for virtualizable third generation architectures.

Communications of the ACM, Gerald J. Popek and Robert Philip Goldberg, July 1974

2. A virtual machine directed approach to Trusted Computing, Haldar, Chandra and

Franz, 2004

3. Hardware Support for Efficient Virtualization, Ogden

4. LPAR For Decision Makers, IBM, 2002

5. Logical Partition Security in the IBM pSeries, IBM, 2002

6. IBM System p Advanced POWER Virtualization Best Practices, IBM, 2006

7. sHype: Secure hypervisor approach to trusted virtualized systems, Sailer, Valdez, Jaeger, Perz, Van Doorn, Griffin and Berger, 2006

8. LPAR for Power4 Security Target, IBM, 2003

9. Linux® on the Mainframe, Eisenhaendler, Mattheus and Salm, 2003

10. IBM eServer i5 and iseries Logiacal Partitioning FAQs, Dave and Disckey, 2004

11. VMwareESX Server: Advanced Technical Design Guide, Oglesby and Herold

12. How to Secure VMwareESX, Bakman, 2006

13. An Emperical Study into the Security Exposure to Hosts of Hostile Virtualized

Environments, Ormandy

14. Thwarting Virtual Machine Detection, Liston and Skoudis, 2006

15. Security Design of the VMware Infrastructure 3 Architecture, VMware

16. z/VM Virtualization Basics, Bitner, 2006

17. Memory Resource Management in VMware ESX Server, Walspurger, http://www.usenix.org/events/osdi02/tech/waldspurger/waldspurger_html/esx-mem-html.html

18. Public Version of the Security Target for PR/SM for the IBM eServer zseries

Z900 GA3 and z9800 EAL5 Certification, Common Criteria for Information Technology Security Evaluation, 2003


41

19. VMware Virtual Machine File System: Technical Overview and Best Practices, VMware, 2007

20. Analysis of the Intel Pentium’s Ability to Support a Secure Virtual Machine

Monitor, Irvine and Robin.

21. z/VM security and Integrity, IBM, 2005

22. Virtualisatie en IT-auditing, Monetro, 2007

23. Virtualization: the other side of the coin, Rutkowska, 2007

24. http://www.ibm.com

25. Breachable virtual machines?, http://www.c0t0d0s0.org/archives/3058-Breachable-virtual-machines.html

26. VMWare Escape Publicized at SANSfire 2007, http://www.foolmoon.net/cgi-

bin/blog/index.cgi?mode=viewone&blog=1185593255

27. Virtualisation could slash IT budgets, Gartner, 2004

28. Escaping From The Virtualization Cave, http://www.pauldotcom.com

29. Adam Nosfinger presentation: Operating System Virtualization

30. www.dell.com

31. www.microsoft.com

32. www.novell.com

33. Whatis.com

34. Wikipedia and Mann, Andi, Virtualization 101

35. A Survey on Virtualization Technologies, Susanta Nanda Tzi-cker Chiueh

36. Schawab, Demystifying Virtualization, 2006

37. Marshall, Reynolds and McCrory, Advanced Server Virtualization, 2006

38. Van Veen, Heliview seminar, 2006

39. W. J. Armstrong, Advanced virtualization capabilities of POWER5 systems

40. Understanding Full Virtualization, Paravirtualization and Hardware Assist, WMware.

41. Rush to Virtualization Can Weaken Security, Gartner, 2007


42

7 Definitions Virtualization allows you to run multiple applications and operating systems independently on a single server. [30] Microsoft virtualization solutions enable IT managers to run multiple operating systems, applications and middleware on a single physical machine, allowing customers to cut cost and response time out without scarifying resources. [31] Virtualization refers to the pooling of IT resources in a way that masks the physical nature and boundaries of those resources from resource users. In more concrete terms, virtualization is the decoupling of the software from the hardware. [32] Virtualization is the creation of a virtual (rather than actual) version of something, such as an operating system, a server, a storage device or network resources. [33] Virtualization is] a technique for hiding the physical characteristics of computing resources from the way in which other systems, applications, or end users interact with those resources. This includes making a single physical resource (such as a server, an operating system, an application, or storage device) appear to function as multiple logical resources; or it can include making multiple physical resources (such as storage devices or servers) appear as a single logical resource. [34] Virtualization is a technology that combines or divides computing resources to present one or many operating environments using methodologies like hardware and software partitioning or aggregation, partial or complete machine simulation, emulation, time-sharing, and many others. [35] Virtualization, in computing, is the process of presenting a logical grouping or subset of computing resources so that they can be accessed by users or systems in ways which give benefits over the original configuration. This virtual view of the resources is not restricted or limited by implementation, geographic location or physical configuration of its underlying resources. This can include, making a physical resource, such as a server, an Operating System(OS) or a storage device(Storage Area Network) device appear to function as multiple logical resources; or, it can include making multiple physical resources, such as, storage devices or servers, appear as a single logical resource. [36] Virtualization is the creation of substitutes for real resources, that is substitutes that have the same functions and external interfaces as their counterparts, but that differ in attributes, such as size, performance, and cost. These substitutes are called virtual resources, and their users are typically unaware of the substitution. [24] Broadly describes the (beneficial) separation of a resource or service from the typical physical means of providing it. [29]


43

Virtualization technology is a way of making a physical computer to function as if it were two or more computers, each non physical of “virtualized” computer is provided with the same basic architecture as that of a generic physical computer. [37] Virtualization is the logical representation of physical systems across the enterprise. [38]

824 scriptie definitief - vurore.nl · virtualization s.c. klaver 3 1 introduction in this chapter...

Documents