everything goes virtual web - uni-muenster.de
TRANSCRIPT
Everything goes virtual
Leveraging virtualization Techniques at every Level of an Enterprise Solution, from Storage and Network to Servers
and Software
Monday, 10-Nov-08 Heinz-Hermann Adam ([email protected])
Agenda
• Who we are and what we do • What means „virtual“? • First virtual steps • Goal • Storage Virtualization • Fileservice Virtualization • Computer Virtualization • Moving from Physical to Virtual
Who we are
• WWU Münster is one of the three major universities in Germany – ~ 40,000 students – ~ 5,000 scientists and
staff – Over 100 fields of
study
• The Natural Sciences Department is ~¼ of the university – Major user and
provider of compute resources
– IT is a Volunteer driven operation
• Not much dedicated staff
What we do
• Provide and maintain resources for students, scientists and staff in Biology, Chemistry and Physics – ~ 4,000 Computers – ~ 12,000 Users
• Compute resources – Scientific Computing
• SMP and Clusters • Development
Environment
– Desktop Applications • Windows • Linux • Mac OS
– File and Print Sharing
What means „virtual“?
• If you can see and it‘s there – It‘s real
• If it‘s there and you can‘t see it – It‘s transparent
• If you can see it, but it‘s not there – It‘s virtual
• If you can‘t see and it‘s not there – It‘s gone
First use of virtualization in production
• License managment – Products like FlexLM
• Tied to an IP-Adress (if you are fortunate) • Mostly tied to a MAC-address
– Moving a license server (P2P) • Upto two weeks of service disruption to acquire new license keys
• VMware GSX-Server – Virtual machine with a vNIC and a fixed MAC-address
00:50:56:11:11:11 – Second GSX-Server for redundancy
• Manual failover for maintenance only • Parallel port dongles are a real challenge
Requesting more from virtualization
• So far 1:2 anti-consolidation
• Additional Services – 2 out of 3 Domain
Controllers (Active Directory)
– Testbeds • Software (re)packaging • Desktop development
• Snap-Shots for Backup – Patching – Configuration changes
• Performance Concerns – How powerful is the GSX
hardware • CPU • Memory
– Snap-Shots • 15 Minutes to complete • VM stops responding,
causing alert – Failover time GSX-to-GSX
• About 1 hour – File copy on the LAN
Initial Situation 2005/2006
• Replacement of IT Infrastructure in Operation since 1998: Overdue
• Isolated Data-Silos of Direct Attached Storage – OpenVMS – Windows – Tru64 UNIX – Linux
• Changed Focus Platforms
Goal
• Consolidation – Versatile Storage System
• Storage Capacity • Data Protection • Reliability, Availabilty, Fault-Tolerance
– Highly Available Fileservice • Transparent to client operating systems
– Unified Computersystem • Scientific Computing ( HPC) • Infrastructure Services (Active Directory etc.)
– Manpower • Data Pools
Storage system
• „single“ SAN storage system • Mirrored between two „independent“
locations • Holds all server and user storage
– Data – No boot-from-SAN
• Flexible allocation of storage space – vRAID for vRAID – vDisk for vDisk – Gigabyte for Gigabyte
Enterprise Virtual Array
• 2 EVA 8000 – 2 Diskgroups (146 and
300 GB disks) – Multi-Pathing SAN (2
complete fabrics) – Continous Access
mirroring between sites – All disks are on the
EVAs • Superdome • Windows Storage Server • Blade Server
Fileservice
• Multi-Protocol NAS-Filersystem – SMB/CIFS + Dfs – NFS – AFP – ...
• Redundant at two sites – Virtual servers (Cluster groups)
inside a Microsoft Cluster Service • 4 ProLiant Storage Server DL 380
– Windows Storage Server 2003 R2 clustered
– GroupLogic ExtremeZ-IP – StorageWorks Cluster Extension
EVA
Unified computersystem
• Proven Concepts – Few, identical systems – Serving both basic needs and scientific
computing • Partitionable SMP-System
– Running multiple operating systems at a time – Windows, Linux, ...
• Redundant at two sites • Extensible • Not feasible
– Split up between • Scientific Computing: Itanium2
• Infrastructure Services: x86
Scientific Computing
• Scientific Computing High Performance Computing – SMP-System > 4 CPU – Shared-memory (ccNUMA) – 64-bit architecture
• Make a difference to PC clusters running Linux or Windows
• Integrity Superdome – 4 Hardpartitions
• 1 Windows • 3 Linux
– 24 Itanium2 Montecito • Dual-Core • Hyperthreading
– 160 GB RAM – Per Partition I/O
• Ethernet • SAN • SCSI/RAID
Infrastructure Services
• 2 BladeSystem c7000 – 6 ProLiant BL480c
• 2 Xeon 5160 Woodcrest Dual-Core @ 3 GHz
• 8 GB RAM • 6 NICs • 2 FC HBAs
– 8 GbE2c LAN switches • 802.1Q VLAN tagging
– 4 Brocade 4/12 SAN switches for HP c-Class
• Switched Fabric
• VMware Virtual Infrastructure 3 Enterprise
12
Ways to become virtual
• Virtual to virtual (V2V) – Copy and import existing GSX Server VMs
• Physical to virtual (P2V) – Transfer running ProLiant installations into VMs
• Setup from scratch – Linux systems – Systems comprising several services moving to
separate systems
Virtual to virtual migration
• VMware GSX or Workstation as a basis – Create directory on Datastore
• [root@esx vmfs1]# mkdir ESX-VM – Copy virtual disks to the ESX server
• /vmimages/GSX-VM – SMB – SCP
– Convert VMDK file to ESX format • [root@esx vmfs1] vmkfstools –i /vmimages/GSX-VM/GSX-VM.vmdk /vmfs/
volumes/vmfs1/ESX-VM/ESX-VM.vmdk – Create a new VM with same hardware configuration as on the GSX
• Use converted VMDK a virtual hard disk – Boot VM
• Update VMware tools – Delete VMDK at /vmimages
Physical to virtual migration
• 2006: VMware P2V Assistant (Windows only) – Create a VM inside the ESX
• same disk layout as the ProLiant – Mount the created VMDKs to a P2Vhelper VM
• Running Norton Ghost console and server – Clone the contents of the ProLiant Disks to the VM over LAN
• BartPE • Norton Ghost Console
– Run the P2V Asistant to change boot controller on the copied system disk • 2008: VMware Converter
– Install on the ProLiant – Create the VMDK while the server is running – Transfer it to the ESX – Boot up virtual Machine
• Shutdown ProLiant first
• HP ProLiant Essentials Server Migration Pack
Setup from scratch
• Only necessary for Linux servers – P2V Assistant does not support Linux – Only experimental support in VMware Converter (2006-2008)
• Or if you want to reassign tasks from one server to many • VM templates save time
– Windows Server 2003 Basis • Installed • Patched • Virus protected
– SuSE Linux Basis • Installed • Patched • Bound to the Active Directory
Deploying Templates
• Windows – Automatically runs sysprep
• Linux – Just creates a copy
• Adjust MAC address – /vmfs/volumes/vmfs1/<vmname>/<vmname>.vmx
• ethernet0.addressType = „vpx“ • ethernet0.generatedAddress = „00:50:56:be:2a:37“
– Change to • ethernet0.addressType = „static“ • ethernet0.address = „00:50:56:00:00:02“
– Rather use VirtualMACTool than fiddle with the vmx files
VI 3 Enterprise Features we use
• Virtual SMP – Windows Terminal Server
• Was a Dual-Xeon DL360 before – Use with caution
• Now using NLB Cluster with 1 CPU VMs
• Virtual Center – Single point of administration for all ESX servers – Can be a VM itself – Rather use a fully featured SQL server than MSDE/SQL Express
• High Availability (HA) – Making a single cluster spanning all ESX servers and sites – Adjust node restrictions
• otherwise no VMs can be powered on if one site fails
VI 3 Enterprise Features we use
• Dynamic Resource Scheduling – Only one rule: Keep same service on different VMs from running
on the same ESX host • VMotion
– 30 seconds to move a running VM from one site to another – Without interuption
• Snap-Shots – Multiple per VM
• As long as you don‘t use VMBK to backup your VMs as a whole – Takes less than 10 seconds
• What we don‘t use – VMware Consolidated Backup
• Does not work in a multi-pathed SAN
What do we gain?
• Consolidation – Less hardware, better utilization
• Availability – Hardware „independence“ of services – HA clustering
• Provisioning – Setup new systems is a matter of minutes
• Management – Less hardware to maintain
• Heterogeneity – Fewer hardware, but still different OS for different tasks
What do we face?
• Anti-consolidation – Virtual sprawl – VM per task for application isolation
• Steep learning curve – More complex hardware – Complex software
Status of Installation
Our Way to Data Pools
• Versatile Storage System – 1 GB units
• All Servers connected to the SAN
• NAS-Cluster for Filesharing • Partitionable SMP Shared
Memory System – Itanium2
– 2-24 CPU • Bladesystem
– X86-64 – VMware Virtual Infrastructure
Q&A – Questions? Please!