Grid testing using virtual machines

Stephen Childs*, Brian Coghlan,

David O'Callaghan, Geoff Quigley,

John Walsh

Department of Computer Science

Trinity College Dublin, Ireland

2

Overview

• Description of Grid-Ireland

• Why use virtual machines?

• Required components for a VM testbed

• Design of TCD TestGrid system

• Using the TestGridBuilder

• Conclusion and future plans

3

Grid-Ireland

• Irish National Computational Grid– 6 sites now, 17 sites by Feb’05

• Grid Operations Centre at TCD

• Homogeneous core infrastructure:– 1 grid gateway per site composed of

FW/LCFG/CE/SE/UI/MON/NM

• Heterogeneous compute resources:– RH 7.3, RH 9, FC 2, AIX, IRIX

4

Why use VMs for Grid testing?

• Need a realistic test environment– Deployment: test new releases on replica

sites before roll-out (n*gateways)

– Middleware development: want a full site image to play with (gateway + n*WNs)

– Porting: to different distros (n machines)

• that’s cost-effective and easy to use– VMs make good use of hardware budget– Fewer machines = easier management

5

VM applications

6

Aims of TCD TestGrid

• Support fast creation of new VMs based on standard configurations

• Provide developers with an interface for configuring and creating VMs

• Facilitate construction of multiple test sites and clusters– Creation of new sites– Addition of nodes to existing sites

7

Required components

VM system that is easy to automate, quick and supports target OS

Library of base node FS images

System for configuring new VMs

Xen: command-line and Web interfaces, low overhead, supports patched Linux kernels

LVM: snapshots allow copy-on-write FS images

LCFG: profile controls network config, package installation, etc.

8

System design

• File system:– Library of FS images stored on VM host

– Each new VM takes a copy-on-write snapshot of a base image

• Networking:– VM eth0 bridged onto real NIC

– IPs specified by user or allocated from site pool

• LCFG profile based on node type and site• Web interface for configuring new VMs

9

System design

Physical machine

10

Creating a new worker node VM

Set node type and other params via web

Create new profile from WN template

Create new snapshot of WN file systemConfigure network settings, copy profile

Boot new VM with appropriate options

User System

Log in using Xen console or ssh

12

Conclusions and future work

• VMs allow developers to build new test environments quickly

• Evaluation of components:– LVM snapshots almost production-ready– Xen performance is impressive– LCFG adequate now; quattor the way forward

• UI needs more work and user feedback• Need to integrate GridSite for security• Work on networking ongoing