futuregrid
DESCRIPTION
FutureGrid. Cyberinfrastructure for Computational Research. FutureGrid Goals. An environment that can be used for middleware developers to develop and test middleware - PowerPoint PPT PresentationTRANSCRIPT
FutureGrid
Cyberinfrastructure for Computational Research
FutureGrid Goals
An environment that can be used for middleware developers to develop and test middleware
An environment that can instantiate a wide variety of computing environments (ranging from current TeraGrid software, VMs, cloud, grid systems other than TG) for development and testing of applications on experimental basis in a test bed, in replicable ways
Education related to all of the above US Industry research related to the above
FutureGrid Components
• Compute – flexible deployment of images, VM's, clouds
• Storage – local storage sufficient to facilitate projects, Lustre for larger projects
• Network – Dedicated networks between FutureGrid sites, Impairment Device
Compute HardwareCompute HardwareSystem type # CPUs # Cores TFLOPS Total RAM
(GB)Secondary
Storage (TB)Site Status
Dynamically configurable systems
IBM iDataPlex 256 1024 11 3072 339* IU New System
Dell PowerEdge 192 768 8 1152 30 TACC New System
IBM iDataPlex 168 672 7 2016 120 UC New System
IBM iDataPlex 168 672 7 2688 96 SDSC Existing System
Subtotal 784 3136 33 8928 585
Systems not dynamically configurable
Cray XT5m 168 672 6 1344 339* IU New System
Shared memory system TBD
40 480 4 640 339* IU New System 4Q2010
IBM iDataPlex 64 256 2 768 1 UF New System
High Throughput Cluster
192 384 4 192 PU Existing System
Subtotal 464 1792 16 2944 1
Total 1248 4928 49 11872 586
Storage HardwareStorage HardwareSystem Type Capacity (TB) File System Site Status
DDN 9550(Data Capacitor)
339 Lustre IU Existing System
DDN 6620 120 GPFS UC New System
SunFire x4170 96 ZFS SDSC New System
Dell MD3000 30 NFS TACC New System
• FutureGrid has a dedicated network (except to TACC) and a network fault and delay generator
• Experiments can be isolated by request• Additional partner machines may run FutureGrid software and be
supported (but allocated in specialized ways)
Logical DiagramLogical Diagram
Network Impairments DeviceNetwork Impairments Device
• Spirent XGEM Network Impairments Simulator for jitter, errors, delay, etc
• Full Bidirectional 10G w/64 byte packets• up to 15 seconds introduced delay (in 16ns
increments)• 0-100% introduced packet loss in .0001%
increments• Packet manipulation in first 2000 bytes• up to 16k frame size• TCL for scripting, HTML for manual configuration
08/25/10 http://futuregrid.org
FutureGrid ArchitectureFutureGrid Architecture• Open Architecture allows to configure resources
based on images• Managed images allows to create similar experiment
environments• Experiment management allows reproducible
activities• Through our modular design we allow different clouds
and images to be “rained” upon hardware.• Will support deployment of preconfigured
middleware including TeraGrid stack, Condor, BOINC, gLite, Unicore, Genesis II
Software GoalsSoftware Goals• Open-source, integrated suite of software to – instantiate and execute grid and cloud experiments. – perform an experiment– collect the results– tools for instantiating a test environment • TORQUE, Moab, xCAT, bcfg, and Pegasus, Inca, ViNE,
a number of other tools from our partners and the open source community• Portal for interacting with tools
– Benchmarking
08/25/10 http://futuregrid.org 10
Draft GUI for FutureGridDraft GUI for FutureGrid Dynamic Provisioning Dynamic Provisioning
Command lineCommand line
• fg-deploy-image– host name– image name– start time– end time– label name
• fg-add– label name– framework hadoop– version 1.0
• Deploys an image on a host
• Adds a feature to a deployed image
08/25/10 http://futuregrid.org 12
FG StratosphereFG Stratosphere
• Objective– Higher than a particular
cloud– Provides all mechanisms to
provision a cloud on a given FG hardware
– Allows the management of reproducible experiments
– Allows monitoring of the environment and the results
• Risks– Lots of software– Possible multiple path to
do the same thing
• Good news– We worked in a team,
know about different solutions and have identified a very good plan
– We can componentize Stratosphere
08/25/10 http://futuregrid.org 13
Change underlying system to support current user demands
Linux, Windows, Xen/KVM, Nimbus, EucalyptusStateless images
Shorter boot timesEasier to maintain
Stateful installsWindows
Use Moab to trigger changes and xCAT to manage installs
08/25/10 14http://futuregrid.org
Dynamic ProvisioningDynamic Provisioning
Experiment ManagerExperiment Manager
• Objective– Manage the provisioning for reproducible experiments– Coordinate workflow of experiments– Share workflow and experiment images– Minimize space through reuse
08/25/10 http://futuregrid.org 15
Getting started with FutureGrid• www.futuregrid.org• Account/Project request
– Nimbus/Eucalyptus account for those systems, if needed
• Getting started information at www.futuregrid.org/tutorials
• Mail [email protected] for any issues
AcknowledgementsAcknowledgements
• FutureGrid - http://www.futuregrid.org/
• NSF Award OCI-0910812• NSF Solicitation 08-573– http://www.nsf.gov/pubs/2008/nsf08573/nsf08573.htm
• ViNe - http://vine.acis.ufl.edu/
• Nimbus - http://www.nimbusproject.org/
• Eucalyptus - http://www.eucalyptus.com/
• VAMPIR - http://www.vampir.eu/
• Pegasus - http://pegasus.isi.edu/