august 2007 advancing scientific discovery through teragrid adapted from s. lathrop’s talk in...
TRANSCRIPT
August 2007
Advancing Scientific Discovery through TeraGrid
Adapted from S. Lathrop’s talk in SC’07
www.teragrid.org
August 2007
SDSC
TACC
UC/ANL
NCSA
ORNL
PU
IU
PSC
NCAR
Caltech
USC/ISI
UNC/RENCI
UW
Resource Provider (RP)
Software Integration Partner
Grid Infrastructure Group (UChicago)
11 Resource Providers, One Facility
LSU
U Tenn.
August 2007
TeraGrid Objectives
• DEEP Science: Enabling Petascale Science–Make Science More Productive through an integrated set of very-high capability resources
•Address key challenges prioritized by users
• WIDE Impact: Empowering Communities–Bring TeraGrid capabilities to the broad science community
•Partner with science community leaders - “Science Gateways”
• OPEN Infrastructure, OPEN Partnership–Provide a coordinated, general purpose, reliable set of services and resources
•Partner with campuses and facilities
August 2007
TeraGrid Resources• Computing - over 250 Tflops today and growing
– 400 Tflop system comes on-line in January at TACC– U Tennesee system to come on-line in 2008– Centralized help desk for all resource providers
• Visualization - Remote visualization servers and software• Data
– Allocation of data storage facilities – Over 100 Scientific Data Collections
• Access– Shibboleth testbed to facilitate campus access– Central allocations mechanism
• Human Support– Advanced Support for TeraGrid Applications (ASTA)– Education and training events and resources– Over 20 Science Gateways
August 2007
Coupled Simulation: Full Body Arterial Tree Simulation
Karniadakis (Brown)
Virtualized Resources, Ensembles:
FOAM Climate
Model
Liu (UWisc)
Sources: Ian Foster (UC/ANL), Mike Papka (UC/ANL), George Karniadakis (Brown). Images by UC/ANL.
Advanced Support for TeraGrid Applications
August 2007
Requesting Allocations of Time
• TeraGrid resources are provided for free to academic researchers and educators
• Development Allocations Committee (DAC) for start-up accounts up to 30,000 hours of time are requests processed in two weeks - start-up and courses
• Medium Resource Allocations Committee (MRAC) for requests of up to 500,000 hours of time are reviewed four times a year
• Large Resource Allocations Committee (LRAC) for requests of over 500,000 hours of time are reviewed twice a year
August 2007
25
50
75
100
125
150
175
200
225
250
275
J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J
2004 2005 2006 2007
NU
s (m
illi
on
s)Specific
Roaming
TeraGrid Usage
33% Annual Growth
Specific Allocations Roaming Allocations
200
100
Normalized Units (millions)
TeraGrid currently delivers an average of 420,000 cpu-hours per day -> ~21,000 CPUs DC Dave Hart ([email protected])
August 2007
Use ModalityUse ModalityCommunity SizeCommunity Size
(est. number of (est. number of people/projects)people/projects)
Batch Computing on Individual Resources 850
Exploratory and Application Porting 650
Workflow, Ensemble, and Parameter Sweep 160
Science Gateway Access 100
Remote Interactive Steering and Visualization 35
Tightly-Coupled Distributed Computation 10
TeraGrid Usage Modes in CY2006
Grid
-y U
sers
August 2007
Science GatewaysBroadening Participation in TeraGrid
• Increasing investment by communities in their own cyberinfrastructure, but heterogeneous:
• Resources• Users – from expert to K-12• Software stacks, policies
• Science Gateways– Provide “TeraGrid Inside”
capabilities– Leverage community investment
• Three common forms:– Web-based Portals – Application programs running on
users' machines but accessing services in TeraGrid
– Coordinated access points enabling users to move seamlessly between TeraGrid and other grids.
Technical Approach
Biomedical and Biology, Building Biomedical Communities
OG
CE
Sc
ien
ce
Po
rta
l
OGCE Portletswith ContainerOGCE Portletswith Container
Apache JetspeedInternal ServicesApache JetspeedInternal Services
ServiceAPI
ServiceAPI
GridProtocols
GridServiceStubs
GridServiceStubs
RemoteContentServices
RemoteContentServices
RemoteContentServersHTTP
GridService
s
Java
Co
G K
it
LocalPortal
Services
LocalPortal
Services
Grid Resources
Open Source Tools
Build standard portals to meet the domain requirements of the biology communitiesDevelop federated databases to be replicated and shared across TeraGrid
Workflow Composer
Source: Dennis Gannon ([email protected])
August 2007
Gateways are growing in numbers• 10 initial projects as part of TG proposal• >20 Gateway projects today• No limit on how many gateways can use TG
resources– Prepare services and documentation so
developers can work independently
• Open Science Grid (OSG)• Special PRiority and Urgent Computing
Environment (SPRUCE)• National Virtual Observatory (NVO)• Linked Environments for Atmospheric
Discovery (LEAD)• Computational Chemistry Grid (GridChem)• Computational Science and Engineering
Online (CSE-Online)• GEON(GEOsciences Network)• Network for Earthquake Engineering
Simulation (NEES)• SCEC Earthworks Project• Network for Computational Nanotechnology
and nanoHUB• GIScience Gateway (GISolve)• Biology and Biomedicine Science Gateway• Open Life Sciences Gateway• The Telescience Project• Grid Analysis Environment (GAE)• Neutron Science Instrument Gateway• TeraGrid Visualization Gateway, ANL• BIRN• Gridblast Bioinformatics Gateway• Earth Systems Grid• Astrophysical Data Repository (Cornell)
August 2007
TeraGrid as a Social Network
• Annual TeraGrid conference - TeraGrid ‘08 - Las Vegas - June
• LRAC/MRAC liaisons• SGW community very successful
– Transitioning to consulting model
• CI Days - campus outreach– OSG/Internet2/NLR/
EDUCAUSE/MSI-CIEC partnership
• HPC University– OSG, Shodor, Krell, OSC, NCSI,
MSI-CIEC partnership
• Education and Outreach – Engaging thousands of people