florida high performance computing summit march 21-22, 2001 gainesville, florida

Florida High Performance Computing Summit

March 21-22, 2001Gainesville, Florida

Sunshine Grid: Motivation

“The foundation we’re laying today, like the physical infrastructure of a building or city, is the supporting framework necessary for the scientific discoveries of the future.”Arden L. Bement, Director National Science Foundation, 2004-2010

Sunshine Grid: Florida’s Research and Education Cyberinfrastructure

• Describe the New Florida Initiative• Motivation for the Sunshine Grid proposal• What is the Sunshine Grid• Deliverables

o Sunshine Grid Web and DB (Dan Majchrzak)o Support for three science areas (Paul Avery)

• Time for your thoughts and insights

New Florida Initiative

• The 2010 Florida Legislature invested a total of $12 million in New Florida Initiative

• Designed to:o Demonstrate the power of the SUS when

it collaborates in areas critical to Floridians.

o Produce meaningful outcomes in a short timeframe

• Should serve as a change agent for FL

• Transform FL’s economy beyond tourism, agriculture, and housing

• Should promote the creation of high-skill, high-wage, knowledge-based employment

• Results should foster economic recovery, growth, and future prosperity

New Florida Initiative

• Commercialization Assistance Grants Programo link research activities at Univ. with commercial productso $2 million of the $12 milliono State University Research Commercialization Assistance

Grants (SURCAG) Program• New Florida Scholars Boost Grants Program

o Assist in hiring, retaining, and equipping the best faculty candidates and existing faculty

o Similar to the former “21st Century World-class Scholars Program,” but with less funding

• New Florida Clustering Grants Programo Focus a collaboration between or among universitieso Fast time-to- deliveryo Associated with health, engineering, and/or science

New Florida Initiative

• Announced in late-summer 2010• Proposals were due mid-Sept.• Each University independently

selected and ranked proposals • Ranked lists were submitted to BOG

for final review

New Florida Initiative

Sunshine Grid

• Announcement was made• We hit the Ground Running

Sunshine Grid: MotivationOther Efforts

• Florida Center for Computational Biology (FCCB)o Lead by UFL, in collaboration

with FSU, UCF, FIU, and USF

o Board of Governors Centers of Excellence Competition, 2007

• Collaborative Acquisition of High-performance Storage and Visualization Infrastructureo UF and FSU collaborationo Major Research and

Instrumentation – Recovery and Reinvestment Act, NSF, 2009

Sunshine Grid: MotivationScience Paradigms (a la Jim Gray)

• Experimentso Describing natural

phenomena• Theory

o models for generalizations

• Computationso Simulate complex

phenomena• Large-scale Data

Explorationo unify experimentation,

theory, and simulation

Sunshine Grid: Motivation• “Shared NIT infrastructure – be it computational resources, communication

networks, community databases, or collaboration tools – has become essential to research in virtually all fields.” o President’s Council on Science and Technology (PCAST), 2010

• “By working together, the HPC and CI communities best serve the mutually reinforcing goals of (1) sustaining the entire computational pyramid while (2) generating economic growth via breakthroughs in science and engineering.”o NSF sponsored workshop on Sustainable Funding and Business Models for

Academic Cyberinfrastructure Facilities, 2010• “Institutions of higher education should lead efforts to fund and invest in

university-specific, state-centric, and regional cyberinfrastructure to create local benefits (in research accomplishment and local economic development) and to aid the global competitiveness of the US and thus the long-term welfare of US citizens.”o NSF Advisory Committee for Cyberinfrastructure Task Force on Campus Bridging,

2011• “Development and acquisition of research instrumentation for shared inter-

and/or intra-organizational use are encouraged, as are development efforts that leverage the strengths of private sector partners to build instrument development capacity at academic institutions.”o NFS Major Research Instrumentation Solicitation, 2011

Sunshine Grid: Motivation

• Florida’s Universities are home to a wealth of world-class resourceso Talented scientistso High-end computing

facilitieso Massive data storage

systemso Specialized research

instrumentso High-speed state-wide

network (FLR)

Sunshine Grid: Motivation

Organize assets in such a way as their collective impact is greater than the sum of their individual parts.

Sunshine Grid: Motivation

• Workforce Florida and Enterprise Floridao 2010-2015 Strategic Plan

for Workforce Development strongly endorses STEM development at FL Univ.

• Florida STEM Councilo Promote ties between

industry and academiao “Florida will be a national

leader in market relevant STEM talent development and retention”

Sunshine Grid: Proposal

• Build a coherent cyber-infrastructure across three state universities:o University of Floridao University of South Floridao Florida State University

• Use this as a base for future development to include ALL of Florida’s public and private Universities.

CyberinfrastructureCampus Cyberinfrastructure Working Group (EDUCAUSE)& Coalition for Academic Scientific Computation (CASC)

Consists of computational systems, data and information management, advanced instruments, visualization environments, and people, all linked together by software and advanced networks to improve scholarly productivity and enable knowledge breakthroughs and discoveries not otherwise possible.

Sunshine Grid: Proposal

• Goals of proposal closely aligned with New Florida Initiativeo Increase research fundingo Enable leading-edge researcho Provide educational opportunitieso Promote healthy economic development

Sunshine Grid: Deliverables

• Two-pronged approacho Catalog High-end Resources

Internal use: track assets and progress related to linking assets

Showcase STEM resources; attract positive attention to Florida

o Support three well defined research projects well defined goals need for data storage and compute cycles

• What we asked from New Floridao 400K per University

$280K in salaries $120K in capital

New Florida: Cluster Awards

• 93 proposals submitted to the BOGo Cumulatively worth $32.4 million

• 31 projects were funded• Successful Cluster Awards:

o Aerospace, Aging Issues in North Florida, Biomedical Engineering, Climate Change, Coastal Watersheds, Community Health, Cyber-infrastructure, Family Medicine, Geophysical Threats, Medical Prostheses, Neuroscience, Professional Science Master’s Degree, Smart Sensors, and Vector Borne Diseases.

• Sunshine grid was funded because close alignment with New Florida’s Goals and quick turn around

Sunshine Grid: Award

Sunshine Grid: Award

• What we got from New Floridao UF: $200Ko FSU: $150Ko USF: $100K

• Internal matching was honoredo UF: $200Ko FSU: $150Ko USF: $140K

Sunshine Grid: Day-to-Day Leadership

• HPC Directors at UF, FSU, USF  - Erik Deumens (UF)  - Daniel Majchrzak (USF)  - Jim Wilgenbusch (FSU)

Sunshine Grid: Advisory Panel Members

• Paul Avery (High Energy Physics) (UF)• Scott Stagg (Bio Imaging) (FSU) • Qingnong  Xiao (Climate Modeling) (USF) • Joel Hartman (Florida Lamda Rail/SUS

CIO) (UCF) • Nick Tsinoremas (Private Florida Univ.)

(UM)• Sunshine Grid PIs (ex officio)

Sunshine Grid: Key Milestones

• Jan/Feb – Advertise new positions• Feb – Invite Sunshine Grid advisory panel members• Feb/May – Hire staff at each University• April/May – Release version of Sunshine Grid

DB/Web• May/June – Convene first Sunshine Grid Advisory

meeting• July – Demonstrate shared storage over FLR• Aug – Release Showcase on web of three science

projects• Nov – Host industry forum to bridge industry and

academia• Nov – Supercomputing 2011 (SC11), Seattle, WA• Nov – Solicit ideas for additional projects

Sunshine Grid: Architecture

• Components o Searchable catalog of research resources in

the state of Floridao Showcase of Florida’s high-end resourceso Web-based outreach and education resourceso Statewide cyber-infrastructureo People

Sunshine Grid: Architecture

• Searchable catalog of research resources in the state of Floridao Researchers are a resource

!o Efficient use of resourceso Help industry find resourceso Enable collaboration

Sunshine Grid: Architecture

• Searchable catalog of networked research resources in the state of Floridao Web based for easy accesso High available cluster infrastructureo Updated by grid personnelo CMS based for ease of maintenance

Sunshine Grid: Architecture

• Using cloud tag technologyo Databases are too inflexible for rapidly changing

technologies which leads to inflexible searching and underutilization

o Tags are contributor generated labels: FSU, UF, HPC, high energy physics

o The Tag Cloud will show the top N tags. The larger or more prominent a tag is, the more it has been used to describe a resource

Sunshine Grid: Architecture

• Cloud tag technology – example music

Sunshine Grid: Architecture

• Web-based Outreach and Educationo Florida’s Universities (e.g., python tutorials)o Community Collegeso K-12

Sunshine Grid: Architecture

• A state-wide cyberinfrastructureo Leverages expertise at Universities with existing

HPC centerso Allows universities without HPC to provide

resources without startup investmento Provides interface to national and international

resource (Open Science Grid, TeraGrid/XD)o Provides competitive edge to state researchers,

universities, and industry

Sunshine Grid: Architecture

• A state-wide cyberinfrastructureo Common Authorization/Access Method

Open Science Grid InCommon Home-grown

Sunshine Grid: Architecture

• A state-wide cyberinfrastructureo High speed storage available from across the

state Data may be the most difficult problem to solve File sharing via distributed Lustre implementation

o Shared computational resources Shared to other Florida Universities Shared to outreach partners Built on Florida Lambda Rail (FLR)

Sunshine Grid: Architecture

• A state-wide cyberinfrastructureo People

Experts Ambassadors Collaborators

Sunshine Grid: Research Projects

• CMS experiment at CERNo UF, FSU (+ FIT, FIU)

• Cryo Electron Microscopy (CryoEM)o FSU and UF

• Coupled Ocean Atmosphereo USF and FSU

CMS Experiment at CERN’s Large Hadron Collider

A Proton-Proton CollisionWill Record ~109 collisions/year

CMS Computing

• ~10s PB of collision data collected per yearo Similar amount of simulated data

• Worldwide distributed computing resourceso US part supported by Open Science Grid

• UF is a “Tier-2” university siteo Data repository for physics analyseso Supports FSU, FIT, FIU

CMS Experiment

CMS Experiment Global Grid

Online System

CERN Computer Center

FermiLabKorea RussiaUK

FIT

300 MB/s

>10 Gb/s

10-40 Gb/s

2.5-10 Gb/s

Tier 0

Tier 1

Tier 3

Tier 2

Physics caches PCs

FIU

UCSDCaltechU Florida

• >2000 physicists, 60 countries• 10s of Petabytes/yr• CERN / Outside = 10-20%

FSU

OSG

Miami 2010 (Dec. 16, 2010)

Paul Avery A. FarbinM. Ernst, BNL

LHC

Worldwide LHC Computing Grid

Miami 2010 (Dec. 16, 2010)

Paul Avery From Ian Bird

Open Science Grid:LHC, Chemistry, Bioinformatics, Etc.

CMS Supported Faculty

• UFo Darin Acosta, Paul Avery, Ivan Furic, Richard Field,

Jacobo Konigsberg, Andrey Korytov, Konstantin Matchev, Guenakh Mitselmakher, John Yelton

• FSUo Todd Adams, Andrew Askew, Harrison Prosper,

Sharon Hagopian, Vasken Hagopian, Kurtis Johnson

• FIT & FIUo Marc Baarmand, Marcus Holhmann, Steve Linn,

Pete Markowitz, Jorge Rodriguez

Other Information

• http://cms.web.cern.ch/cms/• http://www.phys.ufl.edu/ihepa/cms.html• http://www.hep.fsu.edu/cms.html• http://research.fit.edu/hep/• http://casgroup.fiu.edu/physics/pages.php

?id=3091

CryoEM

• Automated Cryogenic Transmission Electron Microscopeo Determine the three-dimensional structures of

large biological molecules at high resolutiono Many medical and basic science applications

• Instrumentso FSU: Titan Krios at Institute for Molecular

Biologyo UF: G2 F20 (forthcoming)o Both have high resolution & high data rates

Titan Krios EM at FSU• Resolution ~1 nm

CryoEM Integration with HPC

• Bring image data and metadata produced by both microscopes to HPC facilities at FSU & UFo Improved retention and availability of cryoEM

datao Data will be placed on Lustre file systems for

transparent sharing between both campuseso Available to other research groups in Florida

CryoEM Supported Faculty

• FSUo Scott Stagg, Kenneth Taylor, Kenneth Roux

Thomas Roberts, Beth Stroupe • UF

o Byung-Ho Kang, Mavis Agbandje-McKenna, Robert McKenna

Coupled Ocean-Atmosphere Models

• State-of-the-art ensemble ocean-atmosphere-wave coupled forecasting system

• Utilizes latest developments in regional mesoscale atmospheric, ocean, and wave models as well as in ocean and hurricane vortex initialization

Wikipedia

Goals

• Quantifying the predictability of the ocean-atmosphere-wave coupled system

• Communicating quantified threats to public agencies for improved decision making

Atmos-Ocean Supported Faculty

• FSUo R. Hart, E. Chassignet, M. Bourassa, C.

Clayson, S. Morey, D. Dukhovskoy• USF

o R. Weisberg, J. Collins, Q. Xiao

Other Information

• http://www.coaps.fsu.edu/• http://ocgweb.marine.usf.edu/