xsede national cyberinfrastructure, nist, and supporting ncsi objectives

May 23, 2017

XSEDE National Cyberinfrastructure, NIST, and Supporting NCSI Objectives John TownsPI and Project Director, XSEDEExecutive Director, Science & Technology, [email protected]

License terms• Please cite as: Towns, John. XSEDE National Cyberinfrastructure, NIST, and

Supporting NCSI Objectives, May 2017, [https://www.slideshare.net/jtownsil/xsede-national-cyberinfrastructure-nist-and-supporting-ncsi-objectives]

• ORCID ID: http://orcid.org/0000-0001-7961-2277• Except where otherwise noted, by inclusion of a source URL or some other

note, the contents of this presentation are © by the Board of Trustees of University of Illinois. This content is released under the Creative Commons Attribution 3.0 Unported license (http://creativecommons.org/licenses/by/3.0/). This license includes the following terms: You are free to share – copy and redistribute the material in any medium or format; and to adapt – remix, transform, and build upon the material for any purpose, even commercially.

• This can be done under the following conditions: attribution – You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.

https://creativecommons.org/licenses/by/4.0/

https://creativecommons.org/licenses/by/4.0/

Motivation for XSEDE:• Scientific advancement across multiple disciplines

requires a variety of resources and services• XSEDE is about increased productivity of the

community and providing expanded capabilities– leads to more science– is sometimes the difference between a feasible project

and an impractical one– lowers barriers to adoption

• XSEDE provides a comprehensive eScience infrastructure composed of expertly managed and evolving advanced heterogeneous digital resources and services integrated into a general-purpose infrastructure

3

XSEDE – accelerating scientific discovery

XSEDE’s Vision: a world of digitally enabled scholars, researchers, and

engineers participating in multidisciplinary collaborations while seamlessly accessing advanced computing resources and sharing data to tackle society’s grand challenges.

XSEDE’s Mission: to enhance the productivity of a growing community

of scholars, researchers, and engineers through access to advanced digital services that support open research by coordinating and adding value to the leading cyberinfrastructure resources funded by the NSF and other agencies.

4

Vision/Mission: Enable Realizing Best Science

5

Total Research Funding Supported by XSEDE to Date

6

$2.84 billion in research supported by XSEDEJuly 2011-March 2017

Research funding only. XSEDE leverages and integrates additional infrastructure,

some funded by NSF (e.g. Track 2 systems) and some not (e.g. Internet2).

NSF, $978.5M, 34%

NIH, $607.2M, 21%

DOE, $558.2M, 20%

DOD, $217.5M, 8%

NASA, $113.1M, 4%

DOC, $53.7M, 2%

All Others, $311.7M, 11%

XSEDE Factoids: high order bits• 5 year, US$110M project

– pursuing additional funding via independent proposals– initial 5 year award: $121M project + ~$4.6M in supplements

• plus $9M, 5 year Technology Investigation Service– separate award from NSF

• No funding for major hardware– coordination, support and creating a national/international

cyberinfrastructure– coordinate allocations, support, training and documentation for

>$100M of concurrent project awards from NSF

• ~90 FTE /~240 individuals funded across 19 partner institutions– this requires solid partnering!

7

– partnership led by NCSA, PSC, TACC, SDSC and NICS– CI centers with deep experience

– partners who strongly complement these CI centers with expertise in science, engineering, technology and education

XSEDE’s Distinguishing Characteristics: World-class Leadership

8

9

Accelerating Scientific Discovery

Darter

SuperMICStampede

Maverick

XStreme

WranglerMason Ranch

Data Oasis

And many more …

XD Service Providers Forum9 Level 1 SPs7 Level 2 SPs

19 Level 3 SPs

People, organizations,& communities

Software

Data

Scientific Instruments Computational

Resources

Networking & Cybersecurity

HYPOTHESIZE

EXPERIMENTANALYZE

THEORIZE

OBSERVE

CYBERINFRASTRUCTURE ECOSYSTEM

Courtesy of NSF

Significant Contributions to Community by XSEDE:Supporting the Community• Annually supporting 15,000 researchers and students via allocated access

to resources– 6,000 through direct allocations + 9,000 through use by science gateways– science gateways allowing many more to leverage XSEDE-allocated resources– many more leveraging unallocated services via the XSEDE User Portal

• More than 21,000 publications supported to date since July 2011– analysis shows significantly higher citation rate than other publications in the

same journals• Over 15,000 active users of XSEDE User Portal in the past 12 months• Over 900 proposals for major compute allocations reviewed annually

– a comparable number of requests for startup and educational allocations• More than 77,000 user requests addressed to date since July 2011

– answering questions and resolving technical issues• To date, completed more than 375 projects that assisted computational

research teams– users reported an average productivity gain of 14.75 months– average three months investment of XSEDE staff time

10

XSEDE offers efficient and effective integrated access to a variety of resources• Leading-edge distributed memory systems• Very large shared memory systems• High throughput systems, including Open Science

Grid (OSG)• Support for VM’s and containers and HPC Cloud• Visualization engines• Accelerators like GPUs and Xeon PHIs• Extensive library of research applicationsMany scientific problems have components that call for use of more than one platform.

11

XSEDE User Portal: THE User Siteportal.xsede.org• XSEDE User Portal (XUP) is designed to be the only site

a user needs to use XSEDE• XUP presents information relevant to users

– user info is easier to find– XUP also provides dynamic data about XSEDE systems– capabilities to manage usage, files, data

• As a user you can– request an allocation, and manage allocations– sign up for training– request help– manage files and data, and much more!

– Portal provides single sign-on to all XSEDE resources

Current XSEDE Compute Resources • Stampede @ TACC

– 9.5 PFLOPS (PF) Dell Cluster w/ GPUs and Xeon PHIs

• Comet @ SDSC– 2.1 PF cluster w/GPUs

• Bridges @ PSC– 1.3 PF w/ large memory

(274 TB)• XStream @ Stanford

– 1.0 PF GPU Cray CS-Storm cluster

• SuperMIC @ LSU– 925 TF Dell Cluster w/

GPUs and Xeon PHIs• Jetstream @ Indiana

– 516 TF HPC Cloud• Wrangler @ TACC

– 62 TF data analytics system• Open Science Grid

– 60,000 CPU cores

13

https://www.xsede.org/web/xup/resource-monitor


Current XSEDE Visualization, Data, and Software Resources• Storage

– Ranch @ TACC• 61 PB tape

– Pylon @ PSC• 10 PB disk

– Wrangler @ TACC• 10 PB disk

– Data Oasis @ SDSC• 4 PB tape

• Visualization– Maverick @ TACC

• 59 TF HP/NVIDIA cluster• 20 PB disk

• Software: 100s of titles– domain software

• chemistry, CFD, bioinformatics, physics, astronomy, biology, engineering, statistics,…

– tools • middleware, visualization,

scripting, performance analysis, data storage and management, …

– compilers and libraries• most languages supported,

math libraries, machine learning, …

14

https://www.xsede.org/web/xup/resource-monitorhttps://portal.xsede.org/software#


https://portal.xsede.org/software

Extended Support Available - ECSS

• The Extended Collaborative Support Service:– improves the productivity of the XSEDE user

community through successful, meaningful collaborations to

• optimize their applications, • improve their work and data flows, and • increase their effective use of the XSEDE digital

infrastructure and – broadly expands the XSEDE user base by engaging

members of underrepresented communities and domain areas

15

What is ECSS?• Extended Collaborative Support

Service• Expert staff can be requested for

collaborations lasting months to a year– requests made through the XSEDE

allocation system when requesting compute/data/viz resources

• Typical collaborations require 20-25% staff time for one year

• Critical mass engenders success– ~28 FTEs (~70 individuals), 10 sites

• 24.5 w/out management– advanced degrees in a variety of

science and technology fields

16

A Few NIST Examples

17

Title:Sequence specific binding of single-stranded DNA (ssDNA) with single-walled carbon nanotubes (SWCNTs): An investigation on the effect of SWCNT chirality via replica exchange molecular dynamics simulations

PI: Kevin Hinkle

Institution: National Institute of Standards and Technology

Field of Science: Materials Research

Resource Allocation

TACC-RANCH 1,000

TACC-STAMPEDE 1,020,092

Title: Multi-Scale Synthesis and Functionalization of 2D Materials for Nano-Electronic Devices and Energy Conversion Applications

PI: Arunima Singh



Resource Allocation

TACC-RANCH 1,000

TACC-STAMPEDE 511,593

TACC-RANCH 1,000


18

Title: Structure and function of membrane-associated proteins: computational approaches to understanding experimental data

PI: David Hoogerheide


Field of Science: Biophysics

Resource Allocation

NICS-DARTER 100,000

PSC-BRIDGES 453,035

PSC-BRIDGES-LARGE 10,000

PSC-BRIDGES-PYLON 500

Title: Simulating 3D Reaction Kinetics and Microstructure Development in Plaster

PI: Judith Terrill



Resource Allocation

TACC-RANCH 500


TACC-RANCH 15,900


Startups Allocations Also Awarded

• Joseph Curtis (biophysics)– Towards a SASSIE-web Scientific Gateway

• Nathan Mahynski (materials research)– Designing Open Colloidal Crystals with Structure

Directing Agents

• Benjamin Neely (biological sciences)– Developing novel tools for analyzing *omic data

sets and for biomarker discovery

19

XSEDE in a Broader Context:Positioning vis-à-vis NSCI• NSF called on to provide leadership

– scientific discovery advances– broader HPC ecosystem for scientific discovery– workforce development

• OAC playing key roles– NSF lead for NSCI: Irene Qualters (OAC AD)– NSF-wide working group: co-chaired by Rudi

Eigenmann (OAC) and Eduardo Misawa (MPS)• Eigenmann is XSEDE Cognizant Program Officer

20

Scientific Discovery Advances

• Anticipate this is largely funding of basic research

• XSEDE potential contribution– provision of integrated resources, services, and

support enabling scientific discovery

21

Broader HPC Ecosystem for Scientific Discovery

• XSEDE potential contribution– defining and instantiating integrating

infrastructure bringing order to the ecosystem– connector of services

• supporting integration of wide range of capabilities into the ecosystem enabling coordinated use of same

– introduction to new technologies and facilitated use without investment

– support services enabling use of ecosystem

22

Workforce Development

• XSEDE potential contribution– significant success already in training the

workforce– supporting the emergence of a new profession

• CI Professional/CI Engineer/etc.– complementing campuses in developing

computational science and engineering certificate and degree programs

• will need to treat this more broadly than CDS&E

23

Questions?

24