1.Delivering a Campus Research Data Service with Globus MAGIC Meeting Ian Foster May 7, 2014

2. Give me your data, your terabytes, Your huddled files yearning to breathe free Building campus research data services 3. Its deja vu all over again. Yogi Berra Globus Toolkit Globus Online Globus Globus 4. What is Globus (today)? Big data transfer and sharing simply, securely, and fast directly from your own storage systems 5. Reliable, secure, high-performance file transfer and synchronization Fire-and-forget transfers Automatic fault recovery Seamless security integration Powerful GUI and APIs Data Source Data Destination User initiates transfer request 1 Globus moves and syncs files 2 Globus notifies user 3 6. Simple, secure sharing off existing storage systems Data Source User A selects file(s) to share, selects user or group, and sets permissions 1 Globus tracks shared files; no need to move files to cloud storage! 2 User B logs in to Globus and accesses shared file 3 Easily share large data with any user or group No cloud storage required 7. 15,000 registered users 8. 8,000 active endpoints (in the past year) 9. 3 billion files transferred 10. Globus is enabling Study of the structure and evolution of galaxies, the nature of dark energy, and cosmological history of the universe Sloan Digital Sky Survey Source: University of Utah Joel Brownstein University of Utah 11. Globus is enabling Development of numerical simulations of severe storms for improved responsiveness to weather events Weather Research and Forecasting Model Source: UCAR Ann Syrowski University of Illinois 12. Globus is enabling Pediatric brain research by enhancing analysis of genetic material in pursuit of the underlying cause Communication impairment by genetic variants Source: Wikimedia Commons William Dobyns U. Washington 13. Globus increasingly used to build campus-wide data service Source: University of Nebraska Holland Computing Center Enable campus computing facilities to better utilize high performance network infrastructure 14. Typical deployment Science DMZ + Globus Omaha Core Holland Computing Center Internet2 via GPN East/West Campus Networks (rewalls + IDS) Lincoln Core Router 2x 10 Gigabit DYNES Equipment UNL Science DMZ Campus Network Researchers WDM Composit Trafc 100 Gigabit 100 Gigabit Capable West Campus Border Router 10x CMS Data Transfer Nodes Omaha HPC Clusters 100 Gigabit Capable East Campus Border Router perfSONAR + BRO IDS additions 10 Gigabit 4x 10 Gigabit 100 Gigabit perfSONAR Bro IDS Future Redundant I2 Path (2015+) Lincoln Core Switch (CMS and HPC clusters) Center for Brain Imaging and Behavior 10x 10 Gigabit Internet2 via CIC Composit Trafc 100 Gigabit Source: University of Nebraska Holland Computing Center 15. Instruments are increasingly driving the need for broader data service deployments Next Gen Sequencer Light Sheet Microscope MRI Advanced Light Source 16. Globus enables users to manage data as research requirements scale up or down Research Computing HPC Cluster Lab Server Campus Home Filesystem Desktop Workstation Personal Laptop XSEDE Resource Public Cloud 17. Globus product development highlights in 2013-14 18. Sharing generally available 19. Much improved Web UI 20. Globus Connect Server Native RPM and Debian packaging Improved configuration management Multi-server setup OAuth support 21. Management console: Flight Control 22. Amazon S3 Endpoints 23. 85 U.S. campuses 24. We are a non-profit, delivering a production-grade service to the non-profit research community 25. Our challenge: Sustainability We are a non-profit, delivering a production-grade service to the non-profit research community 26. Globus Provider Subscriptions Managed Endpoints Priority support Management console Usage reports Mass Storage System optimization Host shared endpoints Integration support Plus Subscriptions Create and manage shared endpoints Personal transfers Branded Web Site Alternate Identity Provider (InCommon is standard) https://www.globus.org/provider-plans 27. NET+ Globus Internet2 members get discounted Globus Provider subscriptions Completing Service Validation phase Sponsors: Cornell, U.Michigan, Yale, U.Missouri, and U.Chicago Available to Early Adopters soon 28. Bridging the gap to sustainability $500,000 from Sloan Foundation Recognition of what it takes to cross the chasm Funds non-R&D activities User Support Operations Marketing 29. Globus Behind the Scenes Identity, Group, Profile Management Services Sharing Service Transfer Service Globus Toolkit GlobusConnect 30. Globus Platform-as-a-Service Identity, Group, Profile Management Services Sharing Service Transfer Service Globus Toolkit GlobusAPIs GlobusConnect 31. globus genomics Flexible, scalable, affordabl e genomics analysis for all biologists 32. + Data management PaaS Next-gen sequence analysis SaaS + Scalable IaaS 33. Globus Genomics on AWS 34. Exome: $3 $20 Whole Genome: $20 $50 RNA-Seq: