Cupid Integrated Development Environment: Strategies for

Training and Outreach

Rocky DunlapNOAA ESRL/University of Colorado

ESMF Executive Board/Interagency MeetingJune 12, 2014

NCWCP, College Park, MD

Outline

• Common Modeling Infrastructure• Cupid Integrated Development Environment• Cupid Demo• New Opportunities for Training and Outreach

Recent developments move us toward the vision for common modeling infrastructure1:

• Increased use and maturity of ESMF 5500+ downloads, 3300+ on mailing list ESMF is part of most US climate and weather models

• NUOPC Software layer designed to increase interoperability of ESMF

components Defines key architectural components; formalizes initialization

sequence; runtime field brokering• ESPS

A collection of weather and climate modeling codes that use ESMF with the NUOPC conventions

Training and Outreach for Common Modeling Infrastructure

1 National Research Council. A National Strategy for Advancing Climate Modeling. Washington, DC: The National Academies Press, 2012. http://www.nap.edu/catalog.php?record_id=13430

• Widespread adoption of ESMF need for comprehensive training options that complement existing

resources (documentation, tests, support list)• NUOPC formalisms

make it possible to create a structured training environment based on well-defined technical rules

• ESPS shift from infrastructure to delivery of applications; training across

agencies and models

Goal: Make ESMF development simpler and more appealing while accelerating the rate at which ESMF is adopted into new components

Training and Outreach for Common Modeling Infrastructure

Emergence of common infrastructure means there are new opportunities for modeler training and outreach.

Cupid Development andTraining Environment

Cupid is a tool designed to make ESMF training and development simpler, faster, and more appealing•NOAA CIRES, GA Tech, and NASA GISS/GSFC collaboration•NASA Computational Modeling Algorithms and Cyberinfrastructure (CMAC) program•Plugin for Eclipse-based “Integrated Development Environment” or IDE•Customized for ESMF applications with NUOPC conventions

Cupid is a working prototype expected to be ready for first public release in FY14.

•https://earthsystemcog.org/projects/cupid/•Cupid Feature Overview and Tutorial

Cupid Development andTraining Environment

Select sample code or model • Pick a training problem (or coupled model)

• Generate a framework-aware outline of the source code

• Navigate around the source code using the outline

• Use an editor to modify the source code

• Automatically generate code needed for NUOPC compliance

• Compile and run locally or on a cloud (currently Amazon Web Services)

Run locally or on a cloud

Source code editor

Console for viewing output

Projectexplorer

NUOPC outline

Selecting the TrainingScenario or Model

Four options are available,

each with a different architecture and coupling behavior:•Single Model with Driver•Coupled Atmosphere-Ocean Driver•Coupled Atmosphere-Ocean with Mediator and Driver•Model E EM20 (atmosphere only) configuration (untested)

Scenario selection:•Populates the Project Explorer•Determines software dependencies

Smart, Framework-aware Code Editing Environment

Central idea: A framework-aware editor provides contextual guidance as code is being written by checking for compliance to framework rules and automatically generating and inserting code fragments into existing code

Reverse engineer

Generate code

Smart, Framework-aware Code Editing Environment

NUOPC View•compact outline of NUOPC component source code•linked with source code editor•contextual reference documentation•compliance issues shown in red•new elements can be added to outline, simultaneously generating code in the editor

Running Code in Cupid: Local or Cloud-based Job Submission

Eclipse projects can be configured to run on a personal or cloud-based computing system.

Why the cloud?

•Configuration of a computational environment for compiling and running models requires expert knowledge, and•Configuration of an IDE for a particular computing system (e.g., NASA/Discover) introduces complexity

Cloud-based Training Environment

Central idea: Integrate cloud computing with the IDE so trainees get quickly to important learning objectives

Advantages:•trainee not required to configure computational environment•on-demand training in low-risk sandboxes•pre-configured scenarios highlight important ESMF/NUOPC concepts

New Cupid Training Project Wizard spins up Amazon EC2 instances

Cupid Demo

New Opportunities• Online course development

Partnership with Georgia Tech Professional Education

• Integration into Graduate Curriculum Participate in proposed interdisciplinary NSF Research Traineeship

Program (NRT) at UC Boulder (PIs: James Syvitski, Greg Tucker) ESMF team asked to develop graduate student workshop Research Operations

• Advance Cupid NASA Advanced Information Systems Technology program (AIST) and

other solicitations Raise Technology Readiness Level, greater NUOPC API support,

improved testing Bring ESPS models into Cupid Operations Research

Georgia Tech Global Learning CenterSlide courtesy Leo Mark, Georgia Tech Professional Education

Who we serve:• Students• Alumni• Professionals, Industry

and Government

How we do it:• Online• Face-to-face• Blended• Open enrollment &

customized programs

Impact:•Serving ~3,500 companies•Enrollments ~25,000/year•Adding MOOCs: 466,203 enrollments!!•Students age: 13-91 years•represented 111 countries, and nearly all ethnic, cultural, and socio-economic backgrounds

Slide courtesy Leo Mark, Georgia Tech Professional Education

Slide courtesy Fatimah Wirth and Kristen Anderson, Georgia Tech Professional Education

New Opportunities• Online course development

Partnership with Georgia Tech Professional Education

• Integration into Graduate Curriculum Participate in proposed interdisciplinary NSF Research Traineeship

Program (NRT) at UC Boulder (PIs: James Syvitski, Greg Tucker) ESMF team asked to develop graduate student workshop Research Operations

• Advance Cupid NASA Advanced Information Systems Technology program (AIST) and

other solicitations Raise Technology Readiness Level, greater NUOPC API support,

improved testing Bring ESPS models into Cupid Operations Research

New Opportunities• Online course development

Partnership with Georgia Tech Professional Education

• Integration into Graduate Curriculum Participate in proposed interdisciplinary NSF Research Traineeship

Program (NRT) at UC Boulder (PIs: James Syvitski, Greg Tucker) ESMF team asked to develop graduate student workshop Research Operations

• Advance Cupid NASA Advanced Information Systems Technology program (AIST) and

other solicitations Raise Technology Readiness Level, greater NUOPC API support,

improved testing Bring ESPS models into Cupid Operations Research

Conclusions

The time is right to start thinking about modeler training in a comprehensive way.

Many opportunities before us, requiring partnerships with universities, experts on instructional design, and practitioners at modeling centers.

ESMF group is uniquely situated for this opportunity!

Thanks!

https://earthsystemcog.org/projects/cupid/

Extra slides

NSF Research Traineeship Program (NRT)

• We have been asked to participate in a UC Boulder NRT proposal and have provided a letter of support

• PIs:• James Syvitski, Dept. Geol. Sci., INSTAAR• Greg Tucker, Dept. Geol. Sci., CIRES

• Research Traineeship Program:• interdisciplinary training of next generation earth

scientists: computational science + applied mathematics + physical science

• interaction with professional community in surface dynamics

• ESMF team asked to develop graduate student workshop

NASA Advanced Information System Technology (AIST)

“bring information system technologies to a Technology Readiness Level (TRL) that allows integration into existing or future technology or science research and development programs”

Special subtopic on Innovative Breakthroughs for Modeling, Analysis, and Prediction:•Use of Integrated Development Environments•Refactoring of legacy codes•Enhancing software engineering practices to reduce errors•Software standards towards common climate modeling infrastructure

Behind the Scenes:Smart, Framework-awareCode Editing Environment

Structural Pattern Expression Structural Elements Matched

module Matches a Fortran module

m moduleName Matches the name of the module m

m subroutine Matches a subroutine defined inside module m

m subroutine(name(type1, type2, ...))orm subroutine(*(*))

Matches a subroutine defined inside module m with the given signature. The wildcard * can be used in place of the name and/or types.

s subroutineName Matches the name of subroutine s

s formalParam(i) Matches the ith formal parameter of subroutine s

Mapping Types

Behind the Scenes:Smart, Framework-awareCode Editing Environment

Structural Pattern Expression Structural Elements Matched

s call Matches a subroutine call within the implementation of subroutine s

s call(name(arg1, arg2, ...)) Matches a subroutine call within the implementation of subroutine s to a subroutine with the given name and given set of argument.

c argByIndex(i) Matches the ith argument value for the call c

c argByKeyword(keyword) Matches the value of the argument with the given keyword for the call c

Mapping Types (cont.)

Behind the Scenes:Smart, Framework-awareCode Editing Environment

NUOPC Driver

driverName

standardNUOPCImport

callsGenericSetServices

module(#driverName)

usesModule(NUOPC)

standardESMFImport usesModule(ESMF)

implementsSetServices subroutine(*(inout type(ESMF_GridComp) #p_gcomp, out integer #p_rc) )

... call(#^genericImports/importsGenericSS

(#^p_gcomp, #^p_rc))

genericImports usesModule(NUOPC_Driver)

importsGenericSS usesEntity(routine_SetServices)

The Earth System Modeling Framework (ESMF) was developed as a multi-agency response to calls for common modeling infrastructure.

ESMF delivered:•Standard interfaces for model components•High performance libraries and tools for time management, data communications, metadata and I/O, and parallel grid remapping

The Earth System Modeling Framework

Metrics:~5500 downloads

~3000 individuals on info mailing list

~40 platform/compilers regression tested

~6400 regression tests

http://www.earthsystemmodeling.org

1. Implementations of ESMF could vary widely and did not guarantee a minimum level of technical interoperability among sites

• The National Unified Operational Prediction Capability (NUOPC) is a consortium of operational weather prediction centers and their research partners

• NUOPC developed the NUOPC Layer software to increase interoperability of ESMF components

• The NUOPC Layer adds to ESMF:• A formalism that describes and splits the phases of initialization• A formalism for checking and reporting whether component requirements

are satisfied during the run sequence• Code templates for drivers, models, mediators (couplers) and connectors,

and example application showing a variety of model interactions (e.g. explicit, semi-implicit, implicit coupling)

• Compliance checkers

The National Unified OperationalPrediction Capability

2. It was difficult to track who was using ESMF and how they were using it

• The Earth System Prediction Suite (ESPS) is a collection of weather and climate modeling codes that use ESMF with the NUOPC conventions.

• The ESPS makes clear which codes are available as ESMF components and modeling systems.

ESPS on CoG:

https://www.earthsystemcog.org/projects/esps

The Earth System Prediction Suite

Currently, components in the ESPS can be of the following types:coupled system, atmosphere, ocean, wave, sea ice

Target codes include:• The Community Earth System Model (CESM)• The NOAA Environmental Modeling System (NEMS) and Climate Forecast System

version 3 (CFSv3)• The MOM5 and HYCOM oceans• The Navy Global Environmental Model (NavGEM)-HYCOM-CICE coupled system• The Navy Coupled Ocean Atmosphere Mesoscale Prediction System (COAMPS) and

COAMPS Tropical Cyclone (COAMPS-TC)• NASA GEOS-5• NASA ModelE

Model Codes in the ESPS

ESPS creates opportunities for the Cupid team:• There is an opportunity for the GEOS-5 and Model E codes to become more

interoperable with other codes in the community, and each other.o The NUOPC Layer and the Modeling Analysis and Prediction Layer (MAPL) that GEOS-5

uses are similarly structured and share use of ESMF as an underlayer. A report details the relationship between the two software packages, here: http://www.earthsystemcog.org/site_media/projects/nuopc/paper_1401_nuopc_mapl.docx

o The ESMF and GMAO teams plan to incorporate MAPL into the ESMF distribution, in much the same way that NUOPC is bundled with ESMF.

o This will create opportunities for reducing redundancy and further merging the NUOPC Layer and MAPL packages.

o A translator will be written to convert between the interfaces.

• There is an opportunity for training users across ESPS modeling systems.

ESPS and Cupid

ESPS Code Status

  LEGEND              Compliant (Completion date) In progress   Candidate             

Coupled Modeling Systems  NEMS CFSv3 COAMPS /

COAMPS-TCNavGEM-HYCOM-CICE

GEOS-5 ModelE CESM

  2014 2014     2015 2015 2014Atmospheres

GFS/GSM              NMMB              CAM             2014FIM 2014            GEOS-5 FV         2015    ModelE Atm           2014  COAMPS Atm              NavGEM              NEPTUNE              WRF             2014

OceansMOM5              HYCOM              NCOM              MPAS-O              POP             2014

IceCICE 2014 2014     2014 2014 2014

WaveWW3 2014   2014   2014   2014SWAN              

Spanning major climate, weather, and ocean codes, ESPS is the most directresponse to calls for common modeling infrastructure yet assembled

Behind the Scenes:Smart, Framework-awareCode Editing Environment

Behind the Scenes:Cloud-based Training Environment

NESII Amazon Elastic Compute Cloud (EC2) account:•preconfigured machine images – NUOPC and ModelE (EM20 rundeck)•can request access to NESII cloud account, or•can provide your own EC2 account credentials

XML-based training scenario configuration:

Slide courtesy Fatimah Wirth and Kristen Anderson, Georgia Tech Professional Education

What we do:

Professional Master’s (since 2007)Distance Learning:

MS degrees (since 1977)UG courses (since 2013)

Language Institute ESL (since 1958)Veterans Training Programs (since 2012)Professional Education certificates (since 1930s)MOOCs w/Coursera and Udacity (since 2013)OMS CS w/Udacity (Computer Science MS for $7K)K-12 Outreach (NASA ePDN, STEM activities)Event/Conference Facility & Services (since 2003)Savannah Campus (since July 2011)

Georgia Tech Professional Education Overview

Slide courtesy Leo Mark, Georgia Tech Professional Education