CCSM Software Engineering Update

Tony Craig

CCSM SEWG Meeting

Feb 4, 2003

Outline

• CCSM SE Organization / CSEG Staff• CCSM Release Status• Process Update• ESMF• SciDAC• CPL6 Update• CLM2.1 Update• New Projects:

– GUI and DataMover– Document control– Data models rewrite

• Testing status• Vectorization

CCSM Entities

• SSC = Scientific Steering Committee• WG = Working Groups (about 8 total)• SEWG = Software Engineering Working Group• CSEG = CCSM Software Engineering Group• ESMF = NASA funded coupling framework project• SciDAC = DOE funded project to improve CCSM

CCSM Organization(SE perspective)

SSC

SEWG WG

WG WG

WG

WG

WGWG

WG

ES

MF

SC

IDA

C

CSEG

CSEG Staff Changes

• 9 Software engineers• 2 new hires, Spring, 2002

– Wei Yu - performance and testing– Dan Miner - infrastructure

• 1 reclassification, Spring 2002– Erik Kluzek - SEIII (ESMF)

• Lawrence Buja moved to CCR, Fall 2002• 1 new hire, Dec 2002

– Tom Henderson - SEIII• 1 open position (SEII/III)

CSEG staff

• Tony Craig (CCSM integration)• Nancy Norton (ocean model, data models)• Tom Henderson (atmosphere model)• Brian Kauffman (coupler model)• Erik Kluzek (ESMF, atmosphere model)• Dan Miner (infrastructure)• Julie Schramm (ice model, doc project)• Mariana Vertenstein (land model, GUI)• Wei Yu (testing, GUI)

CCSM2.0 Release Status

• CCSM2.0 released May 17, 2002– Source code– Testing– Documentation– Input datasets, control run output (b20.007)– Web page development– Patch 1, 2, 3, 4– Tutorial

CCSM2.0+ Release Status

• CCSM2.0.1 release scheduled mid-July, 2002 Oct 2002– Bit-for-bit with CCSM2.0 for b20.007 control– Includes paleo version– Includes updated datm and dlnd components and forcing

datasets– Other minor SE enhancements– Patches

• CCSM2.1 release expected Fall, 2002– Will improve the scientific results– Other SE enhancements

CCSM2.1+ Release Status

• CCSM2.1beta release expected March 2003– “beta” release - no control, no long-term support– CPL6– CLM2.1– Minor physics changes

• CCSM2.2(?) release expected Fall 2003– IPCC version– full support

Process Update

• Release policy, support (2+1 years, patches)• CCSM CRB• CAM CRB• CVS Repository access• Infrastructure

– New CVS server, Update CVS– New bug tracking tool– Database (experiments, bug tracking)– Integration of CVS, bug-tracking, GUI, database

• Testing Brown Bag Discussions

ESMF

• Continue to use ESMF pre-release library in CCSM• Close working relationship with ESMF Core Team• Design and prototyping• Writing pseudo-code• ESMF will not be integrated into the production

system until usability, robustness, and performance are clearly demonstrated

• Erik Kluzek leading ESMF effort in CCSM

SciDAC

• CAM improvements• CAM/CLM coupling• CLM runoff model• CPL6• POP2• CSIM/CICE updates

• Cray X1

CCSM GUI Project Goals

• Greater flexibility in building and running CCSM on a variety of platforms, machines, and configurations; will set better default task and thread counts; will handle batch queues better

• Increased error checking• Capability to edit component namelists• Will provide capability of starting from previous

experiments (including modified source code) and modifying those experiments or creating new experiments

CCSM GUI Project Goals (2)

• Capability to build, pre-stage input data, or run the CCSM independently

• Written in java (can run on a much larger number of platforms).

• New DataMover tool– Functionality to move a file, a list of files or a list of

directories from one location to another by a variety of methods that may be user-prescribed

– Very Flexible– Will support recursive directory movement

CCSM GUI Project Status

• Requirements document exists– Several month process gathering and refining

requirements• Prototyping and Design underway

– Regular internal reviews• Mariana Vertenstein leading effort

Document Project Overview

• Motivation: – As more documentation is being written for CCSM,

documents need to become better organized• CCSM Documents include:

– Version specific documents (Users Guides, …)– Component general documents (Requirements Doc, …)– CCSM general documents (Developers Guide, Test Doc, …)– Document templates

• Goals of project:– Provide a central location for all CCSM documentation– Get all CCSM documentation under revision control– Create an online library where all CCSM documents will be

accessible

Document Project Status

• There are requirements for the document organization• Five proposals have been presented, based on these

requirements.– Two proposals would use current CCSM CVS module

• Easiest to implement• Constrained by directory structure of current repository

– Three proposals create a separate CVS module for documentation

• Source code and document tagging more complex• New directory hierarchy could be a desirable option

• Julie Schramm leading effort

Data Model Project Background

• The CCSM Data Models were originally created to serve as simple "stand-ins" for their active-model counterparts.

• They shared common functionalities, structure, and coding style.

• Individual data models have been modified by developers and scientific groups for science capability, SE left behind.

• Current data models lack consistency and are difficult to upgrade, use, and maintain.

• Why update models now?– Scientist frustrations– Platform for ESMF prototyping– Take advantage of cpl6 share code– Opportunity to develop new share code

Data Model Project Status

• Gathering and recording requirements for data models– Review status of each requirement. – Solicit input from the scientific groups who

maintain specialized data model code.• Create a design plan for the data models, isolating as

much as possible those elements common to all data models. Develop a library of shared code.

• Nancy Norton leading effort

CPL6 update

• CPL5 is still the production coupler• Active and data components can couple with CPL6• New dead component available for testing• Completing validation comparison tests with CPL5• Will begin longer production run soon• Assessing performance• Same science• Expected to be the CCSM production coupler in

March, 2003

Mapping: ocn -> atm

0

0.5

1

1.5

2

2.5

3

3.5

4

1 2 4 8 16 32

number of pes

secscpl6cpl5

Ocn (122,880 points -> Atm (8192 points)bundle of 9 fields

120 mapping calls

CLM2.1 update

• Complete rewrite of CLM2.0– introduction of new data structures– introduction of new history output mechanism

• Improved flexibility for incorporation of new land model physics (e.g. biogeochemistry)

• Upgrade to CAM/CLM coupling• Same performance in CCSM• Same science• CLM2.1/CAM2.0.2 release expected soon

Testing Status

• Weekly testing of CCSM2.0, CCSM2.0.1 releases on chinook, blackforest, bluesky, seaborg

• Automatic testing of CAM every night after a commit on the primary development branch

• Formal Test Plan Document under development

Vectorization update

• “Vectorization is NOT a high priority” (SSC)• Earth simulator still primary target for vectorization• CAM/CICE rewrites happening in community• CLM rewrite will start after CLM2.1 release,

community effort• CPL rework will start after CCSM2.1 release (cpl6),

community effort• New opportunity on CRAY X1, DOE taking lead

Summary

• IPCC release next major target, Summer 2003.• New projects

– CCSM GUI and DataMover– Documents organization– Data models rewrite

• CPL6 and CLM2.1 “complete”• Better testing• Continue to improve process