COLLABORATIVE DESIGN AND DEVELOPMENT OF THE

COMMUNITY CLIMATE SYSTEM MODEL FOR TERASCALE

COMPUTING (CDDCCSMTC)Hereinafter referred to as the CCSM Consortium

Phil Jones (LANL)

On behalf of all the consorts

The SciDAC CCSM Consortium consists of PI: R. Malone4, J. Drake5 , Site-Contacts: C. Ding2, S. Ghan6, D. Rotman3, J. Taylor1, J. Kiehl7, W. Washington7, S.-J. Lin8, Co-Is: J. Baumgardner4, T. Bettge7, L. Buja7, S. Chu4, T. Craig7, P. Duffy3, J.

Dukowicz4, S. Elliot4, D. Erickson5, M. Ham5, Y. He2, F. Hoffman5, E. Hunke4, R. Jacob1, P. Jones4, J. Larson1, J. Lamarque7, W. Lipscomb4, M. Maltrud4, D. McKenna7, A. Mirin3, W. Putman8, W. Sawyer8, J. Schramm7, T. Shippert6, R. Smith4, P.

Worley5, W. Yang2

 1Argonne National Lab, 2Lawrence Berkeley National Lab, 3Lawrence Livermore National Lab, 4Los Alamos National Lab,

5Oak Ridge National Lab, 6Pacific Northwest National Lab, 7National Center for Atmospheric Research, 8NASA-Goddard Space Flight Center

Science Goals• Assessment and prediction

– IPCC, national assessments (alarmist fearmongering)

– Energy policy (Dick Cheney’s private sessions)• Regional climate prediction

– High resolution, downscaling, water!• Atmospheric chemistry/ocean

biogeochemistry– Carbon cycle– Aerosols

Project Goals• Software

– Performance portability– Software engineering (repositories,

standardized testing – No Code Left Behind initiative)

• Model Development– Better algorithms– New physical processes (esp. chemistry,

biogeochemistry)

Coupler ArchitectureIssues:Issues: •sequencingsequencing•frequencyfrequency•distributiondistribution•parallelism parallelism •single or multiple single or multiple executablesexecutables•stand alone executionstand alone execution

Version 1.0 ReleasedNovember 2002

• MPH3 (multi-processor handshaking) library for coupling component models

• CPL6 -- Implemented, Tested, Deployed

• ESMF/CCA

Performance Portability• Vectorization

– POP easy (forefront of retro fashion)– CAM, CICE, CLM

• Blocked/chunked decomposition– Sized for vector/cache– Load balanced distribution of blocks/chunks– Hybrid MPI/OpenMP– Land elimination

• Performance modeling w/PERC

Performance

Regional Prediction

Mississipi State

Stanford

Kentucky

Oklahoma State

Atmosphere/Land

Subgrid Orography Scheme

• Reproduces orographic signature without increasing dynamic resolution

• Realisitic precipitation, snowcover, runoff

• Month of March simulated with CCSM

Eddy-Resolving Ocean

0.1 deg0.28 deg

Obs 2 deg

Greenhouse Gases• Energy production• Bovine flatulence• Presidential campaigning

•Source-based scenarios

Aerosol Uncertainty

Atmospheric Chemistry• Gas-phase chemistry with emissions, deposition, transport and photo-

chemical reactions for 89 species. • Experiments performed with 4x5 degree Fvcore – ozone concentration at

800hPa for selected stations (ppmv)• Mechanism development with IMPACT

– A)    Small mechanism (TS4), using the ozone field it generates for photolysis rates.

– B)     Small mechanism (TS4), using an ozone climatology for photolysis rates.

– C)    Full mechanism (TS2), using the ozone field it generates for photolysis rates.

Zonal mean Ozone, Ratio A/C

Zonal mean Ozone, Ratio B/C

Ocean Biogeochemistry• LANL Ecosystem Model

– nutrients (nitrate, ammonium, iron, silicate)– phytoplankton (small, diatom, coccolithophores)– zooplankton– bacteria, dissolved organic material, detritus– dissolved inorganic carbon (DIC), alkalinity– trace gases (dimethyl sulfide, carbonyl sulfide,

methyl halides and nonmethane hydrocarbons)– elemental cyclings (C,N,Fe,Si,S)

Ocean Biogeochemistry

•Iron Enrichment in the Parallel Ocean Program•Surface chlorophyll distributions in POPfor 1996 La Niña and 1997 El Niño

Global DMS Flux from the Ocean using POP

The global flux of DMS from the ocean to the atmosphere is shown as an annual mean. The globally integrated flux of DMS from the ocean to the atmosphere is 23.8 Tg S yr-1 .

Potential U.S. ParticipantsRuns completed• Starley Thompson LLNL, David Erickson ORNL (PCM-IBIS)

C-cycle code completed, tested coupled to relevant GCM• Inez Fung, Scott Doney UC Berkeley (CCSM1-OCMIP2-CASA derivative)• Fung, Hoffman, Doney, Lindsay (CCSM3-CLM3-CASA’ )?

C-cycle code completed, run off-line completed• Gordan Bonan NCAR (CCSM2-LPJ derivative - see Bonan et al. 2003. GBC

11:1543-1566)• Joerg Kaduk Stanford (CCM? Or UCLA-SiB2)• Robert Dickinson Georgia Tech (CCSM?-CLM enhanced)

C-cycle code under development• Peter Thornton NCAR (CCSM3-Biome-BGC derivative)• Erickson, Post, King, Gu ORNL (PCM-IBIS-GTEC loose coupling) [acclimation,

moisture profile, diffuse radiation effects on veg]• Ocean POP - OBGCM LANL (CCSM3-POP)• DOE Deliverables: aerosol chemistry-carbon coupled model (CCSM3-IBIS)

Extensions for Carbon CycleDecadal-Cent.Priority

CarbonDynamicsGPPRespirationAllocation YESDecompositionNitrogen dynamics YESTrop Ozone effects YESTemp. acclimation YESCO2 acclimation YESDiffuse solar effects YES

Decadal-Cent.Priority

VegetationDynamicsDisturbance-recovery

YES

BiogeographyLand use/land coverchange

YES

HydrologicalEffectsConsistent w/ LSMGrowthForest Mortality YESDecompositionETWetlands-trace gases

Accept CAM chemistry package from SciDAC (July 2004)• Meet with NCAR BGC working group to mesh science plans of this project to those of

the NCAR BGC working group (July 2004)• Perform additional CAM/FV vectorization enabling chemistry/sulfur simulations on the

X1 (Oct 2004)Accept CCSM3/IBIS model from LLNL LDRD (Oct 2004)• Implement sulfur chemistry in CAM ozone photochemistry package (Oct 2004)• Enable ocean produced DMS emissions to be processes through coupler and into CAM

atmospheric physics/chemistry package (Oct 2004)

Where we would like to beAccurate regional modeling of carbon sources and sinksCoupling of biogeochemical processes for climate variability on decadal to century time scales