transp and frameworks presented by d. mccune at the joint ornl/iu workshop on computational...
TRANSCRIPT
![Page 1: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/1.jpg)
TRANSP and Frameworks
Presented by D. McCune at the Joint ORNL/IU Workshop on Computational
Frameworks in Fusion,
June 6, 2005
![Page 2: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/2.jpg)
Outline of Talk
• TRANSP Background Information:– What is TRANSP; Applications of TRANSP.– Users, Developers, Funding profile.
• FusionGrid TRANSP.
• TRANSP Code Generators – Python.
• TRANSP Makefile Generators – Development Environment.
• Requirements for Frameworks.
![Page 3: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/3.jpg)
What is TRANSP?• “1.5D” time dependent tokamak transport
simulation tool:– 2d axisymmetric tokamak reduced to 1d by
using standard “flux coordinate” methods.
• Major application: analysis of experimental data from tokamaks.
• Major strengths:– Physics: e.g. validated fast ion models.– Operations: effective workflow & support.
• Under continuous development since 1978!
![Page 4: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/4.jpg)
Applications of TRANSP
• Transport and confinement studies (with or without ITB, H-mode, etc.).
• Neutral beam heating simulation; computation of fast ion distributions.
• RF heating and current drive simulations.
• Diagnostics simulations.
• Predictive calculations for experimental proposals.
![Page 5: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/5.jpg)
Applications of TRANSP (cont.)
• Many groups report use of TRANSP results in conference papers and refereed journal articles.
• Two examples follow:– Tritium Transport on JET (I. Voitsekovitch).– NPA Simulations on NSTX (S. Medley).
• Numerous other examples could have been chosen.
![Page 6: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/6.jpg)
TRANSP at JET
Trace Tritium: Thermal Tritium Transport:
14 MeV neutrons measured along 19 chords (left) are reproduced in TRANSP by using a proper adjustment of Tritium transport coefficients (right, 6 bottom chords are shown)
2 techniques have been used: (1) Dt and Vt are adjusted in TRANSP; (2) novel use -
combined TRANSP/SANCO analysis -plasma reactivity is extracted from TRANSP using 20 runs and transmitted to SANCO for automatic estimation of Dt and Vt by
minimising 2-value
Publications: K.-D. Zastrow and JET team (EPS 2004, PPCF, 2004 in press), I. Voitsekhovitch, et
al., Trace Tritium transport in H-mode JET plasma with different density, JET pinboard, waiting for clearance, to be submitted to Phys. Plasmas and also in EPS 2004; J. Mailloux et al, EPS 2004; P. Belo et al., EPS 2004; D. Stork and JET team, IAEA 2004; D. Stork and JET team, APS 2004
![Page 7: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/7.jpg)
MHD-Induced Fast Ion Losses in NSTX
SN108730
ln( N
PA F
lux/
Ene
rgy
1/2
) (
ster
-1cm
-2eV
-3/2
s-1
)
Energy (keV)0 20 40 60 80 100
t = 400 ms During H-mode
TRANSP
NPA
Ion Loss Region
15
19
18
17
16
20
21
14
13
12
TRANSP WITH FAST ION DIFFUSION - 108730M07
TRANSP WITHOUT FAST ION DIFFUSION - 108730M08
TRANSP
Normalization
Sn
(x10
13 s
-1)
Time (s)
SN108730
H-Mode Onset
0 0.2 0.4 0.6 0.8
MHD Onset
2
0
4
8
6
10
12
14
16
18
20
22
Measure Neutron Rate
TRANSP with Fast Ion Diffusion
TRANSP without Fast Ion Diffusion
S. Medley et al., Nucl. Fusion 44(2004)
![Page 8: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/8.jpg)
TRANSP Developers and Users
• PPPL TRANSP team:– R. Andre, A. Brooks, F.
Dahlgren, E. Feibush, K. Indireshkumar, S. Klasky, L. P. Ku, C. Ludescher, D. McCune, L. Randerson
• ~5 FTE for TRANSP & related efforts.– Major focus on
production & support…
• User Sites:– Culham (MAST)– GA (DIII-D)– IPP/Garching (Asdex-
U)– JET– MIT (C-Mod)– PPPL (NSTX)– PPPL
(Collaborations)
![Page 9: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/9.jpg)
FY05 PPPL TRANSP Funding
• NSTX/MAST (approx. 2.15 FTE, 60%)• DIII-D collaboration (0.9 FTE, 25%)• C-Mod collaboration (0.25 FTE, 7%)• JET collaboration (0.25 FTE, 7%)• Tokamak Projects Total: 3.55 FTE• R&D Projects (1.45 FTE):
– PTRANSP (predictive upgrade), 0.6 FTE– SciDAC FusionGrid (2 year renewal grant),
0.85 FTE
![Page 10: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/10.jpg)
Outline of Talk
• TRANSP Background Information.
• FusionGrid TRANSP:– TRANSP as a Computational Service.– TRANSP Workflow; Workflow Automation.– Future: Linked, Distributed Services.
• TRANSP Code Generators – Python
• TRANSP Makefile Generators – Development Environment.
• Requirements for Frameworks.
![Page 11: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/11.jpg)
FusionGrid TRANSP
• TRANSP runs as a service on PPPL Linux cluster machines.– User authentication via Globus.– Credit to SciDAC Fusion Collaboratory Project.
• Users submit run requests via supported client software tools.
• Users can monitor system load and watch progress of runs, access log files, etc.
• Website: http://w3.pppl.gov/TRANSP ...
![Page 12: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/12.jpg)
TRANSP Client Tools, Services
• Prepare Input Data.
• Submit runs.
• Monitor run progress; view log files.
• Fetch interim data.
• Option to cancel run prior to completion.
• Explore, visualize output data.
• Crashed runs debugged by TRANSP support staff at PPPL.
![Page 14: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/14.jpg)
FusionGrid TRANSP Annual Run Production
0
500
1000
1500
2000
2500
3000
Total Runs
through10/31/2002*
11/1/2002 --10/31/2003
11/1/2003 --10/31/2004
*FusionGrid TRANSP Operational Since September 2002
![Page 15: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/15.jpg)
FusionGrid TRANSP Workflow
Preliminary dataAnalysis andPreparation
(largely automated)
DiagnosticHardware
Experiments (CMod, DIII-D, JET, MAST, NSTX)
20-50 signals {f(t), f(x,t)}Plasma position, Shape,Temperatures, DensitiesField, Current, RF andBeam Injected Powers.
TRANSP Analysis*:Current diffusion, MHD equilibrium, fast ions,
thermal plasma heating; power, particle and
momentum balance.
TRANSP Analysis*:Current diffusion, MHD equilibrium, fast ions,
thermal plasma heating; power, particle and
momentum balance.
Experiment simulationOutput Database
~1000-2000 signals{f(t), f(x,t)}
Visualization
Load RelationalDatabases
Detailed (3d) time-slice physics simulations (GS2, ORBIT, M3D…)
*FusionGrid TRANSP now executes on PPPL servers;GS2 and other grid services are planned for the future.
Pre- and Post-processing at the experimental site…
![Page 16: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/16.jpg)
Workflow Automation
• User-programmable automated data preparation is key to system productivity.– IDL (many legacy tools in fusion program).– UREAD/SGLIB and other legacy fortran tools*.
• Also important: – convenient tools for submission and web-
based monitoring of runs.– User-programmable automated visualization,
database loading, and further post-processing.
D. McCune 23 Apr 2004
*available at http://w3.pppl.gov/NTCC
![Page 17: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/17.jpg)
TRANSP Workflow Caveats
• The software technology used is ancient.– But: familiar to current users.
• The workflow volumes (dataset sizes) are small, so far.– So, no motivation yet for transition to high
performance solutions.
• The potential for generation of larger datasets exists.– Current practice may be a limiting factor.
![Page 18: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/18.jpg)
TRANSP FusionGrid Future: Linked, Distributed Services.
Serial TRANSP at PPPL
Parallel RF Wave Solver
(TORIC at MIT)
Parallel MonteCarlo Fast Ion
Model(NUBEAM at
PPPL)
• Multi-site Globus Forwarded Authentication• Possible issues:
• Firewalls• Resource sharing
• Queue wait time.• Data transfer times.
![Page 19: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/19.jpg)
Outline of Talk
• TRANSP Background Information.
• FusionGrid TRANSP.
• TRANSP Code Generators – Python:– Ad hoc data specification languages.– Python code generators.
• TRANSP Makefile Generators – Development Environment.
• Requirements for Frameworks.
![Page 20: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/20.jpg)
Code Generators
• Many i/o interfaces of TRANSP are written by code generators (Python scripts).– Measured once to be ~25% of fortran lines.
• Code generator input: list of data items.
• Code generator output:– Anything needing to be driven by the list:
• Fortran namelists, “state” or restart file i/o routines, documentation, allocate/deallocate routines, default/initialization routines, plot labeling routines, Fortran-90 type definition modules, debugging…
![Page 21: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/21.jpg)
Ad Hoc Languages
• Custom, ad hoc, requirements-driven “language” for each list.
• Example: NTCC NUBEAM “nbspec.dat”*:– Sections: Array_Dimensions, Constants,
Inputs, Internal, Outputs.– “nbigen.py” writes code for internal data
structures, public modules defining input/output data structures, input default settings, state file i/o, html documentation, …
*see http://w3.pppl.gov/NTCC
![Page 22: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/22.jpg)
Ad Hoc LanguagesSection: Inputs # from nbspec.dat… … …Block: Atomic ! Atomic physics controls (defaults shown).D xdepmod = 1.0d0 ! Anomolous deposition opacity adjust.L nlbbcx = .TRUE. ! .FALSE. to disable beam-beam ch.ex. … …Section: Outputs … …orb%DAS sorbn0(mj,mig,mibs) ! Thermal neutral source ! due to deposition or charge exchange recapture of fast ! neutrals, vs. x, thermal ion, fast ion; “orb” MPI_reduce.
Key: D means REAL*8, L means LOGICALA means ARRAY, S means “in State File”,…
![Page 23: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/23.jpg)
Code Development Automation
• Code generators organize i/o and “book-keeping” aspects of code.– They make it easy to add inputs & outputs:
• (a) edit specification file.• (b) run generator python script.
– Less error prone than hand-maintained methods, where list-driven requirements exist.
– Allows greater focus on physics coding.
• Key to long term success of TRANSP.
![Page 24: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/24.jpg)
Python Chosen for Code Generators
• Early TRANSP code generators were written in fortran-77 or C, but…
• Much easier / faster to work with Python.– Python “dictionary” and “list” data structures
perfect for code generator application.– Fast prototype / debug cycle.– Python has excellent error reporting.– Python has excellent tutorial and reference
documentation (e.g. O’Reilly books).
![Page 25: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/25.jpg)
Outline of Talk
• TRANSP Background Information.
• FusionGrid TRANSP.
• TRANSP Code Generators – Python
• TRANSP Makefile Generators – Development Environment:– Makefile generator system.– Resulting benefits for code developer.– Issues / Limitations.
• Requirements for Frameworks.
![Page 26: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/26.jpg)
TRANSP Developers do not edit Makefiles…
• Makefiles are written by makefile generator for:– Subroutine libraries (179).– Executables (220).
• Supported by script database with information on various fortran compilers…
• Library makefiles based on contents of library source directory:– F90 module dependency analysis included.
![Page 27: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/27.jpg)
TRANSP Makefile Generator
• Linkage for executables defined by separate file containing ordered list of objects and libraries:
> cat foo_exe.linkfoo_main.ofoo_sub1.afoo_sub2.a$L_NETCDF$L_LAPACK$L_FFTW
Elements:• .o – from TRANSP source.• .a – from TRANSP source.• $L_<name> -- non-TRANSP dependency.
![Page 28: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/28.jpg)
Command Line Interface
• cmsmms <libname> -- generate makefile for library.
• uplib <libname> -- update (build) library.• makelink <progname> -- generate
makefile for executable.• uplink <progname> -- update executable.• These commands build “standard” (i.e.
non-debug) libraries and executables.
![Page 29: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/29.jpg)
Debug builds• dbxadd <source-name> -- add single
source to debug list.– F90 module dependencies added
automatically.
• dbxaddlib <libname> -- add entire library to debug list.
• uplink <progname> debug – build debug executable.– Makefile for custom debug executable is built;
only named items are debug compiled.
![Page 30: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/30.jpg)
Benefits
• Relieves developer of makefile maintenance requirement.– Hand built makefiles become nightmares…
• Precise control of generation of debug executables.
• Easy to learn.
• Saves time.
![Page 31: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/31.jpg)
Drawbacks / Limitations
• Monolithic – not easily transferable outside the TRANSP project.
• Too much dependency on ancient csh scripts (awkward to maintain).
• Certain peculiarities, due to a lapsed VMS compatibility requirement, persist…
• We are looking at SCONS for possible upgrade or replacement…
• Advice of CS experts would be welcome!
![Page 32: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/32.jpg)
Outline of Talk
• TRANSP Background Information.
• FusionGrid TRANSP.
• TRANSP Code Generators – Python
• TRANSP Makefile Generators – Development Environment.
• Requirements for Frameworks.– Generic Requirements.– Requirements Checklist.– Compatibility with CS Research (?).
![Page 33: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/33.jpg)
Generic Requirements
• Non-invasive; light-weight.– No motivation for massive project to convert
an existing, working system.– Evolutionary method needed for introduction
of new technology.
• Simple things must be simple, complicated things must be possible.
• Well engineered.– Robustness, ease-of-use, documentation…
![Page 34: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/34.jpg)
Requirements Checklist
• Technical assessment:– Computational resources needed.– Data management issues to be faced.
• Sociological assessment:– Define benefit to end user.– Define cost to end user (e.g. skills to learn).– Define benefit to code developer.– Define cost to developer (e.g. skills to learn).
![Page 35: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/35.jpg)
Requirements vs. CS Research
• Are the deployment requirements for a production scientific application compatible with CS research constraints?– How are engineering issues handled:
robustness, error handling, documentation…– Who finances such expensive work?
• Shouldn’t production deployment be viewed as the “experimental program” for software technology research?
![Page 36: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/36.jpg)
Example of adoption of new software technology in the
TRANSP project: introduction of Python for code generator
applications.
![Page 37: TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649e2d5503460f94b1c5a2/html5/thumbnails/37.jpg)
The End.
Hopefully, the perspective derived from TRANSP experience will be helpful. But, it is only one perspective…