cgns and pycgns, its python binding poinot marc onera - dsna/elsa /elsa/prs-02036 [email protected]
TRANSCRIPT
CGNSAnd pyCGNS, its Python binding
Poinot MarcONERA - DSNA/ELSA
/ELSA/PRS-02036
http://www.onera.fr
/ELSA/PRS-02036
Overview
CGNSGoals
History
Actors/ Components
Examples
Future/ ISO process
pyCGNSGoals/ Status
Interoperability architecture
Examples
Proprietary data format and semantics
ISO data format and semantics
/ELSA/PRS-02036
CFD (Computational Fluid Dynamics) General Notation System
CFD standardWe want to use a common data representation for interoperability
• Data semantics• Physical representationGrid creation/ Solvers/ VisualizersPre/Post processing of very large amount of data
We want to ensure our data a long life format• Consistency• Timelessness
Provide a general, portable and extensible standard for storage and retrieval of CFD analysis data.
/ELSA/PRS-02036
Brief history
Started 1994NASA + Boeing to improve technology transfert
NASA contract 1995-1998SIDS document written by S.Allmaras, Boeing
Library implementation (+ICEM +NPARC Alliance)
Steering Committee 1999…Gather people
• 17 members• +500 users• web site
Software and documentation releases
ISO Process• Initiated and supported by Boeing
/ELSA/PRS-02036
Steering Committee
Airbus
Boeing commercial
Boeing Phantom Works
Boeing Space & Communications
CD ADAPCO Group
Fluent, Inc.
ICEM CFD Engineering
Intelligent Light
NASA Ames
NASA Langley
NASA Glenn
NUMECA International
ONERA
Pointwise, Inc.
Pratt & Whitney Aircraft
Rolls-Royce Allison
US Air Force
MissionEnsure the continuation of CGNS
ResponsabilitiesSoftware devt., maintenance, test…
Documentation
Insure a soft. and doc. free distribution
Provide expertise and extensions proposals
Promote acceptance of the standard
OrganizationMain meeting once per year
One chairperson per organization
Governs by consensus
Welcomes participation of all parties, members or not
/ELSA/PRS-02036
Overall architecture
Data semantics and Syntax
CGNSLib
File format
Node API
Specific Sub-trees API
/ELSA/PRS-02036
Components
DocumentationSIDS document
API and users documentation
Technical papers
Libraries/ sourcesMLL and ADF libraries (Fortran +C)
• Binaries• Sources
User tutorial examples
Data file examples
/ELSA/PRS-02036
Data exchange example - An informal view
Fluid solverMesh generator
Communication layer
Post processor
WB234
wing
body
coordX
bodyConnectivity
wingZones
solution#1
pressuredensity drag
/ELSA/PRS-02036
Example - A CGNS tree WB234
wing
coordXSolution#1
density drag
WB234CGNSBase_t
DescriptionWing and body test case 234
Descriptor_t
Date13/03/2002Descriptor_t
Zone_twing
ZoneType_tStructured
GridCoordinates_tCoordinateX
solution#1FlowSolution_t
DataArray_tDensity
UserDefined_tDrag
DimensionalUnits_tDimensionalUnits
DataArray_tdragCoef
/ELSA/PRS-02036
Example - An ADF node WB234
wing
coordXSolution#1
density drag
Name: DensityLabel: DataArray_t DataType: DataTypeDimensions: IndexDimension
Parameters: DataType, IndexDimension, DataSize
Cardinality: 0,N
DimensionValues: DataSize[]Data: [values]
Children: [nodes]
/ELSA/PRS-02036
Development process
CommunityMailing list+ Web site+ Users
Meeting in Reno (AIAA annual conf.)
Teleconferences (every 2 months)
ActorsStarted with ICEM CFD as main developper
Now Intelligent Light also
Members/ users contributions
Means/ processCVS server
Heavy testing/ Insure compatibility/ Check portability
/ELSA/PRS-02036
Extension process
Extending the standardThe SIDS and its implementation has been though extensible from start
Documentation and consensus first
ProcessExtension submission and discussions using mailing lists
Sub-group creation, seek for expertise, document drafting
Proposal for extension, implementation
Some in-progress extensionsPeriodic boundary condition/ Interfaces/ Rotating coordinates
Chemical species/ Multi-Phase and liquid spray/ Gravity
Object-oriented API/ In-memory tree representation
/ELSA/PRS-02036
ISO process
ISO/STEPApplication Protocol 237 - Fluid dynamics
Part 110 - Computational Fluid Dynamics
Part 52 - Mesh-based topology
Part 53 - Numerical analysis
ScheduleApproval as IS in the second half of 2003 for parts 110, 52 and 53
AP 237 second quarter of 2005
Drafts commentsPDF files + CGNS mailing list
See CGNS web site
/ELSA/PRS-02036
Our users
UsersAerospace community
• Industry
• Research (Academy, Govt.)
Software editors
• CAD, mesh generation
• Post-processing tools, viewers
Up to 500 registered users (25 countries)
No declared user in Asia or Japan
Sample usersBoeing
• Adopted CFD data format
• Many benefits
– Common data semantics
– Improved modularity
– Workflow optimization
• Conformance to CGNS for new CFD tools
Pratt & Whitney
• Translation tools between proprietary format and CGNS
• Improve workflow
• Automation of design processes
NASA Langley
• Add CGNS capabalities to CFL3D code
/ELSA/PRS-02036
Future directions
Evolve/ MaintainUse web site to scatter/gather information, examples, tools
Adds extension to meet users needs
Document more and more, tutorial, practical guides
Provide or refer to open source tools
SpreadISO
AIAA recommended practice
Emphasize on the use of CGNS in actual computations
Participate to conferences (including NASA/ESA Aerospace PDE workshop)
Keep source and docs as Open Source
/ELSA/PRS-02036
ONERA
The French aeronautics and space research centerFrom basic research to flight testing
All disciplines involved in aircraft, spacecraft and missile design
DSNAComputational Fluid Dynamics and Aeroacoustics Dept.
ELSA: Software unit in charge of elsA software
elsANew generation CFD solver
• Industrial tool• Framework for research• Gather 20 years of CFD methods and software expertise
Technologies leading edge• OO design and implementation (C++, Python)• Efficiency (scalar/vector, space/time)• High level QA
/ELSA/PRS-02036
CGNS related developments at ONERA
elsAMain I/O data representation
• SIDS compliant• Interoperability / Multi-processing
– In-memory and disk tree low level• Validation resources
pyCGNSA Python binding to CGNS
Used as gluing language for application framework• e.g. translators to CGNS trees
– From/To Tecplot and Aerospace actors data format
Open source
/ELSA/PRS-02036
elsA Software
InteroperabilityMulti-platform/ Multi-applications
• Fluid/ Structures/ Optimization/ Thermics/
pre/post processing know how
TechnologiesOO design/ implementation
• C++ kernel• Fortran for computation leaves• Python user interface
A single representation for I/O• XDT trees + parsers for subset translation from/to
CGNS/ XML/ Python
/ELSA/PRS-02036
Python programming langage
Interpreted and OO script languageEasy to learn
Powerful and extensible• Shared libraries• Package system• Very large scope of applications
Portable• Including NEC, Cray, Fujitsu
Efficient modules for numerical applicationsAddress numerical simulation interoperability concerns
• Contiguous memory arrays• Can point to an already allocated zone• Optimized operations on whole data
/ELSA/PRS-02036
Ease of use example
a=1.4+0.2
b=[1.2, 0.3E-10, 1.4]
import Numeric
c=Numeric.ones([3])*a+b
print "Values :", c
import math
for d in c:
print math.cos(d)
Values : array([2.8, 1.6, 3.])
-0.942222340669
-0.0291995223313
-0.9899924966
/ELSA/PRS-02036
In-memory trees
GoalHave a in-memory representation of trees
• Communicate semantics using trees• Have a contiguous memory zone
– share– transfert
• Use memory buffer based systems– MPI/ SHM/ RPC/
• Fast dump to disk/ recovery
ImplementationADF API unchanged
• New implementation • Thanks the two layers design of CGNS libraries
/ELSA/PRS-02036
Architecture with Python modules
Tree on disk: File system/DBMS
Tree in-memory: Python interpreterRPC
SHMMPI
TCP/IP
/ELSA/PRS-02036
Interoperability example
Two solvers and glueStructure solver
Glue FFT+CGNS tree
Fluid solver
Visualization
Main loopStructure computation
Translator HOST/ CGNS• SPARK parser• Numerical Python (arrays+FFT)• pyCGNS tree
Fluid computation• Full CGNS
Tecplot viewer• Read CGNS
/ELSA/PRS-02036
Semantic/structure & efficiency
Same treeData consistency
"Blind" interoperability
Space/time efficiency
Ease of use
/ELSA/PRS-02036
Archival example
Validation databaseRDBMS+ proprietary tool
• Store CGNS file as binary large object – PostgreSQL blob
• Add attributes
Extracted from tree• Hidden to MLL
RDBMS extra features• Access control• Query• Clever archival server
/ELSA/PRS-02036
Code example - pyCGNS#!/usr/bin/env python# CFD General Notation System - CGNS lib wrapper# ONERA/DSNA/ELSA - [email protected]# User's Guide to CGNS - C.L.Rumsey et al. examples translation#from CGNS import midlevelfrom CGNS.wrap import *from Numeric import *#
# open CGNS file for read-onlyfile=pyCGNS('grid.cgns',midlevel.MODE_READ)
index_base=1index_zone=1index_flow=1isize=file.zoneread(index_base,index_zone)[3]irmin=[1,1,1]irmax=[isize[0],isize[1],isize[2]]
(loc,name)=file.solinfo(index_base,index_zone,index_flow)
if (loc != midlevel.Vertex): print "Error, GridLocation must be Vertex! Currently:", print midlevel.GridLocation[loc]
# read flow solutionr=file.fieldread(index_base,index_zone,index_flow, 'Density',midlevel.RealSingle,irmin,irmax)p=file.fieldread(index_base,index_zone,index_flow, 'Pressure',midlevel.RealSingle,irmin,irmax)
# close CGNS filedel fileprint "Successfully read flow solution from file 'grid.cgns'"print "For example, r,p(21,17,9)=",r[20,16,8],p[20,16,8]print "Program successful... ending now"
/ELSA/PRS-02036
Parts of XML tree(ADF tree dump using parseTree.py on web site example multi.cgns)
<NODE name='ADF MotherNode' label='Root Node of ADF File'>
<DATA type='MT' />
<DIMENSIONS total='0' />
<NODE name='CGNSLibraryVersion' label='CGNSLibraryVersion_t'>
<DATA type='R4' />
<DIMENSIONS total='1'>
</NODE>
</NODE>
<NODE name='InputFileUsed 4' label='Descriptor_t'>
<DATA type='C1' />
<DIMENSIONS total='19926'>
19926
</DIMENSIONS>
<VALUES>
FILES:
challenge.cgns
...
</VALUES>
</NODE>
<NODE name='CoordinateX' label='DataArray_t'> <DATA type='R8' /> <DIMENSIONS total='6150'> 2 123 25 </DIMENSIONS> <VALUES> </VALUES> </NODE>
/ELSA/PRS-02036
pyCGNS Status
v0.4Binding
• Full ADF wrapper + 60% MLL wrapper• Experimental OOL layer
Python + Numerical Python• Standard package• Documentation+ test suite
In-memory representation• Python/C/C++/Fortran in-memory tree sharing
Tools• SIFT - SIDS instance description check• ParseTree - Tree dump/display as XML tree• StampNode - Add sub-nodes with specific stamps (date, ownership…)
/ELSA/PRS-02036
pyCGNS v1.0 commitments
v1.0 (4Q2002)
Binding• Full ADF+MLL wrapper• Full OOL layer
Documentation• Features 100% documented + tutorial• Design partly documented
Test suite• Complete test suite for OOL/MLL/ADF
Tools• Tree parser + XML I/O with DTD• DBMS interface for blobs• Translators
/ELSA/PRS-02036
CGNS and pyCGNS references summary
CGNShttp://www.cgns.orgAIAA 2002-0752 - CFD General Notation System (CGNS): Status and future directions - S.Legensky,
D.Edwards, R.Bush, D.Poirier, C.Rumsey, R.Cosner, C.Towne
ONERA DSNA/ELSAhttp://www.onera.fr
http://elsa.onera.frAIAA 2002-0108 - An efficient Object-Oriented solution to CFD complexity - L.Cambier, M.Gazaix
pyCGNShttp://elsa.onera.fr/CGNS/releases
Pythonhttp://www.python.org