agu 2013 in22a-07 :: the es-doc software process
TRANSCRIPT
AGU 2013
IN22A-07 :: The ES-DOC Software Process
Mission
Nurture a standards based eco-system in support of earth system
documentation creation, analysis & dissemination.
Sub-Domains2013
Earth System ModelsStatistical Downscaling
2014Obs4MIPs ?
Strategic Relevance
EXA-Scale requires quality documentation
http://es-doc.org
#esdocumentation
Funders
IPSL (EU)NOAA (US)
EX-ARCH (G8)ISENES-2 (EU)
PI’sVenkatramani Balaji (US - NOAA)
Cecelia DeLuca (US - NOAA)Sébastien Denvil (EU - IPSL)
Eric Guilyardi (EU - IPSL)Bryan Lawrence (EU - BADC)
Karl Taylor (US - PCMDI)
Core Team
Sylvia Murphy (US - NOAA)Allyn Treshansky (US - NOAA)Mark Greenslade (EU - IPSL)
ToolsCreateSearchView
CompareVisualize
API
PublishSearch
CompareVisualize
API Clients
2013Python
2014C, Java
Demo
CMIP5 Comparator
Process
Social or Technical ?
Social Process
before software comes dialog
Social ProcessRequirements Capture
• scientist engagement• prioritization• scoping• politics
Social ProcessSelf Organization
• roles• responsibilities• mutual respect
Social ProcessTransparency
• visibilty• accountability• traceability
Social ProcessEthos
• commitment to quality• determination• sense of humor !
Social ProcessCommunication (internal)
• face to face• telcos• wiki• email
Social ProcessCommunication (external)
• advocacy• splash page (see branding)
• social media
Social ProcessGoals
• achievable• collectively decided
• iterative
Technical Process
after dialog comes software
Technical ProcessInformation Architecture
• domain driven design• ontologies
Technical ProcessMeta-programming
• Code generation• Multi-language support• Supports quaility
Technical ProcessMap Reduce
• 700,00 lines of XML• 4D array
Technical ProcessUser Experience
• UI first• Usability testing
Technical ProcessWeb Services
• HTTP• REST vs RPC
• Encoding (JSON)• Security
Technical ProcessPolygot Programming
• python• javascript• bash
Technical ProcessTesting
• Automated Testing• Unit, Functional, Integration
• CI Server• Executed upon check in
Technical ProcessDeployment
• Today: single click• Tomorrow: fully automated
Conclusions
Conclusion - 1Scripter Developer Engineer Architect
In the absence of systematic training, internships, & mentoring, scientists will not get
beyond developer status
Conclusion - 2Strong Repeatable Social Process
The social process is undervalued yet once learnt is transferable between projects