linc: masterclass “scientific software engineering” · linc: masterclass “scientific software...

LINC: Masterclass “Scientific software engineering”

“Mastering the complexity of numerical simulation software”

Presenters: ir. J.C.M. Dijkzeul and dr.ir. A.J. VrielingLocation: LINC First School, MallorcaDate: September 11-12, 2012

Profile:• Office in Delft, NL

• Founded in 1996, now with 16 employees and growing

• Highly educated employees

• Profound knowledge of mathematics AND software development

• Technical/scientific software for simulation

September, 2012LINC: Masterclass “Scientific software engineering” 2

Introduction VORtech

VORtech

3

About us

Product and services• mathematical modeling• software development

• software maintenance• masterclass

LINC First School, Mallorca September 2012

Expertise:• complex algorithms

• model coupling• data-assimilation• HPC: speed-up/parallellisation• domain knowledge• software engineering

Simulation software

4

Example

• SIMONA: since ca. 1980 for Dutch ministry of transport and public works• operational prediction system for water levels, flows, stormsurges• mathematical physics, parallel processing, data-assimilation• more than 1 Mloc, in Fortran, C, Perl, Java, Matlab, …


Simulation software

5

More examples

• CONTACT: contact mechanics (Fortran, Java)• OpenDA: data-assimilation and model calibration (Java, C)• DaVinci: analysis of trafic data (Visual Basic, Matlab)• FEWS: prediction/warning system water/weather (Java)

• IMAGE: global change model PBL (Fortran)


Challenge for R&D companies:•How to build large computational software?•And maintain it over time?


Introduction Masterclass “Scientific software engineering”


Challenge for universities

Questions: • How can PhD students avoid software pitfalls?• Put more focus on their actual research?• And how do new PhD students

build upon previous works?

Solution: • Create better software

Good scientific software spans multiple disciplines• Knowledge of application area• (Mathematical) modeling• Numerical analysis, High Performance

Computing• Software development, programming


Multi-disciplinarity

YES and NO!

Helpful, but not entirely fit.


Standard Software Engineering Methodology?

Evolution:1. It starts with a small program or software script

2. Software is extended to fit more needs

3. Software is reused in other project(s)

4. Software is incorporated into an operational process

5. Software is consolidated, maintenance is arranged


Good scientific software evolves

Fred Brooks (IBM, The mythical man-month, 1975):• Good software development is hard work

• There's no easy way out

• Provide a collection of best practices


How to combat “software complexity”

Present software development techniques

that work for scientific computing software

Present software development techniques

that work for scientific computing software


Aim of this masterclass


Origins of this masterclass

This specialism is recognized more and more at VORtech• Contributions to software development at Shell

• Assistance of PhD students in our work at TU Delft

• Model scan and assessment of software quality for PBL

• Development of a Top 10 of best practices

• Experiences in training well-skilled employees

Parallelization = 20% algorithms + 80% “bookkeeping”


It started with parallelization of software...

Bottlenecks: • Large code (>100K lines)

• many users & developers

• Worked together on large software projects

• Multiple versions, models, integration, coupling

• Dealt with essential complexity


Domain (de)composition, Kalman filtering

1. Basics of software engineering“Lay the foundation before building a house”

2. What and how to automate“Everything that is worth repeating, should be automated”

3. Testing & Numerical aspects“There are so many numbers and they're never the same”

4. Importance of clarity“It took longer, because I tried to do it quickly”

5. Software development process“To prevent software decay, long-lived software must be nurtured”

6. Summary and discussion


Masterclass contents

linc: masterclass “scientific software engineering” · linc: masterclass “scientific software...

Documents