e-Science Experiences:Software Engineering Practice

and the EU DataGrid

Lee Momtahan and Andrew MartinOxford University Software Engineering Centre

Contents

• EU-DataGrid

• Challenges

• Comparisons

• Conclusions

EU-DataGrid

• 9.8m Euro project over 3 years;• 21 partners in 15 countries;• application in particle physics (and

bioinformatics, and earth sciences);• PetaBytes of data: datasets to be catalogued,

replicated where necessary;• seamless delivery of computing resources• 200 staff, meeting infrequently (60 FTE)

Project Goals

• build application frameworks potentially involving huge amounts of data, compute power and distribution

• provide secure, managed, uniform access to such resource

• facilitate collaboration, and remote access to data and scientific instruments

• manage such facilities as a persistent service

Work Package Structure

Our role

• becoming involved after project started• funded to bring computer science/software

engineering experience to the project• intending to help by modelling aspects of design

in order that the system may be better understood, designed, built, documented

• in passing made the observations documented here• interested in the generality of these issues for

e-Science

Challenges

• Requirements Volatility– Novel paradigm; New diversity;

Volatile off-the-shelf components

• Geographical Separation– communication can easily become a

limiting factor (Brookes); Physicists are used to collaborating in experiments –but software?

• System Decomposition– Political concerns; geographic

determination

Challenges

• Project Processes and Authority– there is a quality plan… how do you get people

to follow it? is a commercially-based process appropriate? what about traditional academic means of QA?

Challenges

• Planning and tracking– exit criteria for an iteration seems to be the completion

of a document detailing the problems found in testing

Comparisons

• academic software production vs. commercial software production

• academic software production vs. other academic activity

• CMM Level for Software? For Paper/Proposal Writing?

• open source software vs. open source development• open source models vs. publicly-funded research• publication in journals vs. publication to a

repository

Conclusions

• Practice of software engineering in an e-Science context is substantially different to industrial practice

• Industrial models do not seem appropriate

• Open source models seem to fit better

• Publication and review are the key to quality and process improvement