f. carbognani software engineering for the virgo project at egogeneva-icalepcs 14/10/2005 software...

F. CarbognaniSoftware Engineering for the Virgo Project at EGOGeneva-iCALEPCS 14/10/2005

Software Engineering for the

Virgo Project at EGO

F. Carbognani for

The Virgo Collaboration

iCALEPCS - Geneva 10-14 October, 2005


2

Virgo Project

Suspended Michelson Interferometer with two 3 Km arms aimed at the detection of gravitational wave signals from cosmic sources


3

Project Numbers

• 700933 Lines of Code (LOC) of C/C++ and Assembler

• Supported external packages amount of 371339 LOC of C/C++

• 50 Workstation + 60 Local Control Units (RIOs) for the Interferometer operation

• 4 operating systems (OS9, alpha/OSF1, Linux, LynxOs

• 2 countries, 6 laboratories, 18 teams, almost 100 developers over the current lifespan of the project.


4

Size


5

Main Ideas

• Scientific software projects like VIRGO are becoming too large and complex to be managed without proper SE procedures

• Heavy-weight procesess and standards, like CMM, IEEE or ISO 9000 somethime limit their existence to office cupboards.

• Standards need to be simple and implemented by embedding them as a set of consistent tools


6

Key Areas

• Configuration Management• Problem Report• Development Environment• Documentation• (Automatic) Testing• Releases• Performance Monitoring


7

Configuration Management

• The supporting tool (SCVS) is a very thin layer on top of CVS aimed to simplify its use and tailor it on the Virgo Software development specificity.

• The basic rules behind SCVS are:– All files belonging to a package are handled at the same

time– Only one person at time can modify a package– Branches are supported

• In archiving a package, the developer tells the others that a new CONSISTENT set of files is ready for use.


8

Software Archive Access and Size


9

Software Problem Report (SPR)

• Provides an easy standard avenue for users and testers to report bugs and for developers and maintainers to fix them and track them in an orderly fashion.

• Very simple interface and SPRs lifecycle to avoid that developers end up in spending more time on the bug tracking system than on the bugs or the projects themselves.

• A central database with Web and email interface to submit, query, modify SPRs (implemented using WREQ)


10

SPR Statistics

VCS-3.0


11

Development Environment

• A standardized development essential requirement to manage different people in different sites.

• Such standardized environment has been obtained using the tool CMT

• CMT provides all key elements of a development environment: – structured make usage– easy encapsulation of third party tools and libraries – standardization of the UNIX environment set-up of

every Virgo user.


12

Documentation

• Document Templates• Whenever possible extract documentation

from the code itself for maintainability• Doxygen support

– CMT fragments for the generation of the Doxygen documents via “make”

– Doxyfile template.


13

Testing

• Has to be automatic• Tool for Automated Testing(tat) has been adopted and

slightly modified for the Virgo software• Key feature of tat are:

– The test execution has a standard and very simple interface (just a single command)

– The test command prepares the environment set-up, executes the test(s), filters out variant data (dates, time, host names), compares results against reference and provides FAILED or PASSED result.


14

Testing (cont.)

Virgo Test Facility (VTF) • Model of control system and control room • Validation of Software Releases • Validation of Operating Systems baselines • Integration tests for subsystems• Implements a full Suspension Crate replica and

minimal DAQ chain. • Being continuously upgraded and expanded


15

Releases

• Slow upgrade cycle (3-6 months) for Major (External) Releases, on demand for Minor (Internal) Releases

• Tested for non-regression• Uniquely identify all components (O.S., tools, external

packages, Common Software) needed as development or operational platform of an application.

• Provide the explicit Integration Plan • Heavily relying on the Virgo Test Facility (VTF) for the build

and test phase

• Latest releases represent a true fully aligned baseline


16

VCS-1.0 Use Dependency


17

VCS-3.0 Use Dependency


18

System Monitoring

• The Software Release cycle must be completed by collecting field data to be carefully analyzed for succeeding releases.

• The normal scope of the Big Brother tool (monitoring System and Network-delivered services) has been extended by proposing and customizing it as a common framework for all Virgo Subsystems performances monitoring

• BB graphs very effective in spotting specific software problems.


19

System Monitoring (cont.)


• Support for the Object Oriented development approach: – New templates for Analysis and Design Documents– UML and Use Cases: Evaluation of the umbrello graphical tool– C++ Class templates (hpp and cpp files) containing the Doxygen

tags• Web based documentation and teamwork support:

– Automatic generation of User Manuals in HTML format via Doxygen

– Use of the Wiki tool as the main project development space, document management system and knowledge base.

• Integrated Development Environment (IDE)– KDevelop being customized for fully automated Doxygen

documentation generation and for CMT integration

20

Future Steps


21

Conclusion

• SE does not come for free, but can be brought into a project at a reasonable cost.

• SE can be implemented in gradual steps, providing first the most urgent tools and procedures, and then letting focused pilot projects lead the way for the “full SE approach”

• Even on a more “code-and-fix” oriented environment, with some effort, SE can be accepted and shared.