SC7 WG6Rome Engineering Ingegneria Informatica S.p.A.

INFSO-RI-223782

Isabel Matranga

ETICSAutomated Building,Testing and Quality Assurance

ETICS

INFSO-RI-223782

2

Rome, 19 October 2009

ETICS projects

E-Infrastructure for

Testing

Integration and

Configuration of

Software

ETICS

INFSO-RI-223782

3

Rome, 19 October 2009

ETICS automates and improves …

ETICS automates and improves theexecution of:

• Builds

• Tests

• Quality Verification

… offered as a service to its users

ETICS

INFSO-RI-223782

4

Rome, 19 October 2009

The context

4

ETICS

INFSO-RI-223782

5

ETICS Keywords

All-in-one solution building/testing/quality– Support for execution of continuous software build– Automatic creation of distributed packages– Dependency management

Quality made easy– Verification of the quality of the software produced– Offers the Automated-Quality Certification Model

compliant with ISO standards– Collection of test information

Language / platform / tool independent– ETICS is non-intrusive and easily extensible– Plugin based system

Distributed environment– Support for syncronization of developpers– Distributed builds and test– Multiplatform build and tests

ETICS

INFSO-RI-223782

6

Rome, 19 October 2009

ETICS supports software professionals

Requirementsand

Specification

Requirementsand

Specification

MaintenanceReview

and Improvement

MaintenanceReview

and Improvement

Architecture and Design

Architecture and Design

Testing, Validation and

Verification

Testing, Validation and

Verification

DeploymentDeployment

Implementationand Build

Implementationand Build

ETICS

INFSO-RI-223782

7

Rome, 19 October 2009

ETICS users

Main users (apart from ETICS 2 project itself)

“We use ETICS to build these software components across a variety of platforms, By using ETICS we are able to benefit from an automated build environment which can be reproducible at a later date”

Steven Newhouse, EGEE III Technical Director

“ETICS makes build and integration nightmare activity a simple configuration activity”

Pasquale Pagano, D4Science Technical Director

EDGeS “A-QCM certification, will contribute to the successful promotion of EDGeS achievements and solutions towards industrial partners”

Prof. Peter Kacsuk, coordinator of EDGeS consortium

ETICS

INFSO-RI-223782

8

Rome, 19 October 2009

What is A-QCM?

How does ETICS work?

ETICS

INFSO-RI-223782

9

ETICS System demoAutomated Testing and Quality Assurance

ETICS

INFSO-RI-223782

10

Table of Content

• Building with ETICS

• Testing with ETICS

• QA with ETICS

ETICS

INFSO-RI-223782

11

Table of Content

• Building with ETICS

• Testing with ETICS

• QA with ETICS

ETICS

INFSO-RI-223782

12

1

Build/TestArtefacts

ConfigurationWeb Service

Report/MetricsDB

ConfigurationDB

Execution Engines(Metronome, gLite, UNICORE, etc)

Command Line User Interface

RepositoryWeb Service

ETICS Infrastructure

Physical WorkerNodes

Web Portal

Virtual OS Images

ArchitectureBehind the scene

ETICS

INFSO-RI-223782

13

ETICS PortalWelcome

ETICS

INFSO-RI-223782

14

WA AdministrationSecurity

Authentication

Access to ETICS services is based on X509 certificates and fine-grained user roles:

Authorization based on RBAC model– Developers: can do local builds of project, subsystems and

components– Integrators: can create and run integration configurations– Testers: can register and submit tests attached to

configurations– Release Managers: can create and publish configurations,

create release notes and other documentation– Project Managers: can create and manage projects, assign

permissions, set policies, etc

ETICS

INFSO-RI-223782

15

ETICS PortalThe configurations

ETICS

INFSO-RI-223782

16

Web ApplicationDependencies

ETICS

INFSO-RI-223782

17

Web ApplicationRemote build

ETICS

INFSO-RI-223782

18

Repository

ETICS

INFSO-RI-223782

19

ReportsPresenting results

Testing with ETICS

ETICS

INFSO-RI-223782

20

Table of Content

• Building with ETICS

• Testing with ETICS

• QA with ETICS

ETICS

INFSO-RI-223782

21

Build & Testpackages

Reports &Metrics

Execution Engines

AWS, Metronome, gLite, Unicore

Repository WS

- source code- test code

continuous build and testing

- job submission- deployment

The ETICS visionin testing

ETICS

INFSO-RI-223782

22

ETICS Testing services

The multi-node testing functionality

• Automating execution of complex tests configuration: involving the deployment of several services, applications, clients on different hosts and their interaction

ETICS supports natively several types of testing:

• Unit test - ETICS build and test system is designed to run the already existing unit tests, to collect the results in form of metrics but not to create new unit test cases

• Static test - this technique provides a foundation for producing solid code by exposing structural errors and preventing entire classes of errors. ETICS integrating static analysis tools and techniques into the build/test process can yield significant reductions in system testing and field failures

ETICS

INFSO-RI-223782

23

Testing with ETICSThe plugins

• The ETICS plugins are small units of functionality or wrappers around external tools

• Plugins are highly configurable in ETICS

• ETICS user can activate them according to policies called profiles. Profiles can be set as a static attribute of a configuration or dynamically using build properties

• The plugins collect information and publish data as Metrics

• Metrics are published with each build/test reports and are also stored individually in the ETICS Repository

Optional plugins can be implemented and installed by users

ETICS

INFSO-RI-223782

24

Plugins and Metrics

Available plugins in ETICS:

• Sloccount (Single line of code count) – static analysis

• PyUnit, Junit, CppUnit – unit testing

• Pycoverage, Jcoverage

• Findbugs – static analysis

• CCCC

• IPv6, WSI

• CKJM

• Checkstyle

• PMD

and more coming soon ...

ETICS

INFSO-RI-223782

25

Testing use caseAn ETICS success

A typical scenario

These are the typical issues of a complex system:

• Having a running DB

• Having a software dependency installed

• Running the test suites

• Accessing the reports

A

ETICS

INFSO-RI-223782

26

ETICS Answers

• Recently presented “Experiences with the ETICS support for testing in voms” at EGEE 09 faced the described scenario:

• ETICS can configure and running additional services during a build by adding a dependency on the third-party modules (like a DB) or towards a version of a software module

• ETICS provides two choices:• ‘Test’ target in the “Build Commands

• Test Commands

• ETICS allows user-defined reports to be added to its own report page.

ETICS

INFSO-RI-223782

27

Table of Content

• Building with ETICS

• Testing with ETICS

• QA with ETICS

ETICS

INFSO-RI-223782

28

The ETICS visionin automating Quality Certification

Defines metrics

Runs measures

Define thresholdsTrials analysis

ETICS grid infrastructure at CERN,INFN, UoW (NMI)

ETICS

INFSO-RI-223782

29

A-QCM: Implementation

QA

EM

METRIC

Plugins

ETICS

INFSO-RI-223782

30

A-QCM: Web interface report example

Project level results

Subsystem level results

Component level results

The result can be expanded to reach the

metrics output related to the specific classes

ETICS

INFSO-RI-223782

31

Future plans

Really soon …• Results from A-QCM trial certification on ETICS users’

software publicly available • Enhancement of the multi-node testing feature• More plug-ins for testing integrated in ETICS• AWS cloud and Microsoft Azure Cloud under testing …

In 2010 …• ETICS offered as a service • ETICS software available under Apache 2.0 license

ETICS

INFSO-RI-223782

32

Thanks!

http://www.eticsproject.eu