www.eu-etics.org

INFSOM-RI-026753

The Reporting EngineThe Reporting Engine

Paolo FabrianiPaolo FabrianiEngineering Ingegneria Informatica S.p.A.Engineering Ingegneria Informatica S.p.A.

22ndnd ETICS All Hands Meeting ETICS All Hands Meeting

Bologna – Oct 23-25, 2006Bologna – Oct 23-25, 2006

INFSOM-RI-026753

Summary

• The ETICS RepositoryThe ETICS Repository– ArtefactsArtefacts

• The Report EngineThe Report Engine– Overview, Disseminators, TransformersOverview, Disseminators, Transformers

• SamplesSamples– Converters, Aggregators, PipelinesConverters, Aggregators, Pipelines

INFSOM-RI-026753

The ETICS Repository

• The ETICS repositoryThe ETICS repository– A Data Management System A Data Management System

thtat allows to store, catalog, thtat allows to store, catalog, manipulate, browse, search manipulate, browse, search and delete ETICS artefactsand delete ETICS artefacts

• The ETICS repository The ETICS repository containscontains– Raw DataRaw Data

– Binary Packages (rpm, tgz, Binary Packages (rpm, tgz, msi, deb, zip, ...)msi, deb, zip, ...)

– Reports (build reports and Reports (build reports and logs, test reports, ...)logs, test reports, ...)

– Metadata and relations among Metadata and relations among artefactsartefacts

Metadata

artefact

Repository Service

Raw Data (xml)

Metadata

artefact

Refs To Artefacts

INFSOM-RI-026753

ETICS Artefacts

• ETICS ArtefactETICS Artefact– identifieridentifier– metadatametadata– data source: packages, reports, logs, custom output, ...data source: packages, reports, logs, custom output, ...– accessible through URLsaccessible through URLs

Repository Service

Metadata

artefact

Raw Data (tgz)urlClient

(Cmd / Web)

INFSOM-RI-026753

Scenario

• Simple ScenariosSimple Scenarios– ETICS Raw Artefact available in HTML formatETICS Raw Artefact available in HTML format

– User wants to get it as PDFUser wants to get it as PDF

– Build Log available as XMLBuild Log available as XML– Present it as HTML (given a XSL)Present it as HTML (given a XSL)

– Build Artefact stored as TGZBuild Artefact stored as TGZ– Download it as ZIP fileDownload it as ZIP file

• Advanced ScenariosAdvanced Scenarios– Produce Aggregate statistics from project-wide build logsProduce Aggregate statistics from project-wide build logs

INFSOM-RI-026753

Approaches

• Attach multiple formats, Attach multiple formats, presentations & processings presentations & processings to each artefactto each artefact

Repository Service

Metadata

artefact

Raw Data (tgz)

Metadata

artefact

Raw Data (zip)

INFSOM-RI-026753

Approaches

• Attach multiple formats, Attach multiple formats, presentations & processings presentations & processings to each artefactto each artefact

• Attach parametric Attach parametric transformers to artefact to transformers to artefact to dynamically produce dynamically produce different formats, different formats, presentations & processingspresentations & processings– can be reused to manipulate can be reused to manipulate

data of the same typedata of the same type– changes to the source format changes to the source format

propagate to derived formatspropagate to derived formats– less content is stored in the less content is stored in the

repositoryrepository

Repository Service

Metadata

artefact

Raw Data (tgz)

...

tgz 2 zip

tgz 2 xxx

INFSOM-RI-026753

Disseminators

• A A DisseminatorDisseminator is a component added to any artefact is a component added to any artefact to produce extended functionality for the artefactto produce extended functionality for the artefact– defines a new access points for the artefact beyond accessing to defines a new access points for the artefact beyond accessing to

the artefact itselfthe artefact itself– identifies a concrete trasformer that provides the functionalityidentifies a concrete trasformer that provides the functionality

Metadata

artefact

Repository Service

Raw Data (XML)

PDF

Disseminatorclient

interface

TransformerClient

(Cmd / Web)

URL+

params

INFSOM-RI-026753

Disseminators (cont’d)

– defines a mapping (data & operation) betweendefines a mapping (data & operation) between– the extended access point provided to the artefact (and a number of the extended access point provided to the artefact (and a number of

related artefacts) andrelated artefacts) and– concrete operations & arguments of the transformerconcrete operations & arguments of the transformer

– multiple disseminators can be attached to each artefactmultiple disseminators can be attached to each artefact

Metadata

Transformer

artefact

Repository Service

Raw Data (XML)

PDF

Disseminator client

interface

WS

m

apping

Client

(Cmd / Web)

URL+

params

PostScript

INFSOM-RI-026753

Transformers

• Implemented as Web ServicesImplemented as Web Services– invoked using URLs (with args) and replying via HTTP invoked using URLs (with args) and replying via HTTP

responsesresponses– SOAP-based WSSOAP-based WS

• Web Services have a Web Services have a profileprofile describing describing– endpoint of the serviceendpoint of the service– lists operationslists operations– input parameters and output types for each operationinput parameters and output types for each operation

– Input/Output can be either simple values (e.g. strings) and artefacts Input/Output can be either simple values (e.g. strings) and artefacts (e.g. jpg images)(e.g. jpg images)

– Input/Output can be passed by value or reference (e.g. an url to get Input/Output can be passed by value or reference (e.g. an url to get a zip from the ETICS repository)a zip from the ETICS repository)

– Input and outputs can be typed (e.g. MIME types)Input and outputs can be typed (e.g. MIME types)

– used to register the plugin in the systemused to register the plugin in the system

INFSOM-RI-026753

Dissemination Manager

• Dissemination ManagerDissemination Manager

– RegistersRegisters– DisseminatorsDisseminators– Web Services Web Services

– Defines a mapping between:Defines a mapping between:– Artefact types and Artefact types and

disseminatorsdisseminators– Artefact instances and Artefact instances and

disseminatorsdisseminators

INFSOM-RI-026753

Characterisation

• InitiatorInitiator– UserUser

– via WebApplication or Command Line Interfacevia WebApplication or Command Line Interface

– The ETICS SystemThe ETICS System– on a scheduled basis or on significant events (e.g. build)on a scheduled basis or on significant events (e.g. build)

• StorageStorage– Persistent (until explicit removal)Persistent (until explicit removal)

– e.g. time-consuming report generators could permanently be store e.g. time-consuming report generators could permanently be store the report int the ETICS Repository becoming an artefact itselfthe report int the ETICS Repository becoming an artefact itself

– Volatile (until user retrieval)Volatile (until user retrieval)– Stored in a per-user or shared workspace outside the repository. Stored in a per-user or shared workspace outside the repository.

Accessible via the ETICS dashboard (MyETICS)Accessible via the ETICS dashboard (MyETICS)

INFSOM-RI-026753

Characterisation

• ExecutionExecution– SynchronousSynchronous

– e.g. for quick, single, transformationse.g. for quick, single, transformations

– AsynchronousAsynchronous– e.g. for time-consuming/mass reportse.g. for time-consuming/mass reports

• LifetimeLifetime– LimitedLimited

– when volatile/asyncwhen volatile/async

– IndefiniteIndefinite– when persistentwhen persistent

INFSOM-RI-026753

Sample Scenarios

• On-demand, synchronous, volatile, format conversionOn-demand, synchronous, volatile, format conversion– User asking for a printable version (PDF) of a build reportUser asking for a printable version (PDF) of a build report

• On-demand, asynchronous, volatile, code analysisOn-demand, asynchronous, volatile, code analysis– User asking for a project-wide, heavy analysisUser asking for a project-wide, heavy analysis

• Scheduled / Triggered, persistent report generationScheduled / Triggered, persistent report generation– ETICS Build System triggering the production of aggregate ETICS Build System triggering the production of aggregate

statistics after a build processstatistics after a build process

INFSOM-RI-026753

Sample Transformers

Format Converters

• Report formattingReport formatting– E.g. XML -> HTMLE.g. XML -> HTML

• Package Format Package Format ConversionConversion– e.g. RPM ->tgze.g. RPM ->tgz

• Graphs, Barcharts, Graphs, Barcharts, Plots ...Plots ...

INFSOM-RI-026753

Sample Transformers

Aggregators

• IN: build report listing components built for a projectIN: build report listing components built for a project• IN: raw quality metrics for each component builtIN: raw quality metrics for each component built

• OUT: raw quality metrics for the whole projectOUT: raw quality metrics for the whole project• OUT: high level quality metricsOUT: high level quality metrics

INFSOM-RI-026753

Sample Transformers

Pipelines

• composes several transformers in a pipelinecomposes several transformers in a pipeline

• enables rapid development of transformers composing enables rapid development of transformers composing existing ones existing ones

• can be implemented as a pipeline enginecan be implemented as a pipeline engine– customized on a specific pipeline definition thatcustomized on a specific pipeline definition that

– describes the composition of transformersdescribes the composition of transformers– maps outputs to inputsmaps outputs to inputs

T1 T2 T3 T4