cooperation models for software development · 2011. 6. 3. · cooperation models for software...

COOPERATION MODELS FOR SOFTWAREDEVELOPMENT

Valentin Todorov1

1United Nations Industrial Development Organization (UNIDO)

Meeting on the Management of Statistical Information Systems(Luxembourg, 23-25 May 2011)

Todorov (Vienna, Austria) COOPERATION MODELS FOR SOFTWARE DEVELOPMENT MSIS’2011 1 / 31

Outline

1 Motivation

2 Statistical Collaboration Models

3 Statistical Collaboration Groups

4 Socio-Technical Issues

5 Summary and Conclusions


Motivation

Collaboration

"Symphony of collaboration: multiple-stakeholder collaborationsprovide a clear path to integrated data exchange and systeminteroperability "by Robin Thomashauer on clinical data exchange and eHealth


Motivation

Collaboration and common software

Using common software solutions in national and/or internationalstatistical organizations

results in savings in time and moneyenhance mutually the institutional knowledgepromote and enable the implementation of statistical standards

Using statistical standards like SDMX will in turnfacilitate the data exchange mechanismsimprove the data quality on both sides


Statistical Collaboration Models

The Software Inventory

Software Inventory platform launched by the Sharing AdvisoryBoard (SAB)http://www1.unece.org/stat/platform/display/msis/Software+Inventory

To collect information about existing shared products, productsunder development or even products that are undergoing planningCompiled in cooperation with the ESSnet project on a CommonReference Architecture (CORA)Already more than 50 productsSeveral more products which are not yet in the Inventory(EUROSTAT, FLEX-CB)


http://www1.unece.org/stat/platform/display/msis/Software+Inventory

http://www1.unece.org/stat/platform/display/msis/Software+Inventory


Development status, Code availability and Charges

Development status, Code availability and Charges

REQ DEV STABLEDevelopment status

010

2030

OSS PROPCode availability

05

1015

2025

30

NO YESCharges

05

1015

2025



Code availability and Charges vs Dev. Status

Code availability and charges by development status

Development status

Cod

e av

aila

bilit

y

DEV REQ STABLE

OS

SP

RO

P

Development status

Cha

rges

DEV REQ STABLE

NO

YE

S



Programming environments used

Pogramming environments

SAS

JAVA

VB

C

C#

R

.Net

AS 3

PL/I

XML

Number of products0 2 4 6 8 10

Three main groups ofproducts according to theprogramming environment

SASJavaAny other (VB, C,C#, etc.).

Only three products writtenin R and only one of themstates to have a stableversion in production

Of course there are manymore R packages at CRAN



Who is developing shared software?

Software Vendors

STAT−NLSTATCAN

ISTATABS

EUROSTATSTAT−FI

INENO

STAT−SEECB

INSEENZ

OECDRS

STAT−EESTAT−HRSWEDEN

UNECE

Number of products0 2 4 6 8 10

Most of the products offeredfor sharing were developedby StatCan or Netherlands -eleven each

ISTAT has seven and ABSfive

EUROSTAT has also five,but they are not yet enteredin the Inventory



Types of Collaboration Projects

Three main approaches for collaboration

1 Software developed by one organization and released as OpenSource or Free Software

EUROSTAT: eDamis, SDMX Reference Architecture2 Software developed by one organization and offered for

collaborative sharing for a feePC-Axis: a suite of software designed for disseminating andvisualizing statistical data (GSBPM 7.x)OECD.Stat: designed for disseminating and visualizing statisticaldata (GSBPM 7.x)Generalized Systems Suite (StatCan)

3 "True" open source software which was developed as OSS andfollows the OSS models.

FLEX-CB: to create useful visualizations of statistical data frominstitutions that employ SDMX.R packages at CRAN: http://cran.r-project.org/


http://cran.r-project.org/


CRAN Task View for Official Statistics

CRAN Task Views, Zeileis (2005):http://cran.r-project.org/web/views/

Collections of packages which belong to the same areaRelevant descriptions of the packages, further grouped in functionalareasPossible to install or update all packages from a given task view

Recently a Task View dedicated to Official Statistics was addedMethods typically used in official statistics and surveymethodologyCurrently more than 40 packages are listed


http://cran.r-project.org/web/views/


CRAN Task View: Official Statistics and SurveyMethodology

CRAN Task View: Official Statistics and Survey MethodologyComplex Survey Design: GeneralComplex Survey Design: DetailsComplex Survey Design: Point and Variance EstimationComplex Survey Design: CalibrationEditing and Visual Inspection of Micro dataImputationStatistical Disclosure ControlSeasonal AdjustmentStatistical Record MatchingIndices and IndicatorsAdditional Packages and Functionalities

Most popular packages: Amelia, impute, mice, RecordLinkage,robCompositions, rrcovNA, sampfling, sampling, sdcMicro, survey(core), VIM, x12Other task views: TimeSeries, Econometrics and SocialSciences



Generic Statistical Business Process Model (GSBPM)

Published by METIS in 2009 as a tool for describing andbenchmarking statistical production processesRapidly becoming a defacto global standardInitially developed to provide a basis for statistical organizations toagree on standard terminology to aid their discussions ondeveloping statistical metadata systems and processesHowever it is apparent that the model can be very useful in otherareas like

harmonize the statistical computing architecturesfacilitate the sharing of statistical softwareprovide a basis for explaining the use of SDMX in a statisticalorganizationprovide a framework for process quality assessment andimprovementand many more ...



The Role of GSBPM

SP

EC

IFY

DE

SIG

N

BU

ILD

CO

LLE

CT

PR

OC

ES

S

AN

ALY

SE

DIS

SE

MIN

ATE

AR

CH

IVE

EV

ALU

ATE

Products by phases of GSBPM

0

5

10

15

20



Licensing Issues

Necessary to define the commercial and legal foundations for theexchange of softwareOpen Source Software (OSS) approach is a viable alternative forexchange activitiesHowever limited by different factors

The particular legislation of the country (e.g. Canada)Total Cost of Ownership (TCO)Other issues of "compatibility" between OSS rules and the rules ofthe organizations

EUROSTAT COmmon Reference Architecture (CORA) ESSnet -detailed description of the available licensing models

GNU type licensesApache/BSD type licensesEuropean Union Public License (EUPL)

Details: EUROSTAT (2010). CORA: Models for licensing, supportand legal properties, Work package 4.1



Multilingual Support

In the national context - either not a requirement or its priority isvery low (budget, resources and time constraints)Rarely becomes a key part of the software architectureThe lack of multi-language support: a major barrier to sharing(statistical) softwareStatistics Canada’s experience (see Karen Doherty, 2011)Three scenarios:

1 All controls are displayed in the both languages2 The user once (by starting or installing the application) selects the

language3 The user can switch between languages at any time without loosing

the context



Multilingual Support

Must be approached early in the design of the system andbecome a key part of the architectureThe preferred way of implementing multi-lingual support is toapply frameworks and relevant toolsPrinciples and Guidelines for Developing Multi-lingual StatisticalSoftware prepared by the Sharing Advisory Board (SAB) -available at the MSIS Wiki: http://www1.unece.org/stat/platform/display/msis/Home+Page


http://www1.unece.org/stat/platform/display/msis/Home+Page

http://www1.unece.org/stat/platform/display/msis/Home+Page


Multilingual support in the inventory

Multilingual support

en

en, fr

multi

nl

en, nl

se

sp

en, sp

fr

no

sr

Number of products0 5 10 15

Multilingual support of theproducts from the softwareinventory

Reflects the experience ofStatCan

Eleven bilingual(Englis/French) productsoffered by StatCan

Several other bilingualproducts (English/Dutch,English/Spanish)

Ten products which aredeclared multi-lingual:Blaise, OECD.Stat andPC-Axis are among them.



Support, Documentation and Training

Documentation and training activities: an important component ofthe total cost of a product or systemThis is what the open source software products are mostlycriticized. Example: R-HelpEvaluation version - important if the software product is not free ofchargeInventory: for less than the half of the products support andtraining are available

Support and training are offered for Blaise, Banff, PC-Axis,OECD.Stat and eDamisStatCan offers support for most of the products comprising theGeneralized Systems SuiteFLEX-CB acts in the same way as most of the open sourceprojects: sparse documentation; the source is available to modifyand contribute; help forum provides for quick support granted by thefew involved developers


Statistical Collaboration Groups

Inventory of International Groups

An overview of the different international groups whose work isrelated to the enterprise architecture of statistical organisations.Available at the MSIS Wiki pageSeveral of the groups are especially interesting for the topicconsidered here - their main objective is to foster collaborationapproach in striving for improvement in Statistical InformationManagement and to facilitate better cooperation in the field ofdevelopment of software and uptake of IT related standards withinthe respective statistical production systems

Statistics Open Standards (SOS) GroupStatistical NetworkUser groups of different statistical software products



Relations between the International StatisticalCollaboration Groups



Statistical Network

Formed at the Informal CSTAT Workgroup on StrongerCollaboration on Statistical Information Management Systems inParis last JuneMain Objective: "Working together with pace and passion to bettermeet our societies’ information needs while driving down costs"Participants: Australia, New Zealand, Sweden, Norway, UK andCanadaAreas of common interest and cooperation are

1 Innovation in Dissemination (New Zealand),2 Confidentiality and Disclosure Control (Sweden),3 Common Metadata/Information Management Framework

(OCMIMF) (Australia),4 Editing (Norway) and5 Web Data Collection (Canada).

Details: Value Creation Group (2010), Draft Output Report



Statistics Open Standards (SOS) Group

A consortium of NSIs who have agreed that they wish to and areable to contribute to common development of their statisticalproduction environments

homogeneous group in relation to use of ITsimilar underlying production modelsthey share a set of common visions

Participants: Denmark, Finland, Iceland, Netherlands, Norwayand SwedenAreas of common interest and cooperation are

1 Dissemination databases2 Metadata3 data collection4 Architecture and tools5 Standards.

Details: Rune Gløersen (2008)



User Groups

Groups organized around a particular software productKey objectives: promoting the implementation and use of thesoftware family in national statistical offices and otherorganizations; serve as forums for discussion and exchange ofideas and experiencesPC-Axis Reference GroupOECD.Stat User GroupInternational Blaise Users Group


Socio-Technical Issues

Socio-technical issues of collaborative softwaredevelopment

Global software development (Noll et al. 2010)Main practices associated with collaboration:

1 Identify common goals, objectives and rewards2 Collaboratively establish and maintain the product ownership

boundaries3 Collaboratively establish and maintain interfaces and processes4 Collaboratively develop, communicate and distribute work plans.

Barriers to these practices:geographic distancetemporal distance (locations in different time zones)language and culture differences (including corporate culture)infrastructure and product architecture


Socio-Technical Issues

Collaboration tools for software development

Solutions to the collaboration barriersVirtual teams and Global teaming modelBeecham et al. (2010), Todorov (2009), Theußl and A. Zeileis(2009)

Software development infrastructure and the relevant collaborationtools - Lanubile et al. (2010)

R-Forge: a central platform for the development of R-relatedsoftware, Theußl and A. Zeileis (2009)OSOR.EU: Open Source Observatory and Repository, a platformfor exchanging information, experiences and FLOSS-based codefor use in public administrations.Google Code: Google’s site for developer tools, APIs and technicalresources


Summary and Conclusions


Review and analysis of collaboration activities for sharing ofsoftware among national and international statistical organizationsSeveral main issues and opportunities

licensing, position in the statistical business process model,ownership and governance, sustainability, standards utilization,methods for distributed software development and technicalcommunication advances

Three main approaches for collaboration:software developed by one organization and released as OpenSource or Free Softwaresoftware developed by one organization and offered forcollaborative sharing for a fee"true" open source software which was developed as OSS andfollows the OSS models.




The collaboration adds new dimensions of complexity in thetechnical, administrative and legal aspectsBut will substantially contribute to

the harmonization of the software architecturesthe adoption of international standards like SDMXwill be beneficial for reducing the costs for developing andmaintaining of statistical softwarewill improve the user support and capacity building



References I

United Nations Economic Commission for Europe, (2009)Generic Statistical Business Process Model, Version 4.0,URL: www.unece.org/stats/gsbpm.

Noll, S. Beecham and I. Richardson (2010)Global software development and collaboration: barriers andsolutions,ACM Inroads, 1, 66–78.

EUROSTAT (2010)CORA: Models for licensing, support and legal properties, Workpackage 4.1,URL: http://cora.forge.osor.eu/WP4__Design_of_the_Organizational_Architecture.htm.


www.unece.org/stats/gsbpm

http://cora.forge.osor.eu/WP4__Design_of_the_Organizational_Architecture.htm

http://cora.forge.osor.eu/WP4__Design_of_the_Organizational_Architecture.htm

cooperation models for software development · 2011. 6. 3. · cooperation models for software...

Documents