TOWARDS A NEW GENERATION OF

INDICATORS: MEASURING DIGITAL

GOVERNMENT

Barbara Ubaldi Senior Project Manager – Digital Government and Open Data Guillaume Lafortune Policy Analyst OECD GOV Paris, 6 September 2016

• What has the OECD done in the field of e-government/digital government measurement?

• How was the digital government performance survey structured and what areas did it cover?

• What does the OECD Recommendation on Digital Government Strategies imply for the development of OECD Digital Government Indicators?

• What’s next in terms of goals and objectives for measuring digital government?

Questions from the agenda

• Previous OECD work on measuring e-government

• Data previously collected

• Approach to be adopted in the new data-collection

CONTENT

Previous OECD work on measuring e-government

2010

• E-Government Indicators: Proposal for a New Framework and Methodology [GOV/PGC/EGOV(2010)2]

• ICT expenditures data collection

2011

• Towards indicators for benchmarking government ICT [GOV/PGC/EGOV(2011)3]

• Decision to refine data collection and focus on performance

• ICT expenditures data collection

2012

•Towards E-Government Performance Indicators GOV/PGC/EGOV(2012)8 •Building the basis for new e-government performance indicators: ICT spending by central government [GOV/PGC/EGOV(2011)3/REV1]

The genesis of DigGov Performance Indicators

2014 Digital Government Performance Survey

Data previously collected

Survey constitutes the first step in gathering key data to assess governments’

performance in using ICTs among other measures of productivity in the public

sector

2014 Digital Government Performance

March-April 2014

• Background research

• Survey draft and testing

April-June 2014

• Data collection

June-October 2014

• Data validation

• Results analysis

Survey timeline and structure

ICT strategy

Digital rights and obligations

Governance

ICT Project management

ICT Business cases

Financial benefits for the central government

Financial benefits outside the public sector

Central government ICT expenditures

Sub-national governments ICT expenditures

ICT Procurement

Online service delivery and transaction costs

Using national online portals

Survey Sections

Central Government ICT expenditure template

• 2014 OECD Survey on Digital Government Performance quite technical and data-intensive: how to ensure that validity, reliability and international comparability are guaranteed?

• Five common issues related to methodology and data collection require additional discussion:

a) Build on existing definitions in the survey to provide precise and easy-to-grasp definitions of concepts in order to ensure a shared understanding

b) Combine objective and subjective data in order to ensure high relevance, despite subjective assessments can be often affected by political concerns and cultural differences

c) Integrate ad-hoc estimates provided by respondents with consolidated national statistics

d) Agree on the most relevant areas for key Digital Government Performance Indicators in order to ask the questions most aligned to national needs in performance measurement

e) Improve the coordination of data collection across different levels of government in order to overcome issues of international comparability across countries, as well as those related to data availability

2014 OECD Survey on Digital Government Performance: Methodological challenges

Government at a Glance 2015

0

20

40

60

80

100

120

140

AUS CHL CZE DNK EST FIN FRA HUN ISL ITA JPN LUX MEX NLD NZL POL SVN ESP SWE CHE GBR COL LVA

Central government ICT projects with a total project value greater than USD 10 million, 2014

Source: OECD (2014), Survey on Digital Government Performance.

Assessing digital government performance

25-50%:

50-75%:

75-100%:

0-25%:

25-50%:

50-75%:

75-100%:

More

18-24

months

24-30

months

30-36

months

Share of total direct financial benefits from ICT

investments that OECD governments report, 2014

Average length of central government ICT projects with

a total project value greater than USD 10 million, (2014)

Source: OECD (2014), Survey on Digital Government Performance.

Approach to be adopted in the new data-collection

Measuring Digital Government

OECD Recommendation on Digital Government Strategies

Openness and Engagement

Governance and Coordination

Capacities to Support Implementation

1) Openness, transparency and inclusiveness

2) Engagement and participation in a multi-actor context in policy making and service delivery

3) Creation of a data-driven culture

4) Protecting privacy and ensuring security

5) Leadership and political commitment

6) Coherent use of digital technology across policy areas

7) Effective organizational and governance frameworks to coordinate

8) Strengthen international cooperation with other governments

9) Development of clear business cases

10) Reinforced institutional capacities

11) Procurement of digital technologies

12) Legal and regulatory framework

Creating Value Through the Use of ICT

• Built on international instruments

• Framework based on the three pillars of the Recommendation

• Focus: measuring the principles

• Methodology: surveys of government experts

• Behavioural: De facto not De Jure

• Different but complementary with the UN E-Government Survey, the EC Digital Economy & Society Index, EU E-Government Benchmark.

Digital Government Indicators

An example: measuring OGD

Governance De jure

• Existence of laws, directives, strategies, policies to promote OGD at the central/federal level, international commitments (IODC, G20 AC principles)?

Implementation De facto

• How far is data available (scope and breadth)?

• How far is data easily accessible?

• How far does the government support the re-use in practice?

Impact

• What are the main measurable impacts of OGD? (e.g. business creation, new forms of service delivery leading to savings…)

Short - medium term Qualitative table (s) and OUR Government Data Index

Long run OGD Survey + Case Studies

Challenges?

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Data availability Data accessibility Government support to re-use

No

na

tional O

GD

portal

The 2015 OECD OGD pilot Index

The Open-Useful-Re-Usable Government Data Index (OURData Index)

Where to start?

Openness and Engagement

Governance and Coordination

Capacities to Support Implementation

1) Openness, transparency and inclusiveness

2) Engagement and participation in a multi-actor context in policy making and service delivery

3) Creation of a data-driven culture

4) Protecting privacy and ensuring security

5) Leadership and political commitment

6) Coherent use of digital technology across policy areas

7) Effective organizational and governance frameworks to coordinate

8) Strengthen international cooperation with other governments

9) Development of clear business cases

10) Reinforced institutional capacities

11) Procurement of digital technologies

12) Legal and regulatory framework

Monitoring the Recommendation

1. Questions on the approach?

2. Which principles should we prioritize?

3. How can we reconcile the previous data collection with the new one accompanying governments in the transition?

4. Wish list vs feasibility of data collection

5. The indicators as part of the Toolkit to measure the implementation of the Recommendation

Questions and points for discussion

Thank you!