study to develop an upgraded single market scoreboard as a … · 2019-09-18 · centre for...

Centre for Strategy & Evaluation Services LLP

Westering House 17 Coombe Road

Otford, Kent TN14 5RJ United Kingdom

E: [email protected] T: +44 (0) 1959 525122

Study to Develop an Upgraded Single Market Scoreboard as a Governance Tool for the Single Market

An Inception Report to the Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs of the European Commission

765/PP/GRO/IMA/19

Implementing Framework Contract 575/PP/2016/FC

18 September, 2019

i

Contents

1 Introduction 1

1.1 Structure of the inception report ................................................................................... 1

1.2 Study objectives and scope ............................................................................................ 1

1.3 Study background and Issues ......................................................................................... 2

1.3.1 Tracking progress in Single Market implementation – the Single Market Scoreboard ..... 2

1.4 Progress update – Phase 1 ............................................................................................. 4

1.4.1 Literature, sources and stakeholder review (task 1) ......................................................... 4

2 Methodology – Phases 2 and 3 7

2.1 Overview of methodology ............................................................................................. 7

2.2 Phase 2 - Data identification, collection and mapping .................................................... 8

2.2.1 Task 2. Mapping of existing tools for monitoring Single Market policies .......................... 8

2.2.2 Task 3. Identification of available data from different sources ......................................... 8

2.2.3 Task 3.2 Stakeholder consultations ................................................................................... 9

2.2.4 Timing of consultations .................................................................................................... 13

2.2.5 Data quality ...................................................................................................................... 14

2.3 Phase 3 - Data analysis ................................................................................................. 14

2.3.1 Task 4 Identification of gaps ............................................................................................ 15

2.3.2 Analysis of the stakeholder consultation feedback ......................................................... 15

2.3.3 Task 5. Definition of new indicators including end user’s perspective indicators ........... 16

2.3.4 Task 6. Recommendations on indicators ......................................................................... 18

2.3.5 Task 7. Presentation of upgraded scoreboard ................................................................. 18

3 Project Management 19

3.1 Detailed workplan and timeline .................................................................................... 19

3.2 Reporting and meetings ............................................................................................... 19

3.3 Assumptions................................................................................................................. 21

4 Next steps 22

4.1 Finalisation of Phase 1 .................................................................................................. 22

4.2 Launch phase 2 ............................................................................................................ 22

Annex 1 - Bibliography 23

ii

Tables Table 2.1: Stakeholder groups for the targeted online survey ................................................................... 11 Table 2.2: Interview programme structure ................................................................................................. 12

Figures Figure 2-1: Methodology and workplan ........................................................................................................ 7 Figure 3-1: Detailed workplan – SMS 2.0 .................................................................................................... 20

1. Introduction

1

1 Introduction

This document sets out the (draft) inception report for the “Study to develop an upgraded Single Market Scoreboard as a governance tool for the Single Market”. The study was contracted through Framework Contract 575/PP/2016/FC for DG GROW and is being led by CSES with support from Idea Consult, Oxford Research and Trilateral Research.

1.1 Structure of the inception report

The inception report sets out the detailed work plan and methodological approach to be adopted. Section 1 sets out the study objectives, scope and background. It presents progress made since the kick-off meeting and takes account of the findings from the preliminary desk research and review of data/information sources. Section 2 describes the methodology that will be used for the remainder of the study. Section 3 on project management provides the detailed project plan, reporting-related tasks and assumptions. Section 4 outlines the next steps.

The main report (maximum 20 pages) is complemented with more detailed and practical aspects which are annexed. The annexes include:

• A bibliography (Annex 1);

• Summary of Kick-off meeting (Annex 2)

• The contacts database (Annex 3). This will be updated on an ongoing basis during the study.

1.2 Study objectives and scope

The overall study objective is to provide advice and input in order to upgrade the Single Market Scoreboard to better monitor the state of the Single Market in its four dimensions: free movement of goods, services, people and capital.

The specific objectives are to:

• assess the existing Single Market Scoreboard and its accompanying indicators;

• map these against policy needs;

• identify new and relevant indicators – covering legal and economic issues and taking stock of the situation for users; and

• recommend how to improve the user-experience within an upgraded online scoreboard.

The study also needs to incorporate the international dimension of the Single Market, by identifying possible indicators for benchmarking/comparing market integration within the EU and within its major trading partners, including what it delivers for citizens and businesses.

In terms of identifying indicators, the ToR stresses that the study will need to look at:

• the causality and significance of Single Market policies and their implementation by Member States in relation to the identified indicators;

• data quality and stability, including how data could be collected periodically and interpreted, possible links with other relevant existing scoreboards at EU level (e.g. the European Innovation Scoreboard); considering how “real time” data could be collected and integrated into data analytics and data visualisation over the longer term;

• data analytics and data mining, to help identify barriers within the Single Market across sectors,

1. Introduction

2

by checking different sources of information within EU institutions and beyond and;

• data visualisation, to improve the Scoreboard’s user-friendliness and presentation of the most meaningful data.

1.3 Study background and Issues

The Single Market, established in 1993, supports goods, services, people and capital to move more freely between 32 European Union and EFTA countries. The Commission identifies the Single Market as one of the greatest achievements of the European project, essential in increasing the success and prosperity of European Union (“EU”) citizens and businesses.

Through its Single Market Strategy, the Commission is continuing to develop the Single Market and its performance, in particular through measures that aim to:1

• Enable the balanced development of the collaborative economy

• Help SMEs and start-ups to grow

• Improve the opportunities for businesses and professionals to move across borders

• Address restrictions in the retail sector

• Prevent discrimination against consumers based on nationality or place of residence

• Modernise our standards system

• Create more transparent, efficient and accountable public procurement

• Consolidate Europe’s intellectual property framework

• Ensure a culture of compliance and smart enforcement to help deliver a true Single Market

The Single Market Strategy complements other key Commission initiatives such as the Investment Plan for Europe, the Capital Markets Union (“CMU”), the Digital Single Market (“DSM”), the Energy Union and labour mobility.

1.3.1 Tracking progress in Single Market implementation – the Single Market Scoreboard

The Single Market Scoreboard (“SMS”) is one of the mechanisms which tracks progress in the implementation, enforcement and performance of the EU Member States (and sometimes EEA) vis-à-vis the Single Market.

Introduced in 1997, its purpose and scope have evolved over time, such that it now aims at giving an overview of the practical management of the Single Market. The Scoreboard organises data, information and analysis by: stage in the governance cycle; Member State performance; governance tool; policy area – for a small sub-set of policy areas; state of trade integration and market openness.

The SMS contains a range of indicators and analysis, updated and published annually. The data is principally based on a combination of official statistics, online (public) data portals and studies. As a monitoring tool, the aim is that the indicators and analysis provide the evidence-base, trends and comparison as a basis for action, allowing for better implementation and enforcement of Single Market rules.

Among the ways in which the SMS can be upgraded include:

• Development of additional indicators that provide greater utility for assessment of policy performance and subsequent action, including underscoring the causality and significance of

1 https://ec.europa.eu/growth/single-market/strategy_en#cross_border

https://ec.europa.eu/growth/single-market/strategy_en#cross_border

1. Introduction

3

single market policies;

• Broadening the coverage of policy areas, sectors and help avoid emerging barriers in strategic sectors (for example digital, artificial intelligence or the circular economy) or where there is further potential for economic integration such as services, taxation or network industries;

• Extending the set of indicators and analysis to better reflect the quality of transposition of directives;

• Complementing the existing regulatory/input-oriented indicators with more perception-based indicators to have a broad and coherent picture of how the Single Market works on the ground for citizens and business, including to cover the social dimension and issues related to trust;

• Making the scoreboard easier to use, more accessible to existing audiences, and more appealing to new audiences – drawing on best practice in scoreboard development and presentation; and

• Investigating the degree to which selected indicators can be compared with Europe’s main competitors.

Taking an example as to the debate on how data visualisation might be improved, during the kick-off meeting (KoM), the extent to which the existing ‘traffic light’ colour-coded system for reporting on data remains fit for purpose was discussed. This was introduced to report on the transposition deficit, and on the percentage of Directives where national transposition led to subsequent infringement proceedings in the early stages of the scoreboard’s development, and was subsequently rolled out across other areas. However, its ongoing pertinence may be questionable, given that it does not necessarily fit with some of the other governance tools and policy areas in the SMS.

The possibility of extending the scoreboard to include new areas such that it could feed into the European Semester reporting mechanism was also raised. The importance of reviewing the SMS to consider the micro-economic not only the macro-economic dimension was also highlighted.

A further aspect discussed at the KoM was the extent to which good practices can be identified and lessons learned from the development of other scoreboards by the Commission and international organisations (e.g. the OECD, WB). In particular, the success of the (relatively new) Justice Scoreboard, and the relevance of some of its indicators to the SMS was stressed. The rapid acceptance of the Justice Scoreboard and indicators contained therein was given as an example of good practice. Furthermore, the Digital Economy and Society Index (DESI) Scoreboard2, a composite index, was cited as an example of a scoreboard that is attractively presented in an engaging way. A particular good practice is the use of easily understandable ‘headline’ indicators. However, it was also recognised that there are drawbacks of the use of composite indicators, and it was not suggested to go down this avenue, but rather to draw on other elements of good practice from DESI.

Whilst our clear focus will be on reviewing the existing scoreboard and in parallel, identifying possible new indicators, and on strengthening the policy focus of the SMS, the benchmarking of existing scoreboards offers the potential not only to hone in on useful indicators for the SMS, but also alternative means of presenting data and information in a more visually appealing way. Consideration will therefore be given not only to using benchmarking to derive good practices for the website on which data is presented, but also in terms of the scope for standalone PDF scoreboards, with attractive visuals (graphs, charts, Figures) that could help to modernise the presentation of the SMS.

The target audience for the SMS was also discussed. Whilst it was acknowledged that this remains broadly unchanged, and should focus on EU and national policy makers (including politicians), national

2 https://ec.europa.eu/digital-single-market/en/desi - DESI summarises relevant indicators on Europe's digital performance

and tracks the evolution of EU member states in digital competitiveness, across five main dimensions: Connectivity, Human Capital, Use of Internet, Integration of Digital Technology and Digital Public Services.

https://ec.europa.eu/digital-single-market/en/desi

1. Introduction

4

and regional authorities, journalists and others with a more technical understanding of the different dynamics of the IM, equally there is a need to provide interesting and engaging data and information that helps to facilitate external communications on progress against the four freedoms of the IM using the SMS as a mechanism.

A possibility floated by our study team at the KoM was the need to investigate whether the indicators could be streamlined, with differentiation between a much smaller set of headline indicators (possibly structured differently, for example, around the four freedoms rather than the governance tools, policy areas etc. with a more numerous set of broader operational indicators of managerial use, reporting on the many different single market-related initiatives, networks and cross-border cooperation activities that feed into the single market. Getting the structure right, and considering different possibilities in terms of how data might be presented, will be an essential precursor, before including additional policy areas and indicators to reflect the evolution and modernisation of the IM.

1.4 Progress update – Phase 1

During Phase 1, a number of preparatory tasks have been carried out so as to structure the study assignment effectively. These are briefly summarised in bullet point form below: • A kick-off meeting (KoM) was held on 17 July 2019 with the client and wider Commission services

participating in the Steering Group (SG). A summary of the meeting was prepared by CSES and commented by the Commission. The final version is annexed to this report (Annex 2).

• A draft letter of introduction was prepared by CSES and sent to DG GROW. When signed, this will be used to facilitate the interview programme and wider stakeholder consultations.

• A small number of Commission officials from different DGs in the SG have been contacted, and scoping interviews have taken place.

• A detailed workplan has been developed. While no major changes have been made to the methodology following the kick-off meeting, the sequencing and timing of sub-tasks has been refined to take into account comments from the Commission.

• A template and guidance for the literature review has been created in Excel. The specialised research software Zotero is being used by the study team to support the literature search and review. Further details are provided below.

• An excel database has been created to clearly and systematically store information at various levels of granularity, allowing users to see 1) the ‘big picture’ of existing tools and indicators within the SMS and 2) the detailed information and metadata for each of the indicators.

• A contacts database has been developed which expands on the list of stakeholder groups in our proposal. Some additional names have been suggested by the Commission. The contacts database will be treated by our team as a work in progress, since those interviewed may have useful suggestions as to which other stakeholders from the target audience for the SMS should be added.

1.4.1 Literature, sources and stakeholder review (task 1)

An extensive review of the literature and other sources has been conducted to ground the study in a thorough understanding of the Single Market, its status and challenges concerning integration, the main related EU policies and the different data sets and scoreboards that could be exploited to upgrade the Single Market Scoreboard.

Starting with the publicly available sources outlined in the ToR, our study team has identified further relevant literature, documentation, data and information sources. We have done this using different techniques in parallel, such as Google, Google scholar searches and a snowball citation-chasing approach to check that the very latest literature on the subject has been included.

1. Introduction

5

The sources include EU regulations, directives and strategies, Council conclusions and Commission communications. They also include a wide range of professional and academic research studies linked not just to existing areas of the SMS, but also to policy areas not currently covered. Numerous other scoreboards and official statistics have also been identified. Furthermore, the Commission services have shared a number of data sources at the kick-off meeting and in the inception stage which have been added to the bibliography as appropriate.

The identified sources are detailed in the bibliography in Annex 1.

• The initial findings from the desk research and review of data/ information sources suggest that there are an extensive range of data sources available on different aspects of the existing SMS and on the single market more broadly.

• Regarding the existing SMS, we have mapped the main webpages, indicators and report outputs linked to the 13 governance tools and 4 policy areas identified in the SMS. In addition, we have begun benchmarking other scoreboards against the SMS, and mapping data sources available in new areas that could potentially be covered by the SMS in future.

• A finding was that the type of data varies widely, depending on the specific area concerned. Across some of the governance tools, the information available is relatively managerial in nature, i.e. pertains to the operations of a network and the frequency of the use of particular services, such as the CPC Network and Solvit.

• It was also noticeable that in some areas of the SMS, although there is reporting data available through the SMS, there is much less analytical literature than in the past. For example, the transposition deficit used to attract a lot of attention from researchers, academics and policy makers, and there were conferences, research and working papers, and studies on this topic, for example in the early 2000s, when the deficit was higher, and non-timely implementation of EU legislation was more an issue. Likewise, interest was maintained in the post-EU enlargement period. However, since just after the mid-2000s, there appears to have been a considerable decline in the level of interest and attention in the research and evaluation community and among policy makers in the issue. This may paradoxically be due to the success of SM implementation in terms of reducing the problem of the deficit, and this is an example of how strong performance in a particular area of the SM may lead to a decline in interest.

• The desk research has already pointed to the rapid evolution in the relevance (and/ or non-relevance) of particular indicators. Taking postal services as an example, a key SM performance indicator has for some years been the percentage of items posted that arrive the following day. However, digitalisation has led to a major reduction in conventional letters being sent (e.g. due to instant e-communications), but paradoxically, the rapid growth of e-commerce has led to an increase in the number of packages being sent. Therefore, structural changes in the market have led to changes in the ongoing relevance of the performance indicators selected. At the same time several Member States postal services have questioned whether the frequency of home delivery can be reduced below the minimum service required in the postal services directive e.g. by derogation. This is a good example as to why modernisation is needed to keep pace with the evolution in the different dimensions of the internal market itself.

• From our initial literature search and discussions, it has become apparent that there is an over-focus in terms of relative weighting and number of areas covered on the governance tools, and insufficient attention on reporting on policy areas (and in structuring these according to the 4 freedoms). The Single Market Scoreboard currently identifies relevant legislation as including “measures considered to have an impact on the functioning of the internal market, as defined in Articles 26 and 114(1) of the Treaty on the Functioning of the European Union (TFEU). This includes the four freedoms (freedom of movement of persons, goods, services and capital across borders within the EU), and supporting policies with a direct impact on the Single Market, such as taxation,

1. Introduction

6

employment, culture, social policy, education, public health, energy, consumer protection, transport, environment (except nature protection), information society and media.” 3 The extent and scope of this description is noteworthy. The study team will therefore seek to agree with the Commission which are the main policy areas that it will be important to cover in an upgraded SMS and the extent to which if these areas were reported on, sufficient linkages can be made to delivering on the single market. The stakeholder consultations will also be crucial in this regard and will help to arrive at an SMS 2.0 that is relevant and focused, but also extended beyond the current emphasis on Governance Tools.

• In some areas, the causal links between a given policy area and progress towards single market objectives was unclear, for instance FDI and trade data. This depends on the data sources available and the specific indicators on which data is being reported, where there may not always be a Single Market dimension.

• Regarding scoreboard presentation, this initial phase has underlined the large number and wide variety of different scoreboards that exist. For example, in the JRC draft inventories being prepared, there are close to 70 scoreboards and in the data hub list being prepared by the JRC Competence Centre on Composite Indicators and Scoreboards (COIN), there are more than 500. Rather than reviewing all of these, for the purposes of this study it will be necessary to draw lessons from a sub-set of the most relevant scoreboards including those mentioned at kick-off meeting. Already from these there is a wide variety of different approaches, ranging from the very quantitative and highly interactive scoreboard prepared on the DG MARE Blue economy indicators (which allows users to select indicators and their combinations), to the more static approach adopted but with additional qualitative analysis in a pdf document for the Justice Scoreboard. The Digital Economy and Society Index (DESI) and European Innovation Scoreboard both categorise their shortlisted indicators into a small number of dimensions e.g. connectivity, human capital, and – based on an analysis of performance – the European innovation scoreboard also groups Member States into innovation leaders, moderate innovators, etc. Each approach requires a different investment, in terms of set-up, maintenance and annual update.

Once the literature review has been finalised, an output will be a “conceptual and mapping overview” of the Single Market indicating its objectives, associated policies and corresponding data-sets. This will provide a building block for the subsequent finalisation of a framework, criteria and heuristics tool for assessing the relevance, coherence and effectiveness of the SMS indicators to measure performance in Single Market policy implementation, identify gaps and support the definition of indicators.

Stakeholders identified

Section 2.2 provides a draft plan for the stakeholder consultations, including the stakeholder groups identified. It outlines clearly the scope, objective and activities, methods and suggested stakeholder groups. The timing and sequencing of stakeholder consultations is such that this can help guide the gap analysis for the existing SMS from an end-user viewpoint as well as validate new policy areas and indicator suggestions. Further feedback as to which stakeholders should be targeted through the consultations during either data collection or analysis phases will also be welcomed and integrated.

3 https://ec.europa.eu/internal_market/scoreboard/performance_by_governance_tool/infringements/index_en.htm

https://ec.europa.eu/internal_market/scoreboard/performance_by_governance_tool/infringements/index_en.htm

https://ec.europa.eu/internal_market/scoreboard/performance_by_governance_tool/infringements/index_en.htm

2. Methodology – Phases 2 and 3

7

2 Methodology – Phases 2 and 3

This section sets out an abbreviated version of the methodology and explains how the study team will carry out the practical research tasks required in Phases 2 and 3. Accompanying information is provided in annex and in the original Technical Proposal (the latter sets out a detailed research framework for undertaking the study).

2.1 Overview of methodology

The assignment will be carried out in three phases as outlined in Figure 2.1.

Figure 2-1: Methodology and workplan

During the first weeks since the kick-off meeting was held, the focus has been on ensuring that the study is grounded in a thorough understanding of the key study issues through familiarisation with, and a review of key documentation sources. The research activities and progress made during Phase 1 were summarised in Section 1.4 – progress update. The rest of this section therefore focuses on the work to be undertaken during Phases 2 and 3. Once feedback has been received on the inception report and the associated methodology, we will commence the core data collection phase and put in place a robust framework for the data analysis.

Inception Report and meeting (Month 1)

Draft Final Report (Month 5) Final Report (Month 6)

• Task 4 Identification of gaps

• Task 5. Definition of new indicatorsincluding user's perspective indicators

• Task 6. Recommendation of indicators

• Task 7. Presentation of upgraded scoreboard

• D3 - Draft Final Report (Month 5)

• M4 - Draft Final Report meeting

• D4 - Final Report (Month 6)

• D5 - Executive summary and supporting data (D6) (Month 6)

Phase 2

Data identification, collection and mapping

Phase 1

Inception Phase

• M1 - Kick-off meeting

• Task 1 - Literature, sources and stakeholders review

• Develop data collection and analysis tools/framework

• Finalisation of methodological framework & workplan

• D1 - Inception Report (Month 1)

• M2 - Inception Report meeting

Interim Report (Month 3)

Months 0-1 Months 2-3 Months 4-6

• Task 2. Mapping of existing tools for monitoring Single Market policies

• Task 3. Identification of available data from different sources

• Stakeholder consultations

- Targeted consultation

- Interview programme

- Focus Group

• D2 - Interim report, including task 4 (Week 10)

• M3 - interim meeting (presentation of the interim report)

Phase 3

Data analysis


8

2.2 Phase 2 - Data identification, collection and mapping

Phase 2 involves data identification collection and mapping, building on the initial assessment of key documentation during Task 1, but taking this further through in-depth analysis and review of existing and possible new data sources. This will then be complemented by stakeholder consultations to accumulate key informant opinions on the use, utility and additional needs from the Single Market Scoreboard in terms of data, presentation and usability. Specifically, our plan includes 1) targeted stakeholder consultations 2) an interview programme with maximum 45 stakeholders and 3) focus group discussions.

Each task is considered in turn.

2.2.1 Task 2. Mapping of existing tools for monitoring Single Market policies

Task 2 will involve mapping the existing tools for monitoring single market policies as desk research, and will build on the desk research already undertaken in Task 1. In particular, we will conduct a detailed review of the identified tools and map these tools, their sources and indicators and other relevant information.

This will help build a comprehensive picture of the strengths and weaknesses of the existing Single Market Scoreboard and facilitate preparation for tasks 3 and 4 on identifying additional sources and performing gap analysis, respectively.

The tools and indicators will be mapped against an overall framework for the Single Market and its different elements and policies as prepared in the framework, criteria and heuristics tool. Looking ahead to the gap analysis, particular attention will also be given to tools used to monitor single market polices but that are not currently integrated into the Single Market Scoreboard, and tools beyond those used for monitoring single market policies. We will also pay specific attention to identifying data sources that have the potential to be integrated into the Scoreboard as real-time data sources in the longer-term.

The information will be clearly and systematically stored in an Excel database.

2.2.2 Task 3. Identification of available data from different sources

2.2.2.1 Task 3.1 Detailed identification, review and analysis of data from different sources

Task 3 involves broadening the research to identify available, relevant data from additional sources, to analyse the quality of data coming from these sources as well as to assess the feasibility of using the data periodically for the purposes of the Single Market Scoreboard.

We will conduct a detailed review of the identified additional sources and, as in Task 2, store this information clearly and systematically in the Excel database.

Different data and information sources will be investigated in this task, including:

• Existing data sources relating to the SMS Scoreboard – data from across the current set of governance and policy indicators.

• European Commission, ECB, EIB and other EU-related – this includes existing scoreboards such as the Digital Economy and Society Index (DESI), the EU Justice Scoreboard, European Innovation Scoreboard, Consumer Markets Scoreboard.

• The OECD – for instance, the Product Markets Regulation database or Government at a Glance

• The World Bank – for example Doing Business

• Commercial datasets – such as micro-level firm data from Orbis for possible consideration of econometrics, the FT’s FDI Markets database.


9

The literature search in Annex 1 provides further examples.

The examples of existing data and information sources identified in the original proposal include a combination of perception-based surveys and micro-level firm statistics as well as examples of real-time data. The international dimension (beyond the EU) is also available through some of these sources.

2.2.3 Task 3.2 Stakeholder consultations

In Task 3.2 we will carry out consultations with key stakeholders, keeping in mind the requirements to make the scoreboard a more useful tool for a (broader) professional audience. In particular, the following different types of consultations will be undertaken:

• Task 3.2.1 – Design a detailed implementation plan for a consultation based on an agreed strategy

• Task 3.2.2 – Targeted stakeholder consultations (online surveys)

• Task 3.2.3 – Interview programme

• Task 3.2.4 – Focus group discussions

2.2.3.1 Task 3.2.1. – Design a detailed implementation plan for a consultation based on an agreed strategy

The first step in managing the stakeholder consultations will be to finalise the overall strategy for conducting the consultations. The Tender Specifications suggest that the EC would take the lead in this regard. There is a logic to this, in that unlike many studies for DG GROW, where there is a strong focus on consulting with external stakeholders using a variety of mechanisms (e.g. online questionnaires, interviews), here, many of the interviewees will be from across the Commission services, such as those involved in the data for policy initiative, Units already contributing data to the SMS, and those that may be brought in in future within the context of an expanded scoreboard.

We would therefore welcome any further suggestions on stakeholders to be contacted/ interviewed, building on those already identified in the contacts database. During Phase 1, the client made various useful suggestions, such as providing contact details for EC officials involved in a variety of mapping and new initiatives that could be relevant to the future SMS, as well as for internal market specialists within the permanent representations. Members of the IMAC will be added as and when their contacts become available. This will be an iterative process.

In accordance with the Better Regulation guidelines, we will ensure that the draft detailed implementation plan for the consultation, the draft questionnaires and the selection of stakeholders are drafted and submitted to the Commission in a timely manner. Following any comments, the implementation plan, draft online questionnaires and interview guides will then be finalised and agreed.

Below we outline the draft plan for stakeholder consultations.

Consultation scope and objectives

We propose that stakeholders be consulted for several purposes:

• to collect primary data, where a need is identified for additional information to strengthen the evidence base available from secondary data

• to obtain key informant opinions on the use, utility and additional needs from the Single Market Scoreboard in terms of data, presentation and usability. This will help guide and inform the possible selection of important policies, policy areas, indicators usability and information so as to inform the data analysis phase.


10

• to test and validate the analyses from different data sources with a selection of the targeted stakeholders.

Consultation activities, methods and tools

We propose to focus the stakeholder consultation process on collecting data via targeted consultations, a wide-ranging interview programme and selected focus groups to test and validate the analyses.

In accordance with our standard practices, attention will be paid to ensuring an appropriate balance between the different types of stakeholders consulted: • By type of stakeholder – e.g. policy makers, national authorities/administrations, professional

organisations at EU level, representatives of businesses including SMEs and start-ups, consumer organisations, academics/researchers, journalists, thinktanks, organisations representing the interests of citizens;

• By geography to ensure appropriate geographic coverage and a balance of small and large Member States as well as EU level organisations such as Business Europe or the European Consumer Organisation

• By sector / policy area to attain representative coverage of different sectors, including strategic sectors of interest for upgrading the scoreboard

The consultation data collection tools will contain questions directly linked to the needs of the gap analysis so that the results can be easily fed into Task 4.

In designing and structuring the detailed implementation plan for the stakeholder consultation, CSES will take into close consideration the relevant parts of the Commission’s Better Regulation guidelines.

The way in which each of the proposed consultation tools will complement one another, and the way in which particular consultation methods might help to address possible information and evidence gaps will be also be specified in the finalised implementation plan. It should be noted that wherever potential gaps in the availability of information and data are identified (both in relation to particular consultation methods, and overall once the different consultations tools used to gather information have been carried out, we will propose mitigation methods to overcome any gaps. A list of the mitigation measures already identified was provided in the risk assessment in our proposal (Section 5.2). These will be integrated into the detailed implementation plan.

2.2.3.2 Task 3.2.2. – Targeted stakeholder consultations (online surveys)

The next step will be to undertake targeted stakeholder consultations. This will involve in particular the development of questionnaire-based surveys which will be launched once approved using the Commission’s EU Survey tool or, as an alternative, using SurveyMonkey, which could help to mitigate the risk of time delays.

The purpose of the targeted consultations will be to gather opinion-based feedback from different categories of stakeholders concerning their expectations of the Single Market Scoreboard. This includes opinions on the use, utility and additional needs from the Single Market Scoreboard in terms of data, presentation and usability, as well as important policies, policy areas and possible indicators. The consultation data collection tools will contain questions directly linked to the needs of the gap analysis so that the results can be easily fed into Task 4.

Since, as far as we know, there is no existing database of SMS users that we can draw upon as a sampling frame for statistical random sampling purposes, we propose a non-probability sampling method. Additional efforts will be made to ensure that the respondents are qualitatively representative of the stakeholder population by gathering responses from a mixture of Member States and stakeholder organisation types.

https://ec.europa.eu/info/files/better-regulation-guidelines_en


11

In our proposal, we suggested that the online surveys remain open for 4 weeks. This is the maximum time that the survey could remain open for if preliminary analysis is to be integrated into the interim report. It assumes that both the draft questionnaires and contacts database can be rapidly finalised, noting that little to no buffer exists in the sequencing of tasks.

To facilitate quantitative analysis of the opinion-based survey feedback, we will use mainly closed ended questions. However, respondents will have the opportunity to explain several of their responses in open text boxes, which will allow for more technical feedback to be provided. Additionally, a small number of open questions will be included.

The suggested stakeholder groups are outlined in Table 2.1.

Table 2.1: Stakeholder groups for the targeted online survey

Stakeholder type Stakeholder group

EU/international public administration

• Steering committee members

• DG DIGIT, DG SEC GEN, DG EMPL, DG FISMA, DG ENV, DG TAXUD, DG SANCO …

• Joint Research Centre

• Eurostat

National public administration • Ministries of economy, industry

• National Statistical Offices

National contact points /networks for aspects of the scoreboard

• Contacts for the transposition & infringements chapters;

• Networks such as European Enterprise Network (EEN), EURES, ECC-NET, SOLVIT centres, Product contact points, Product contact points for construction, National assistance centres for professional qualifications, National contact points of cross-border healthcare, ECC-Net, Points of single contact, Online dispute resolution, European Judicial Network in civil and commercial matters (EJN-civil), etc.

Representative EU and national (professional) organisations

• Business associations and SME associations: Business Europe, Euro-chambers, SMEUnited, Eurocommerce

• European Start-up Network

• European Trade Union Confederation (ETUC)

Consumer and civil society organisations

• Consumer organisations e.g. BEUC, consumers international

• Social Platform

• Member organisations of the EESC liaison group

Academics/ researchers • Selected researchers identified during the literature review

Journalists • News associations e.g. Associated Press, Reuters, Politico

Any further feedback as to which stakeholders should be targeted through the consultations during either data collection or analysis phases will also be welcomed and integrated.

The questionnaires will also need to be customised to different stakeholder groups. We will also develop suggested text for consultation launch page, as required and pilot the questionnaires.

Once the targeted online survey has been launched, we will monitor the response rate and take steps to improve response rates if necessary.

2.2.3.3 Task 3.2.3 – Interview programme

An interview programme will be undertaken with key stakeholders with a direct or indirect interest in the Single Market.

The purpose of the interview programme will be to:

Undertake stakeholder screening to:

o Obtain feedback from stakeholders that are identified as existing key users of the SMS that


12

can provide in-depth feedback;

o Obtain feedback from stakeholders that have a strong interest in the new policy areas that the SMS could be extended to cover

• Obtain qualitative feedback related to the relevance, coherence and effectiveness of the Single Market Scoreboard for measuring the performance of the Single Market policy implementation

• Acquire feedback on possible policy and sector areas that should be covered in an upgraded Single Market Scoreboard

• Solicit stakeholder input concerning the presentation and usability features desirable in an upgraded scoreboard

• Follow-up with respondents to the targeted (online) consultations that are willing to be interviewed

The proposed structure of the interview programme is provided below:

Table 2.2: Interview programme structure

Stakeholder type Examples Number Notes

National Administrations / Organisations from EU Member States

• Ministries of economy, industry,

• National Statistical Offices, etc.

5 National actors using, assessed by and providing some data used in the SMS

DGs with policy links to the Single Market

• Steering committee members

• DG DIGIT, DG SEC GEN, DG EMPL, DG ENV, DG TAXUD, DG SANCO, DG FISMA

• Joint Research Centre

• Eurostat

10 European data provider, DGs with a potential stake in an upgraded SMS, expert in the development of composite indicators – and research on the SMS.

Selected EU and national (representative) organisations

• Business associations and SME associations e.g. Business Europe, Euro-chambers, SMEUnited, Eurocommerce

• European Start-up Network

• European Trade Union Confederation (ETUC)

13 Stakeholders representing employers and employees with a view on the Single Market. Business Europe has published a strategy paper on the opinion on the EC Communication on the Single Market in a changing world.

NGOs and civil society organisations

• Consumer organisations e.g. BEUC, consumers international

• Social Platform

• Member organisations of the EESC liaison group

5 NGOs representing organisations and citizens linked to the four freedoms.

Other Networks • European Enterprise Network (EEN, EURES, ECC-NET

10 Each of the networks listed in the ToR has a stake in the SMS and may also provide and use data and results.

Other interviewees identified via the online survey

• Journalists, academics 2

Total 45


13

*To be confirmed and only if agreed as being relevant by the client. Further details regarding the structure of the interview programme and an explanation as to how the interviews will be conducted is now provided.

Following agreement on the stakeholder types, we will first identify a suitable set of target interviewees and integrate these in a contacts database and will then ask for feedback from the client as to whether there are any stakeholders missing that should be included. A further source of information for identifying suitable interviewees will be to analyse the information received through the targeted online consultations. A question will be included asking whether respondents are willing to be contacted and interviewed as part of a follow-up process.

The final selection of interviewees will need to be agreed with the Commission Services prior to launching the interview programme.

In structuring the interview programme presented in the above table, we will ensure not only an appropriate balance in the selection of interviewees across different categories of stakeholder, but also in terms of ensuring an appropriate geographical balance across the interview sample as a whole. Appropriate geographical balance will also be reflected in the selection of associations, enterprises and NGOs.

The interviews to be carried out will be undertaken as semi-structured interviews, customised to the different categories of stakeholders mentioned above. Initially we will conduct mainly face to face interviews to gain a full appreciation of the issues. Subsequently, phone conference interviews will be prioritised.

We will produce a Specific Privacy Statement on data protection and privacy explaining how the data will be collected and explaining their rights under the GDPR. For reasons of preserving the impartiality of third-party contractors in conducting independent technical studies, we can summarise the key messages from particular stakeholder types and once anonymised, provide these to the Commission services.

2.2.3.4 Task 3.2.4 – Focus group discussions

We propose to use a focus group setting to test and validate the analysis work, and in particular the mapping of existing data (Task 2), new data identified (Task 3) and gap analysis (Task 4) performed. The first focus will collect participants in an informal IMAC meeting, foreseen for 24 October 2019. This will focus on the tasks that are covered in the Interim Report. We propose, to the extent possible to use this occasion to get feedback on preliminary thinking concerning new indicators, including user perspective indicators, identified in Task 5.

In addition, we will organise targeted online focus group discussions where needed, in order to validate analysis regarding the gap analysis, possible data sources and/or definitions of new indicators. We will invite selected stakeholders who are experts in particular tools or policy areas that are relevant for the SMS. The detailed composition of the groups will be proposed following the analysis of stakeholder feedback via the survey and interview programme. The timing of the targeted focus groups will mainly coincide with the timing of task 5, where new indicators will be defined and which follows the gap analysis. This can be conducted efficiently via e.g. Skype with a small number of stakeholders.

2.2.4 Timing of consultations

The timing and sequencing of the consultation is clearly laid out in the detailed project plan in section 3.1. This approach allows for stakeholders both to help guide the gap analysis for the existing SMS from an end-user viewpoint and to validate new policy areas and indicator suggestions.


14

2.2.5 Data quality

All of the data collected as well as data sources and indicators mapped during Phase 2 of the study will clearly and systematically stored and described. As far as the indicators are concerned, we will aim to establish whether the indicators are RACER (relevant, accepted, credible, easy to understand and robust), in line with the Better Regulation guidelines TOOL #41. As a minimum, this detailed information will contain those areas underlined in section 8.1.4 of the Request for Services, namely:

List 1: information on data collected in the SMS and as part of this study

• Objective of the exercise

• Target population and sampled population

• Data (variables) to be collected

• Degree of precision i.e. are there some missing data or breaks in time series?

• Collection mode, i.e. how will the data be collected (by email, web platforms, dedicated application)

• Periodicity/frequency of a process i.e. is it a one-off exercise or a regular one?

• Validation

• Publication format i.e. in which electronic (open) format will the data be published (plain text CSV files,. Excel, R, or Stata files)

• Metadata i.e. what background information about the data needs to be disclosed to the public:

Data collection methodology;

Target population;

Sampled population;

Glossary and definitions of indicators/variables and their respective measurement units;

Codes, acronyms, flags used (those should normally be harmonised with Eurostat codes, e.g. two-letter country codes);

The timing and frequency of data collection;

The publication date;

Limitations, confidentiality issues, disruptions of methodology etc.

Contact point for potential questions and comments from the public.

In addition, we will tag the data sets and indicators as appropriate by governance area, results chain, policy area and sector, such that users can easily obtain comprehensive information to help assess the quality of the data, but also easily map this to different areas of the Scoreboard and to Single Market implementation.

At this stage, we will consider to what extent the data identified has the potential to be integrated into the Scoreboard in real-time over the longer-term.

2.3 Phase 3 - Data analysis

The data analysis phase includes a gap analysis, defining and recommending new indicators and providing recommendations to improve the presentation of the upgraded scoreboard.


15

Throughout the analysis phase the study team will utilise a combination of methods in order to analyse the data. We will support findings and recommendations by explaining the degree to which these are based on opinions, analysis and objectively verifiable evidence. Where opinions are the main source, the degree of consensus and the steps taken to test the opinions will be given.

2.3.1 Task 4 Identification of gaps

In Task 4, we will conduct the gap analysis, with the aim of identifying possible alternate indicators / data sources needed to give a more complete picture of the state of the Single Market and its four freedoms, as well as extending it to additional policy areas.

Task 4 will flow naturally from the previous Tasks, and will involve an analysis of the framework, criteria and heuristics tool developed and an assessment of the relevant consultation feedback.

We consider this task as a multidimensional assessment of the “identification of gaps”, including a gap analysis of the:

1. The broad type of indicator e.g. regulatory/policy, economic, users of the Single Market;

2. Policy areas not current covered by the SMS e.g. network industries, data economy, circular economy etc;

3. Related to the above, key policy sub-areas or critical issues not monitored currently;

4. Whether the indicators fully, partially, or to a limited extent measure the desired policy areas / concepts to be monitored;

5. Consideration of whether the indicators represent the optimal measures of internal market performance i.e. if the measures are causally and significantly linked to key Single Market policies and can demonstrate their effects / impacts.

Therefore, Task 4 will involve a detailed review of this tool to generate a reporting-output specifying “where the gaps exist”, while providing suggestions on how the “gaps can be filled” by new or modified indicators.

This reporting output will be structured according to the criteria / assessment points described above indicating where the main gaps are, while providing analytical commentary assessing the relevance, coherence and effectiveness of the SMS in terms of its gaps and whether the introduction of the alternate indicators / data sources can strengthen the system.

Moreover, the gap analysis will score the coverage of the policy areas by the current SMS indicators and also the alternate possible indicators, to get an idea of possible improvements.

We will also examine the relevant parts of the consultation feedback within this reporting-output. The consultation feedback will present the stakeholder views in terms of whether: their needs are being met by the current SMS; the type of indicators they require; their key areas for improvement of the existing SMS, including additional policy areas.

2.3.2 Analysis of the stakeholder consultation feedback

2.3.2.1 Analysis of the questionnaire responses to the targeted consultation

Having cleaned the data from the online survey, we will analyse the data to provide a clear set of findings and conclusions from the targeted consultation. This will involve aggregated and disaggregated analysis, for instance differentiating between:

• Responses by category of stakeholder;

• Responses by EU Member State


16

• Part of the Single Market Scoreboard e.g. Governance tool, sector, policy area, indicator.

• New policy area

We will also provide background information regarding the response to the survey, explain data limitations encountered and give an indication of the degree of consensus in the responses - either through disaggregating the feedback into groups, or through statistics that provide a measure of spread – e.g. the standard deviation or the interquartile range.

Quantitative opinion-based responses received through the targeted consultation will be triangulated against qualitative feedback gathered through the desk research and other forms of consultation i.e. interviews and where appropriate focus groups. At this point additional desk research will be conducted so as to complement the findings from the consultations with official statistics and studies, where possible.

The consultation outcomes will be analysed by our study team and included in the interim report.

2.3.2.2 Analysis of the interview feedback

The next step will be to undertake an analysis of the interview feedback, drawing on the detailed interview notes that will be produced by members of our team. All interview feedback will be collected and stored on a single platform, ensuring cross-cutting analysis of all the feedback. Most of the data collected in the interview programme will be qualitative so the feedback will be analysed and presented in a structured and thematic way. However, some quantitative statistics will also be calculated based on the feedback to identify, for example, the degree of consensus across stakeholder groups.

The research team will also search for themes that have a broader pattern of meaning, respecting the European Commission standards for data protection when analysing responses. The methods used and analytical building blocks (coding system, tags and sub-tags) will be made transparent to readers by providing supporting text in the main report to this effect, clarifying how the findings and conclusions were made.

2.3.2.3 Analysis of the Focus groups discussions

We suggest focus group work to test and validate the analysis work including on the new indicators identified in Task 5. To the extent possible this will be included as part of the informal IMAC meeting following the interim report. We will also conduct selected online focus groups or email exchanges concerning “showcased indicators” as needed (see Task 5).

We will prepare detailed focus group notes and store them on a single platform, ensuring cross-cutting analysis of all the feedback. Most of the data collected in the focus groups will be qualitative so the feedback will be analysed and presented in a structured and thematic way.

2.3.3 Task 5. Definition of new indicators including end user’s perspective indicators

In Task 5, we will define new indicators, including those from the perspective of users such as business and citizens. The indicators defined need to be relevant, as well as causally and significantly linked to Single Market policies demonstrating the real effect of those policies.

We will define the users in two ways:

• Users of the internal market (i.e. citizens and businesses); in the context of the SMS the aim is to identify the obstacles they are facing and monitor the extent to which they can fulfil their EU rights;

• Current users of the Single Market Scoreboard are also those stakeholders that use the indicator results to support their organisational and research activities (i.e. including the stakeholders


17

participating in the consultation for this study);

The process to defining the new indicators will involve the following steps:

1. Responding to the needs of end users: the appropriate aspects of the consultation feedback and needs will be considered and integrated into the process to define new indicators;

2. Making links between the Task 2 results around possible tools/datasets and the Single Market intervention logic and relevant policy areas to determine the possibilities for indicator definition;

3. Indicator concept exploration: investigate the specific elements of the policies that can be measured using the tools/datasets reviewed including new types of (user perspective) indicators, and measures to monitor the four freedoms, but also examining if the existing SMS indicators can be improved, considering the methods to be employed;

4. Indicator concept validation considering whether the selected variable alone is sufficient, if a derived indicator should be developed, if estimates are needed e.g. via econometrics/data analytics methods, or if a composite indicator could be appropriate. 4 Then involving:

a) Internal indicator concept validation between the project team members to determine the extent to which the indicators proposed measure appropriately the relevant policies/concepts considering the extent to which they are causally and significantly linked;

b) External indicator concept validation with DG GROW and selected stakeholders to obtain 360-degree feedback on the suggested indicators considering their strengths and weaknesses in measuring the effects of the suggested policy areas and to agree a subset of indicators to be showcased further by the study.

5. Data analysis and showcasing of “prime” indicators defined: for prime indicators identified in the validation exercise provide in-depth case research and testing of possible indicators and related methods, plus visualisations, demonstrating the “prime” indicators that would add significant value in meeting the needs of SMS end users if it were upgraded;

2.3.3.1 Explanation of the data analytics / econometric methods that could be suggested by the indicator showcasing exercise

A series of methods will be reviewed in-depth for the definition of the indicators selected. The explanation of the methods will be indicator specific.

In some instances, a variable used to represent an indicator may not be subject to further transformation if it already represents a meaningful measure of the performance of a relevant policy area. For pragmatic reasons, linked to time and the acceptability criterion in the RACER criteria, we will concentrate our attention on data and indicators that already exist.

However, in some cases, we may need to derive indicators by combining different variables or investigate other data analytics or econometric methods. Indicative methods include cluster analysis, factor analysis, principal components analysis or latent class analysis, social network analysis, regression, latent growth modelling or sentiment analysis.

4 As suggested by the briefing report to the EP IMCO committee ‘Towards Indicators for Measuring the Performance of the Single Market’, composite indicators to measure the internal market performance at a high level across economic and regulatory areas are likely not to produce an insightful measure of performance. However, the SMS could contain already ‘accepted’ composite indicators in specific sub-areas of policy, if appropriate.


18

All research methods are associated with advantages / disadvantages, limitations and risks. These will be flagged-up for all indicators defined with further in-depth assessments of these aspects explored in the indicator showcasing exercise. In doing so, we will develop a series of criteria so that these items can be examined evenly for each indicator.

2.3.4 Task 6. Recommendations on indicators

In Task 6, we will recommend the set of indicators to be included in the upgraded Single Market Scoreboard. This set of indicators will allow for an annual benchmarking of Member States in the most relevant Single Market areas. It will be based on:

1. A synthesis analysis of all study outputs;

2. Research on how “real time” information can be obtained from the relevant data-sources (the data sources’ potential for providing real-time updates is one of the aspects that we will investigate when conducting tasks 1-3);

3. An assessment of the relevance, effectiveness and coherence of the proposed and existing indicators;

4. Defining a final set of recommendations validated by stakeholders.

2.3.5 Task 7. Presentation of upgraded scoreboard

In this task we will make recommendations and give ideas on how to 1) improve the user-experience and user-friendliness of the current scoreboard 2) provide easy and simple access to the most meaningful data, for example through a dashboard displaying the most relevant areas for the Single Market.

Recommendations will be provided in areas including:

• Improving the user experience and providing simplicity in accessing quantitative and qualitative data. We will provide recommendations concerning a range of different data visualisations, such that users can understand the performance and challenges of the implementation of the Single Market Scoreboard, viewing and comparing charts, maps, time series, selecting single or multiple variables

• Interactivity and multiple data-set integration. Among the options we will consider and make recommendations on are: making the chart interactive, ranking countries, allowing users to visualise where the data came from, combining data from multiple datasets.

• Cloud-based visualisation dashboards that can be updated in real-time. We will investigate the possibility of using open data sources with an Application Program Interface (API) and cloud-based platforms to build data visualisation tools. Similar solutions could be foreseen using Eurostat’s JSON and UNICODE Web Service.

The recommendations will also be based upon reflections regarding the structure and dimensions of the existing SMS, the use of traffic lights or other ways to signal performance levels, catering to different user needs in terms of level of detail and technical information (through for example headline and operational/managerial indicators) as well as the various platforms and communication channels.

https://ec.europa.eu/eurostat/web/json-and-unicode-web-services

https://ec.europa.eu/eurostat/web/json-and-unicode-web-services

3. Project Management

19

3 Project Management

3.1 Detailed workplan and timeline

The detailed workplan in Figure 3-1 outlines our proposed approach to successfully deliver this study. It takes into account issues discussed at the kick-off meeting. This workplan will be finalised following any remaining comments from the Commission at the inception report meeting.

3.2 Reporting and meetings

During the remaining phases of this study, we will produce/integrate the following reporting-related outputs:

• An Interim report D2 (N+10 weeks), summarising the state of play, highlighting results to date and problems encountered (Tasks 2, 3 and 4), and setting out the workplan to project completion.

• Following the Interim Report and informal IMAC meetings, integrate feedback to improve the quality, preliminary conclusions and the proposed workplan.

• A Draft Final Report D3 (Month 5), which will address all the key study issues identified in the ToR and include sound analysis, draft conclusions and recommendations

• Following the Draft Final Report and IMAC validation meetings, the analysis and recommendations will be refined to bolster the report’s technical quality.

• A Final Report D4 (Month 6), which will take into account the comments and feedback obtained from the meeting and subsequent reviews of the report.

• Alongside the final report, CSES will submit an Executive summary (D5) an abstract in English, French and German and supporting data (D6).

3. Project Management

20

Figure 3-1: Detailed workplan – SMS 2.0

PROJECT PLAN: SINGLE MARKET SCOREBOARD

Contract signature 16/07/2019 July August September October November December JanuaryWeek 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11 Week 12 Week 13 Week 14 Week 15 Week 16 Week 17 Week 18 Week 19 Week 20 Week 21 Week 22 Week 23 Week 24 Week 25 Week 26 Week 27

Task ID What 15/07 22/07 29/07 05/08 12/08 19/08 26/08 02/09 09/09 16/09 23/09 30/09 07/10 14/10 21/10 28/10 04/11 11/11 18/11 25/11 02/12 09/12 16/12 23/12 30/12 06/01 13/01

P1 Phase 1

0.1 Kick-off meeting - 17/07/2019 Ω

0.2 Team leadership/project management

0.3 Prepare letter of introduction for signature by EC

0.4 Finalise draft work plan

0.5 Prepare short note for informal IMAC meeting

1 Task 1. Literature, sources and stakeholders review (N+3 weeks)

1.01 Review of lit. and other sources

1.02 Create contacts database

1.03 Stakeholder identification

1.04 Develop draft implementation plan for stakeholder consultations

1.05 Develop a framework, criteria and heuristics tool

1.06 Develop tools to facilitate data analysis

1.07 Quality Assurance

1.08 Inception report - 05/08/19

1.09 Inception report meeting - 09/08/2019 Ω

P2 Phase 2

2 Task 2. Mapping of existing tools for monitoring Single Market policies

2.01 Detailed review of the identified tools

2.02 Store info in Excel database

3 Task 3. Identification of available data from different sources

3.1 Task 3.1 Detailed identification of available data from different sources

3.1.01 Detailed review of additional data sources

3.1.02 Store info in Excel database

3.2 Task 3.2 Stakeholder consultations

3.2.01 Finalise implementation plan for stakeholder consultations

3.2.02 Develop online questionnaires for targeted consultations

3.2.03 Identify survey sample

3.2.04 Pilot survey

3.2.05 Conduct survey

3.2.06 Produce data quality note on survey

3.2.07 Semi-structured interview guides by stakeholder category

3.2.08 Produce privacy statement linked to GDPR

3.2.09 Finalise list of interviewees

3.2.10 Conduct interviews

3.2.11 Summarise key messages from interviews

3.2.12 Prepare focus group plans and guides

3.2.13 QA and backstopping

P3 Phase 3

4 Task 4. Identification of gaps

4.01 Analyse stakeholder consultation feedback

4.02 Perform gap analysis for SMS

4.03 Interim report - 09/10/2019

4.04 Interim report meeting - 16/10/2019 Ω

4.05 Informal IMAC meeting - 21/10/2019 Ω

5 Task 5. Definition of new indicators including end user’s perspective indicators

5.01 Develop indicator definition inventory

5.02 Explore indicators concepts

5.03 Internal validation discussion/meeting on indicator concepts

5.04 Hold focus group to validate indicators

5.05 In-depth review of 'prime' indicators

6 Recommendation on indicators

6.01 Research on how real-time info can be obtained from relevant data sources

6.02 Synthesis analysis of study outputs

6.03 Develop recommendations

6.04 Validate and present recommendations

7 Presentation of upgraded scoreboard

7.01 Investigate existing dashboards/scoreboards

7.02 Review and analyse SMS data-set including new indicators

7.03 Provide proof of concept of real-time DV

7.04 Provide recommendations on DV of a new SMS

7.05 Draft Final Report - 16/12/2019

7.06 Draft Final Report meeting 19 or 20/12/2019 Ω

7.07 Revise draft report

7.08 Quality Assurance and backstopping

7.09 Submit Final Report, Exec summary and supporting data - 16/01/2020

7.10 Present report at IMAC meeting - TBC

Total

Legend

Phases

Workflow/ activity

Meetings Ω

Deliverables

Annex 1 - Bibliography

21

3.3 Assumptions

The are several study challenges and issues to be considered in the process of defining indicators for the SMS 2.0. A main challenge in upgrading the SMS is that it implies a broader measurement of the single market, not only as a policy strategy, but also as a concept in itself. Given the multifaceted nature of the Single Market, the definition of the concept to be measured is complex and associated strategic choices will be needed to decide what should not be included as well as what should.

While the existing SMS indicators measure the transposition and enforcement of the existing policies, there is a gap around examining the quality of the application of the legislation typically assessed through qualitative evaluations. In this regard, it will be important that databases that provide value judgements on performance of the policies can be identified and not just information on the outputs that do not suggest the need for action by the Member States. Examples like the World Bank’s Doing Business database helps to overcome such challenges by providing information on the extent of the regulatory and administrative burden for companies when entering and operating in different country markets by compiling data from multiple sources.

Further assumptions we are making to perform this study in the six months available include:

• We will continue to get timely access to data/information, noting that the Commission services have provided information rapidly since the contract began.

• We will need to balance the resources allocated to assessing and improving indicators for existing governance tools and policy areas with identifying new policy areas and associated indicators. In the detailed project plan, we have shifted additional resources towards identifying new policy areas and indicators.

• To keep the survey open for four weeks and integrate preliminary results into the Interim Report, we will need to launch the survey during the holiday period. The interview programme will also need to be launched rapidly and relies on some stakeholders being available in August. Alternatively, the survey could be kept open for a shorter period, but which runs the risk of reducing the number of survey responses.

• Another option is to delay the interim report and interim report meeting, for example by two weeks, which would allow time for additional stakeholder responses, gap analysis and early new indicator concepts to be developed ahead of the informal IMAC meeting. This is not expected to influence the timing of delivery of the final report since the project is currently front-loaded, while the phase 3 tasks have more time and fewer parallel interdependencies (see detailed project plan).

• Task 7 in the ToR requires that we make recommendations and give ideas on how to present new and existing data within the upgraded scoreboard, using visualisation tools ensuring that it is more innovative and user friendly. We will provide recommendations as per our proposal and ensure that we provide suggestions regarding structural aspects of an SMS 2.0. We will attempt a data visualisation prototype for an indicator/indicator concept subject to resource availability.


22

4 Next steps

A series of next steps will be required to finalise the methodology and practical elements of the data collection and analysis framework as part of the structuring phase. In summary, the next steps are:

4.1 Finalisation of Phase 1

The following Phase 1 sub-tasks proposed in our offer will be finalised shortly:

• A framework, criteria and heuristics tool for assessing the relevance, coherence and effectiveness of the existing and proposed SMS indicators to measure performance in Single Market policy implementation and identifying gaps (matrix). This tool will be shared with the client for comment.

• Finalise the initial version of the contacts database, although this will remain a work in progress.

• Finalisation and approval of this inception report, following feedback from the European Commission’s DG GROW.

4.2 Launch phase 2

The immediate tasks to launch are tasks 2-4 as well as preparation for the informal IMAC meeting planned for October 24th. The interim report will be submitted by 9th October with a meeting to discuss the report scheduled for October 16th.

The draft research tools to facilitate data collection and analysis have been finalised. These were: (1) online questionnaires for the targeted consultations (2) a series of customised and semi-structured interview guides for the different categories of stakeholder and (3) a series of focus group plans and guides and (4) the qualitative tools to analyse key patterns in interview notes and focus groups.

The online survey was launched on 6th September and will remain open until 27th September.


23


The following bibliography builds on the initial document examples provided on our technical offer. It will continue to be developed and analysed during Phase 2.

SMS area SMS sub-area Title Author

General. General An easier consumer journey when buying online | BusinessEurope

An EU New Year's Resolution: Keep Boosting the Single Market Berkjem, Johan &

Harbour, Malcolm

Annual Report on European SMEs 2017/2018 Special Background Document on the internationalisation of SMEs

Business services: Konzepte, Technologien, Anwendungen; 9. Internationale Tagung Wirtschaftsinformatik, Wien, 25. - 27. Februar 2009. Bd. 2: ...

Consumer Conditions Scoreboard - Trust in e-commerce dramatically increases - 2017 Edition

Eurasian Economic Perspectives: Proceedings of the 20th Eurasia Business and Economics Society Conference -

Bilgin, Mehmet Huseyin; Danis, Hakan; Demir, Ender; Can, Ugur

Indicators for measuring the Performance of the Single Market - Building the Single Market Pillar of the European Semester

Pelkmans, Jacques

Indicators for the internal market? An unfinished business Saltelli, Andrea

justice_scoreboard_2019_en.pdf

Making EU trade in services work for all

Mapping the cost of non-Europe, 2014-19 study Hiller, Wolfgang;

Europäisches Parlament

Opentender - Opentender Portals digiwhist

Rebooting the Single Market: The Top Priority for Eu Growth Harbour, Malcolm

Study to monitor the business and regulatory environment affecting the collaborative economy in the EU final report.

Karanikolova, Kristina; Naumanen, Mika; Juskevicius, Ricardas; Rabuel, Lison; Porsch, Lucas; European Commission; Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs

Tenders Electronic Daily (TED) (csv subset) – public procurement notices - ecodp.common.ckan.site_title

THE EU DIGITAL SINGLE MARKET FROM A CONSUMER STANDPOINT: HOW DO PROMISES MEET MEANS?

Havu, Katri

Towards Indicators for Measuring the Performance of the Single Market Pelkmans, Jacques

Scoreboards Consumer markets scoreboard

Consumer conditions scoreboard

EU Monitoring Framework for the Circular Economy

Regional Ecosystem Scoreboard (Innovation and Entrepreneurship)

European Innovation Scoreboard (also the Regional Innovation Scoreboard)

EU Justice Scoreboard

Euro indicators/PEEI Scoreboard


24

EU Open Data

European Enterprise Network

European Central Bank financial integration in Europe Annual Report

European Citizens Consultations (conducted by the European Movement International)

Tenders Electronic Daily (TED)

Public procurement database

Product Markets Regulation database

Government at a Glance

OECD business statistics

Regulatory Policy Outlook

Country and thematic reports on public procurement and digital government

Doing Business in guides

Worldwide Governance Indicators

Business Environment and Enterprise Performance Surveys

World Economic Forum Global Competitiveness Report

Institutional Profiles Database

Bertelsmann Transformation Index

Gallup world poll, with UN Sustainable Development Solutions Network

KETs database (publicly available)

Eurostat SBS

Eurostat Prodcom

Amadeus/ Orbis

FDI Markets database - Financial Times (FT)

Governance Tool

Administrative cooperation between national authorities

Consumer Protection Cooperation Network

Consumer protection in the EU: policy overview : in-depth analysis. Valant, Jana; EP;

Directorate-General for Parliamentary Research Services

EC Consumer Law Howells, Geraint G.;

Wilhelmsson, Thomas; Wilhelmsson, Thomas

EMOTA Position Paper on the European Commission Consultation "Towards a Coherent European Approach to Collevtive Redress", SEC (2011) 173 Final

EMOTA

Ensuring consumer rights are properly enforced: Revising EU Consumer Protection Cooperation

European Commission

Networks to Enforce European Law: The Case of the Consumer Protection Cooperation Network

Poncibò, Cristina

Regulation (EU) 2017/ of the European Parliament and of the Council of 12 December 2017 on cooperation between national authorities responsible for the enforcement of consumer protection laws and repealing Regulation (EC) No 2006/2004


25

Assistance services for citizens and businesses

2018 Annual Report on Intra-EU Labour Mobility

Access to public procurement contracts in EU: perspective of SMEs Bobowski, Sebastian;

Gola, Jan; Szyd\lo, Wojciech

Altius - E-procurement in Belgium: Towards a complete automation of public procurement?

Gellezak, Justin

Answer to a written question - No Polish-language version of e-Certis brochure - E-006587/2011

COMMISSION IMPLEMENTING DECISION (EU) 2018/ 170 - of 2 February 2018 - on uniform detailed specifications for data collection and analysis to monitor and evaluate the functioning of the EURES network

COMMISSION IMPLEMENTING REGULATION (EU) 2016/ 7 - of 5 January 2016 - establishing the standard form for the European Single Procurement Document

Commission Implementing Decision (EU) 2018/170 of 2 February 2018 on uniform detailed specifications for data collection and analysis to monitor and evaluate the functioning of the EURES network (Text with EEA relevance. )

Commission Implementing Regulation (EU) 2016/7 of 5 January 2016 establishing the standard form for the European Single Procurement Document (Text with EEA relevance)

Correct Application of EU Law by National Public Administrations and Effective Individual Protection: the SOLVIT Network

Lottini, Micaela

Developments in Public Procurement in Hungary–Opportunities and Challenges Rigó, Csaba Balázs

Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement and repealing Directive 2004/18/EC

European Commission

Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement and repealing Directive 2004/18/EC Text with EEA relevance

Do Invisible Borders Still Restrict Consumer Access to Services in the EU? ECC-Net

eCertis

European Commission

e-Certis KOLINKOVA, Eliska

Employers’ skill preferences across Europe: between cognitive and non-cognitive skills

Kureková, Lucia Mýtna; Beblavý, Miroslav; Haita, Corina; Thum, Anna-Elisabeth

Employment: Commission proposes to improve EURES job search network Andor, László

EU social security coordination

EURES European

Commission

European Consumer Centres Network

European Single Procurement Document (ESPD) ALEKSANDROVA, Zhenya

European single procurement document and eCertis Anonymous

Governance Issues in the EU's e-Procurement Framework Khorana, Sangeeta;

Ferguson-Boucher, Kirsten; Kerr, William A.

Governance through Cooperation: Solvit System and its Role in the Correct Implementation of the European Law by the National Public Administrations

Mătușescu, Constanța; Mares, Christopher; Gilia, Claudia


26

Home | Europass

Labour Mobility in Europe: An Untapped Resource? Barslund, Mikkel;

Busse, Matthias; Schwarzwälder, Joscha

Labour Mobility in Europe: An untapped resource? CEPS Policy Brief No. 327, March 2015

Barslund, Mikkel; Busse, Mathias; Schwarzwälder, Joscha

Labour Mobility in the Euro Area: Cure or Curse for Imbalances? auf dem Brinke,

Anna & Dittrich, Paul-Jasper

Moving & working in Europe - Employment, Social Affairs & Inclusion - European Commission

Network of legal experts (MoveS) - Employment, Social Affairs & Inclusion - European Commission

New Skills Agenda for Europe - Employment, Social Affairs & Inclusion - European Commission

Points of Single Contact European

Commission

Posting of workers - Consilium

REGULATION (EU) 2016/ 589 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL - of 13 April 2016 - on a European network of employment services (EURES), workers' access to mobility services and the further integration of labour markets, and amending Regulations (EU) No 492/ 2011 and (EU) No 1296 / 2013

Regulation (EU) 2016/589 of the European Parliament and of the Council of 13 April 2016 on a European network of employment services (EURES), workers' access to mobility services and the further integration of labour markets, and amending Regulations (EU) No 492/2011 and (EU) No 1296/2013 (Text with EEA relevance)

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL on the review of the practical application of the European Single Procurement Document (ESPD)

Single Market Scoreboard – eCertis

Skills and qualifications - Employment, Social Affairs & Inclusion - European Commission

Social Agenda

Social Scoreboard - Competence Centre on Composite Indicators and Scoreboards - EU Science Hub - European Commission

SOLVIT European

Commission

The European Commission Single Market Scoreboard Fernández Moriana, Vanessa; Vida, Alexandra Melissa; De Lombaerde, Philippe

The European Single Procurement Document Telles, Pedro

The Solvit National Coordination Mechanisms – an Organizational Analysis Moldoveanu,

George; Năstase, Bogdan

THE USE OF DIGITAL PUBLIC SERVICES BY ROMANIANS, IN LIGHT OF THE DIGITAL SINGLE MARKET.

ANIELA, BĂLĂCESCU

Will a European Unemployment Benefits Scheme Affect Labour Mobility? Alcidi, Cinzia;

Barslund, Mikkel; Busse, Matthias; Nicoli, Francesco


27

Your Europe European

Commission

Your Europe Advice European Commission

ECC-net indicators on the SMS

ECC home page

ECC 10 year anniversary report

ECC - legal role

Your Europe SMS Indicators

Your Europe Portal

Single Digital Gateway

Your Europe Advice Portal

Your Europe Advice SMS Indicators

European Consumer Action Service Portal

SOLVIT SMS Indicators

SOLVIT Portal

SOLVIT Action Plan 2017

Recommendation on Principles Governing SOLVIT 2013

Recommendation on Principles for Using SOLVIT 2013

Points of Single Contact DG GROW Webpage

Services Directive setting up the PSCs

PSC Charter

PSC SMS Indicators

An assessment of the PSCs 2015

Formal and informal cooperation between EC and MS

2019 Communication from Commission — Modification of the calculation method for lump sum payments and daily penalty payments proposed by the Commission in infringements proceedings before the Court of Justice of the European Union

European Commission, DG GROW

Best practices in the internal market - European Commission European

Commission, DG GROW

Comparing transposition of EC law in an enlarged Union - the swift, the slow, and the slack.

ECPR

Complaint Form for breach of EU law – European Commission European

Commission, DG GROW

EPC WORKING PAPER No.25 Making Europe work: improving the transposition, implementation and enforcement of EU legislation, June 2006

Lorenzo Allio and Marie-Hélène Fandel, EPC

European Commission - PRESS RELEASES - Press release - January 2019 infringements package: key decisions


Infringement Decisions European

Commission, DG GROW

Infringement procedure European

Commission, DG GROW

Infringement Proceedings in EU Law Luca Prete


28

Infringements European

Commission, DG GROW

Ministerial Transposition of EU Directives: Can Oversight Improve Performance? Radoslaw Zubek*,

Katarina Staronova

Monitoring implementation of EU directives European Commission


Monitoring the implementation of EU law: tools and challenges Ballesteros, Marta

Rule of law infringement procedures A proposal to extend the EU’s rule of law toolbox

Petra Bárd and Anna Śledzińska-Simon, CEPS (CEPS Papers in Liberty and Security in Europe, No. 2019-09, May 2019)

Sectors at Different Speeds: Analyzing Transposition Deficits in the European Union Frans van Waarden,

University College, Utrecht University and Markus Haverland, Erasmus University Rotterdam - Faculty of Social Science

Single Market Scoreboard – Transposition European

Commission


THE EU TRANSPOSITION DEFICIT Paper Haverland, Markus

Single Market Scoreboard – EU Pilot

Communication from the Commission - EU law: Better results through better application

Annual reports on monitoring the application of EU law

IMI Portal

IMI Regualtion

IMI indicators on the SMS

PolicyArea Circular and low carbon economies

2030 climate & energy framework European Commission

A European strategic long-term vision for a prosperous, modern, competitive and climate neutral economy

European Commission

A European strategy for plastics in a circular economy European

Commission

A monitoring framework for the circular economy European Commission

Accelerating the transition to the circular economy: improving access to finance for circular economy projects

European Commission

Advancing to a Circular Economy: three essential ingredients for a comprehensive policy mix

Milios L.

Analysis of certain waste streams and the potential of Industrial Symbiosis to promote waste as a resource for EU Industry

IDEA Consult, Bilsen et al


29

Binding annual greenhouse gas emission reductions by Member States from 2021 to 2030 contributing to climate action to meet commitments under the Paris Agreement and amending Regulation (EU) No 525/2013

European Commission

Cases of implementing resource efficient policies by the EU industry - Final Report IDEA Consult, Bilsen

et al

Circular economy European Commission

Delivering the circular economy: a toolkit for policymakers Ellen Macarthur

Foundation

Development and implementation of initiatives fostering investment and innovation in construction and demolition waste recycling infrastructure

IDEA Consult, Bilsen et al

Environment and Internal Market European

Commission

EU Emissions Trading System (EU ETS) European Commission

European Circular Economy Stakeholder Platform

Exchange of good policy practices promoting Innovative/Green Business Models IDEA Consult, Bilsen

et al

Fourth report on the State of the Energy Union European Commission

Governance of the energy union and climate action (EU) Regulation 2018/1999 European

Commission

Growth Within: a circular economy vision for a competitive Europe Ellen Macarthur Foundation

Identifying Levers to unlock Clean Industry - background report IDEA Consult, Bilsen

et al

Identifying Levers to unlock Clean Industry - summary report IDEA Consult, Bilsen et al

Integrating Rather than Juxtaposing Environmental Policy and the Internal Market Hans H. B. Vedder

Interrogating the circular economy: the moral economy of resource recovery in the EU

Gregson, N., Crang, M., Fuller, S., Holmes, H.

Movements of waste across the EU's internal and external borders EEA

Our Vision for A Clean Planet for All: Economic Transition European

Commission

Regulatory barriers for the Circular Economy Technopolis Group in consortium with Fraunhofer ISI, Thinkstep and Wuppertal Institute.

Report EC on the implementation of the Circular Economy Action Plan European

Commission

Staff working document with details on the 54 actions included in the action plan European Commission

Study on the Energy Saving Potential of Increasing Resource Efficiency – Final Report Oeko Insitut, IDEA

Consult, …

Technical Report accompanying the Study on the Energy Saving Potential of Increasing Resource Efficiency

Oeko Insitut, IDEA Consult, …

The Circular Economy and Benefits for Society Wijkman, A.,

Skånberg, K.

Towards a circular economy: A zero waste programme for Europe European Commission


30

Towards the circular economy Ellen Macarthur

Foundation

Treating Waste as a Resource for EU Industry ECSIP Consortium

Understanding the Circular Economy in Europe, from Resource Efficiency to Sharing Platforms: The CEPS Framework

Taranic, I., Behrens, A., & Topi, C.

Upgrading the Single Market: more opportunities for people and business European

Commission

Collaborative Economy

An empirical analysis of factors affecting sharing economy growth Virginija Grybaitė, Jelena Stankevičienė

European Perspectives on Participation in the Sharing Economy Andreotti et. al

Impact of the Sharing Economy in Germany BMBF

Measuring the collaborative economy: a case study in tourism statistics Dr Vasiliki Benaki-

Kyprioti

Multidisciplinary Framework on Commons Collaborative Economy decode

Sharing Economy Index Epicenter

Shifting Perceptions of Collaborative Consumption National League of Cities

Single Market Scoreboard: Collaborative Economy European

Commission

Study to Monitor the Economic Development of the Collaborative Economy in the EU Technopolis, VVA, Triconomics

The Collaborative Economy Nesta

The Collaborative Economy Based Analysis of Demand: Study of Airbnb Case in Spain and Portugal

Pedro R. Palos-Sanchez1 and Marisol B. Correia

The Collaborative Economy: Unlocking the power of the workplace crowd Deloitte

The Feasibility of Measuring the Sharing Economy Michael Hardie

The Impact of the Collaborative Economy on the Labour Market Willem Pieter De

Groen and Ilaria Maselli

The Sharing Economy: New Opportunities, New Questions Credit Suisse

Timbro Sharing Economy Index Timbro

Understanding the Sharing Economy World Economic

Forum

Environment A measure of protection: A baseline report on performance measurement in the Scottish Environment Protection Agency

SEPA

Assessing Environmental Performance in the GMs GMS-CEP

Environment, Energy, and Economic Development RAND Corporation

Environmental indicator frameworks to design and assess environmental monitoring programs

Tomás B. Ramos , Sandra Caeiro & João Joanaz de Melo

Environmental Indicator Report 2017 EEA

Environmental Indicator Report 2018 EEA

Environmental Indicators EEA

OECD

Environmental Measures and Modeling: Monitoring Programmes EPA

Environmental metrics: government’s approach to monitoring the state of the natural environment

National Audit Office (UK)

Environmental Performance Index Yale


31

Environmental Performance Report 2017 National renewable

Energy Laboratory (USA)

Environmental Performance Review of Germany OECD

Environmental Performance Review: Albania UNECE

Environmentl Indicators: A Systematic Approach to Measuring and Reporting on Environmental Policy Performance in the Context of Sustainable Development

World Resources Institute

Ideas for defining environmental objectives and monitoring systems for a results-oriented CAP post 2020

Institute for European Environmental Policy

National Environment Monitoring Swedish EPA

Financial markets

Banking and financial services European Parliament

BetterFinance European

Commission - European Commission

Capital markets union Financial Times

Capital Markets Union: An Action Plan of Unfinished Reforms Constantin Gurdgiev,

Trinity College, Dublin C, April 2019, with Tracy Lee Lyon, Alexandra Cohen, Margaret Poda and Matthew Salyer (Middlebury Institute of International Studies at Monterey (MIIS); GUE/NGL Group, European Parliament, Policy Analysis Paper, March 2019

Capital Markets Union: Creating a stronger and more integrated European financial supervisory architecture, including on anti-money laundering

European Commission

Communication from the Commission on anti-money laundering (COM(2018) 645 final. Strengthening the Union framework for prudential and anti-money laundering supervision for financial institutions.

European Commission

Communication from the Commission on capital market union: Progress on building a single market for capital for a strong economic and monetary union (SWD(2019) 99 final)

European Commission

Financial services policy | Fact Sheets on the EU | European Parliament European

Parliament, Lexalert

Legislative train schedule European Commission

Rebranding Capital Markets Union European Parliament

Regulation (EU) No 345/2013 of the European Parliament and of the Council of 17 April 2013 on European venture capital funds

European Commission

Summary overview of EU policies on banking and financial services European

Commission

The EU needs to pick up the pace on capital markets union CEPS

The Free Movement of Capital and Financial Services (Cambridge Scholars) Graeme Baber, Cambridge Scholars

The Single Market: Financial Services and the Free Movement of Capital


32

General Trade in Goods and Services SMS Indicators

DG GROW Single Market for Goods

DG GROW Single Market for Services

PRODCOM Trade in Goods Statistics

Eurostat Balance of Payments

Eurostat international trade in services

Network industries

2015-12-analysis-and-recommendation-on-collection-of-data-in-the-field-of-general-aviation-in-europe.pdf

Air Smith, John

Air transport: market rules | Fact Sheets on the European Union | European Parliament

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS Progress towards completing the Internal Energy Market

Directive (EU) 2019/944 of the European Parliament and of the Council of 5 June 2019 on common rules for the internal market for electricity and amending Directive 2012/27/EU (Text with EEA relevance.)

Energy consumer rights user_administrator

Energy Union Data & analysis user_administrator

Energy Union indicators user_administrator

Evaluation of the Performance of Network Industries Providing Services of General Economic Interest

Buelens, Christian

EVALUATION of the Regulation (EC) No 1008/2008 on common rules for the operation of air services in the Community

Executive summary of the evaluation of the Regulation (EC) No 1008/2008 on common rules for the operation of air services in the Community

Internal energy market | Fact Sheets on the European Union | European Parliament

Internal market Smith, John

Market Smith, John

Market Functioning in Network Industries Electronic Communications, Energy and Transport

Brons, Martijn; Bucher, Anne; Europäische Kommission; Generaldirektion Wirtschaft und Finanzen

Market functioning in network industries: electronic communications, energy and transport

Market legislation user_administrator

Monitoring progress towards the Energy Union objectives - Concept and first analysis of key indicators

Monitoring progress towards the Energy Union objectives – key indicators

Network industries: efficient regulation, affordable and adequate services Centre on

Regulation in Europe

Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and repealing Regulation (EC) No 216/2008 of the European Parliament and of the Council COM/2015/0613 final - 2015/0277 (COD)

Rail Market Monitoring (RMMS) BORMANS, Yves

Rail transport | Fact Sheets on the European Union | European Parliament

Recast of the first railway package (Dir 2012/34/EU) Anonymous


33

Regulation (EU) 2019/943 of the European Parliament and of the Council of 5 June 2019 on the internal market for electricity (Text with EEA relevance.)

Regulation of network industries in the European Union and in Central and Eastern Europe

Major, Iván; Kiss, Károly M.

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL: Sixth report on monitoring development of the rail market

Standing Up for Competition: Market Concentration, Regulation, and Europe’s Quest for a New Industrial Policy |

The Future of Rail: Regulation & Competition for an Innovative Industry

The Future of the Digital Single Market

Towards Interoperability within the EU for Electricity and Gas - Data Access and Exchange (Report)

Anonymous

Towards Interoperability within the EU for Electricity and Gas Data Access & Exchange

European Smart Grids Task Force; Expert Group 1 – Standards and Interoperability; Working Group on Data Format & Procedures

What do we want to achieve? Anonymous

Postal Services

Development of cross-border e-commerce through parcel delivery : final report. Union, Publications Office of the European

Directive 2008/6/EC of the European Parliament and of the Council of 20 February 2008 amending Directive 97/67/EC with regard to the full accomplishment of the internal market of Community postal services

Directive 98/5/EC of the European Parliament and of the Council of 16 February 1998 Smith, Rhona

ERGP PL (17) 36 Report on Core Indicators 15 December 2017_.pdf

ERGP Report on the outcome of the ERGP public consultation on the evolution of the Universal Service Obligation

Maraschin, Emma

EU postal legislation, implementation and enforcement Anonymous

How customer demands are reshaping last-mile delivery | McKinsey

IPC Global Postal Industry Report

IPC Postal Sector Sustainability Report 2018

Main developments in the postal sector (2010-2013). Union, Publications

Office of the European

Main developments in the postal sector (2013-2016). Union, Publications

Office of the European

Postal sector studies Anonymous

Postal statistics Anonymous

Statistics | Internal Market, Industry, Entrepreneurship and SMEs | European Commission

Public Procurement

Benchmarking Public Procurement 2017 World Bank Group

Buying Green! Handbook European

Commission

Commission Implementing Regulation (EU) 2016/7 of 5 January 2016 establishing the standard form for the European Single Procurement Document (Text with EEA relevance)

European Commission


34

Corruption in Public Procurement: Finding the Right Indicators Joras Ferwerda,

Ioana Deleanu, Brigitte Unger

Determinants of direct cross-border public procurement in EU Member States Zornitsa Kutlina-

Dimitrova, Csilla Lakatos

Digiwhist public procurement data Digiwhist Project

Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement and repealing Directive 2004/18/EC (Text with EEA relevance) Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement and repealing Directive 2004/18/EC (Text with EEA relevance)

European Commission

EU Open Data Portal European

Commission

Evaluating the Effectiveness of Public Procurement Performance Management Systems in Local Governments

Andrea Stefano Patrucco, Davide Luzzini & Stefano Ronchi

Measuring procurement performance in Europe Anthony Flynn

New ways to measure institutionalised grand corruption in public procurement U4

Opentender Digiwhist Project

Preferential Public Procurement for Small and Medium Enterprises DCED

Project Performance Indicators as an Innovative Tool for Identifying Sustainability Perspectives in Green Public Procurement

Konstantinos I.Vatalis, Odysseus G.Manoliadis, Dimitrios G.Mavridisa

Public Procurement for a Circular Economy European

Commission

Public Procurement Indicators 2015 European Commission

Public Procurement Toolbox OECD

Public Procurement Topic Guide Transparency

International

Single Market Scoreboard: Public Procurement European Commission

SMEs in Public Procurement: Practices and Strategies for Shared Benefits European

Commission

Strategic Public Procurement European Commission

TED dataset: Public Procurement Notices European

Commission

Social 2018 Annual Growth Survey and Joint Employment Report - Council conclusions (15 March 2018)

Coordination of Social Security Systems in Europe Giubboni, Stefano;

Iudicone, Feliciano; Mancini, Manuelita; Faioli, Michele

Document database - Employment, Social Affairs & Inclusion - European Commission

EU social indicators - Employment, Social Affairs & Inclusion - European Commission

EU social indicators dataset - Employment, Social Affairs & Inclusion - European Commission

EU social security coordination


35

Free movement - EU nationals - Employment, Social Affairs & Inclusion - European Commission

https://composite-indicators.jrc.ec.europa.eu/social-scoreboard Social Scoreboard

https://ec.europa.eu/social/main.jsp?catId=737&langId=en&pubId=8183&furtherPubs=yes

Indicators' sub-group - Employment, Social Affairs & Inclusion - European Commission

Portfolio of EU Social Indicators for the Monitoring of Progress Towards the EU Objectives for Social Protection and Social Inclusion

Social Protection Committee; Indicators Sub-group

Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL amending Regulation (EC) No 883/2004 on the coordination of social security systems and regulation (EC) No 987/2009 laying down the procedure for implementing Regulation (EC) No 883/2004 (Text with relevance for the EEA and Switzerland) - Analysis of the final compromise text with a view to agreement

Social Protection Committee - Employment, Social Affairs & Inclusion - European Commission

Professional Qualifications

Professional Qualifications SMS Indicators

Professional Qualifications Directive 2005

Regulated Professions Database

Regulated Professions Interactive Map

Digital single market

The Future of the Digital Single Market Alexandre De Streel, Univ of Namur

Digital Single Market: bringing down barriers to unlock online opportunities European

Commission

Shaping the Digital Single Market European Commission

Eurobarometer survey on new rules ending unjustified geo-blocking and cross-border portability

European Commission

Factsheet on the DSM: A digital single market for the benefit of all Europeans European

Commission

A Europe that protects: EU reports on progress in fighting disinformation ahead of European Council

European Commission

Commission's report shows that targeted investment and robust digital policies boost Member States' performance

European Commission

Digital Single Market: Europe announces eight sites to host world-class supercomputers

European Commission

New EU rules on e-commerce European

Commission

Building a European data economy European Commission

Connectivity for a European Gigabite society European

Commission

Digital skills and jobs European Commission

Digital skills training blueprints for upskilling SME employees and unemployed persons

BlueSpecs, et al.

Artificial Intelligence, Real Benefts Viola R., European

Commission

Digital Agenda Scoreboard key indicators — Digital Scoreboard - Data & Indicators European Commission

The Data Monitoring Tool European

Commission


36

2019 DESI Report - Electronic communications markets overview per Member State (Telecom chapters)

European Commission

TradeMarket

FDI European Union Foreign Direct Investment [1999 - 2019] [Data & Charts] CEIC Data

FDI Regulatory Restrictiveness Index - OECD OECD

Foreign direct investment and its drivers: a global and EU perspective ECB

Foreign direct investment report: continuous rise of foreign ownership of European companies in key sectors

European Commission

Foreign Direct Investment Screening in the EU Degreef, C

Intra-EU trade in goods - recent trends - Statistics Explained Eurostat

Market access: Trade barriers OECD

Measures Of Restrictions On Inward Foreign Direct Investment For OECD Countries Golub, Stephen, S.

The determinants of Foreign Direct Investment F Mistura, C Roulet

The determinants of foreign direct investment into European transition economies Alan A bevin,S Estrin

Trade Policy COMMISSION STAFF WORKING DOCUMENT on FOREIGN DIRECT INVESTMENT IN THE

EU Following up on the Commission Communication "Welcoming Foreign Direct Investment while Protecting Essential Interests" of 13 September 2017

European Commission

Scoreboard - Eurostat Eurostat

Services Trade Restrictiveness Index - Trends up to 2019 OECD

Services Trade Restrictiveness Index Regulatory Database OECD

The OECD Services Trade Restrictiveness Index provides a snapshot of services regulatory regimes to help policymakers to assess reform options.

OECD

Trade in Value Added - OECD OECD

World Integrated Trade Solution World bank

study to develop an upgraded single market scoreboard as a … · 2019-09-18 · centre for...

Documents