gaps and errors in the ted database - european parliament€¦ · gaps and errors in the ted...

Gaps and Errors

in the TED database

Budgetary Affairs

Policy Department D for Budgetary Affairs

Directorate General for Internal Policies of the Union PE 621.804 - February 2019

IN-DEPTH ANALYSIS requested by the CONT Committee

EN

DIRECTORATE GENERAL FOR INTERNAL POLICIES

POLICY DEPARTMENT D: BUDGETARY AFFAIRS

Gaps and Errors in the TED database

NOTE

Abstract The accessibility of TED public procurement data has improved in recent years. However, downloadable TED data up to 2017 is complex and includes numerous gaps and errors. Some indicators suggest that much public procurement in the European Union is not published on TED at all. The Commission is in the process of implementing a major reform of public procurement rules and processes which is expected to improve the quality of TED data. However, the new rules still permit much data to be marked as ‘not for publication’.

IP/D/CONT/IC/2018-154 25/02/2019

PE621.804 EN

This document was requested by the European Parliament's Committee on Budgetary Control. It designated its Chair, Dr Ingeborg Grässle, to follow the note. AUTHORS Blomeyer & Sanz: Mr Roderick Ackermann Ms Margarita Sanz Mr Antonio Sanz RESPONSIBLE ADMINISTRATOR Ms Vera Milicevic Policy Department on Budgetary Affairs European Parliament B-1047 Brussels E-mail: [email protected] LINGUISTIC VERSIONS Original: EN ABOUT THE EDITOR To contact the Policy Department or to subscribe to its newsletter please write to: [email protected] Manuscript completed in February 2019. Brussels, © European Union, 2019. This document is available on the Internet at: http://www.europarl.europa.eu/studies DISCLAIMER The opinions expressed in this document are the sole responsibility of the author and do not necessarily represent the official position of the European Parliament. Reproduction and translation for non-commercial purposes are authorised, provided the source is acknowledged and the publisher is given prior notice and sent a copy.

http://www.europarl.europa.eu/studies

Gaps and Errors in the TED database ____________________________________________________________________________________________

3

CONTENTS

CONTENTS ........................................................................................................................................... 3

LIST OF ABBREVIATIONS .................................................................................................................... 4

LIST OF TABLES ................................................................................................................................... 4

LIST OF FIGURES .................................................................................................................................. 4

EXECUTIVE SUMMARY ........................................................................................................................ 5

1 INTRODUCTION ........................................................................................................................... 7

1.1 Context ............................................................................................................................................................................................... 7

2 MAIN QUESTIONS ........................................................................................................................ 9

2.1 Different single-bid contracting figures ................................................................................................................................ 9

2.2 Deficiencies of TED data ............................................................................................................................................................. 11

2.3 Origins of TED gaps and errors ................................................................................................................................................ 17

2.4 Patterns of TED gaps and errors .............................................................................................................................................. 20

2.5 Areas for enhancement of TED................................................................................................................................................ 21

3 RECENT DEVELOPMENTS AND FUTURE CHALLENGES ............................................................ 22

3.1 Recent developments ................................................................................................................................................................. 22

3.2 Future challenges ......................................................................................................................................................................... 24

4 CONCLUSIONS ........................................................................................................................... 26

REFERENCES ...................................................................................................................................... 27

Policy Department D: Budgetary Affairs __________________________________________________________________________________________

4

LIST OF ABBREVIATIONS

CAN Contract Award Notice

DG Directorate General

EC European Commission

EU European Union

EUR Euro

GROW European Commission Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs

IT Information Technology

POEU Publications Office of the European Union

REGIO European Commission Directorate-General Regional and Urban Policy

TED Tenders Electronic Daily

XML Extensible Markup Language

LIST OF TABLES

Table 1: Explanation of selected TED fields 11

Table 2: Percentage of TED Contract Award Notice records missing values from three specific fields 12

Table 3: CANs with most impossibly high award amount 16

LIST OF FIGURES

Figure 1: Percentage of TED records (2009-2017) missing data in three fields (by country) 14

Figure 2: Number of national announcement types in use in Member States 18

Figure 3: Extent of missing information in TED data in 2009-2015 20


5

EXECUTIVE SUMMARY

INTRODUCTION

This note has been requested by the European Parliament to provide insight into the limitations of the Tenders Electronic Daily (TED) data as a tool for analysing public procurement in the European Union (EU). The Parliament’s request emerged from a discussion on single-bid contracting involving EU funds that took place at the presentation of the European Parliament study ‘EU-funded large-scale infrastructure: deficient project preparation and procurement processes?’ (hereinafter ‘the previous study’)

This briefing paper addresses the following issues:

• Why are the single-bid contracting figures presented in the previous study different from the Directorate-General Regional and Urban Policy (DG REGIO) figures discussed at the meeting of European Parliament Committee on Budgetary Control on 26 November 2018?

• What are the deficiencies of TED data that limit its usefulness as a tool for analysis of public procurement in the EU?

• Why/ how do these issues arise? • Do the issues arise mainly in specific Member States or institutions? • Recent changes and future challenges.

BACKGROUND

European Commission Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs (DG GROW) is responsible for developing EU public procurement policy. The Publications Office of the European Union (POEU) is responsible for the publication of TED, which is the EU-wide platform for the publication of public procurement notices. According to the European Commission TED ‘publishes 520 thousand procurement notices a year, including 210 thousand calls for tenders which are worth approximately EUR 420 billion’. 1

TED deficiencies and their causes

The accessibility of TED data has significantly improved in recent years. However, the quality and reliability of TED data for 2009-2017 is undermined by large amounts of missing data, inconsistency in the way that data is entered, and errors in the data entered, including impossibly high figures for the value of some contract awards.

Gaps appear to be more prevalent in the data submitted by Member States in Western and Northern Europe, while those in Central and Eastern Europe appear to provide more complete data.

Perhaps more significantly, the value of TED as a tool to analyse public procurement in the EU is constrained by the fact that there is great variation in the extent to which Member States publish public procurement information on TED.

Gaps and errors in the 2009-2017 dataset are attributed to: the limited number of mandated (compulsory) fields; the fragmentation and complexity of procurement systems in Member States; lack of compatibility between TED and Member State systems and the need to enter data twice – once in Member State public procurement systems and then in TED; lack of compliance with and enforcement

1 Publications Office of the European Union (13/02/2019), TED tenders electronic daily Supplement to the Official Journal of the EU, https://ted.europa.eu/TED/main/HomePage.do

https://ted.europa.eu/TED/main/HomePage.do


6

of EU legislation; the structure of the TED platform, which is for publication, not data analysis; and the limited mandate of the Publications Office of the European Union.

Recent developments and future challenges

The 2009-2017 TED dataset is covered by an old standard. The Commission is in the process of implementing a major reform of public procurement rules and processes that is expected to improve the quality and reliability of TED data. The new standard is based on a series of directives of 2014, and the Commission is planning to publish a new implementing regulation in May-June 2019 that will fully operationalise the 2014 directives by 2023, the deadline for Member states to implement its requirements. Thus, the entire reform process will have taken approximately 12 years from when the impact assessment was published in 2011. Key improvements include a move from ‘paper logic’ to a full ‘IT logic’ and the elimination of the need for contracting authorities to enter procurement data twice – once in Member State public procurement systems, and then again in TED.

Nevertheless, the new TED standard still allows much data to be marked as ‘not to be published’, with the result that much data may still missing. Moreover, some indicators suggest that much public procurement is not published in TED at all, and it is unclear to what extent ongoing reforms will address this unless they are accompanied by political pressure and effective enforcement actions against non-compliant contracting authorities.


7

1 INTRODUCTION

This briefing paper has been requested by the European Parliament to provide insight into the limitations of the Tenders Electronic Daily (TED) data as a tool for analysing public procurement in the European Union (EU). The Parliament’s request emerged from a discussion on single-bid contracting involving EU funds that took place at the presentation of the European Parliament study ‘EU-funded large-scale infrastructure: deficient project preparation and procurement processes?’ (hereinafter ‘the previous study’).2

During that discussion, it was noted that the figures for single-bid contracting presented in a case study within the previous study were lower than a figure previously provided by (EC) European Commission Directorate-General Regional and Urban Policy (DG REGIO). The previous study highlighted significant constraints in working with the downloaded TED data (on which the case study was based). 3 In particular, numerous records were missing data necessary for the analysis, there were errors in the data, and the accuracy of some information appeared to be highly doubtful.

This briefing paper addresses the following questions:

• Why are the single-bid contracting figures presented in the previous study different from the figure attributed to DG REGIO on 26 November 2018?

• What are the deficiencies of TED data that limit its usefulness as a tool for analysis of public procurement in the EU?

• Why/ how do these issues arise? • Do the issues arise mainly in specific Member States or institutions? • How might the reliability of TED data be enhanced?

The following were consulted in the course of the research for this review:

• The Publications Office of the European Union (POEU) • Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs (DG GROW) • Directorate-General Regional and Urban Policy (DG REGIO) • Mr. Mihály Fazekas

1.1 CONTEXT

DG GROW is responsible for developing EU public procurement policy. The POEU is responsible for the publication of TED, which is a Supplement to the Official Journal of the EU publication platform for the publication of public procurement notices. According to the EC, TED ‘publishes 520 thousand procurement notices a year, including 210 thousand calls for tenders which are worth approximately EUR 420 billion’. 4

The EC recognises the importance of TED data as a governance tool and is well aware of the problems with the procurement data published via TED, and the constraints this places on analysis of public procurement in the EU. The regulatory impact assessment, published in 2011, identified non-

2 Blomeyer & Sanz (12/09/2018), EU-funded large-scale infrastructure: deficient project preparation and procurement processes? https://bit.ly/2CQ3orK 3 EU Open Data Portal, ‘Tenders Electronic Daily (TED) (csv subset) – public procurement notices’, https://data.europa.eu/euodp/en/data/dataset/ted-csv 4 Publications Office of the European Union (13/02/2019), TED tenders electronic daily Supplement to the Official Journal of the EU, https://ted.europa.eu/TED/main/HomePage.do

https://bit.ly/2CQ3orK

https://data.europa.eu/euodp/en/data/dataset/ted-csv



8

compliance with publication requirements as one of three major weaknesses in the system at that time.5 A series of new directives was introduced in 2014 to address this issue, among other things. Directive 2014/24/EU states ‘The electronic systems for publication of those notices, managed by the Commission, should also be improved with a view to facilitating the entry of data while making it easier to extract global reports and exchange data between systems’.6

The EC is planning to publish a new implementing regulation in May-June 2019 dealing with seamless, automatic transmission of public procurement data between Member State public procurement systems and TED. This focuses on ‘a major update to the public procurement standard forms’, which is intended ‘to significantly improve the usefulness of data on European procurement for all users: companies, governments and citizens’.7

This is a major undertaking as it requires a change from ‘paper logic’ to ‘IT logic’, as well as a major investment in IT infrastructure at the POEU.8 DG GROW considers that, overall, the necessary IT changes will take five years to implement.

5 European Commission (20/12/2011), Impact Assessment Accompanying the Document Proposal for a Directive of the European Parliament and of the Council On Public Procurement and the Proposal for a Directive of the European Parliament and of the Council On Procurement by Entities Operating in the Water, Energy, Transport and Postal Sectors, p23, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52011SC1585 6 (26 February 2014), Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement and repealing Directive 2004/18/EC Text with EEA relevance, introduction (126) https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32014L0024 7 European Commission DG GROW (11/02/2019), Commission Implementing Regulation establishing standard forms for the publication of notices in the field of public procurement ("eForms") – public feedback explanatory note, https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2019-797630_en 8 The Publications Office of the EU is preparing to launch a tender for new IT systems.

https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52011SC1585


https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32014L0024


https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2019-797630_en


9

2 MAIN QUESTIONS

KEY FINDINGS

• The accessibility of TED data has significantly improved in recent years. However, the quality and reliability of TED data for 2009-2017 is undermined by large amounts of missing data, inconsistency in the way that data is entered, and errors in the data entered, including impossibly high figures for the value of contract awards.

• Gaps appear to be more prevalent in the data submitted by Member States in Western and Northern Europe, while those in Central and Eastern Europe appear to provide more complete data.

• Perhaps more significantly, the value of TED as a tool to analyse public procurement in the EU is constrained by the fact that there is great variation in the extent to which Member States publish public procurement information on TED.

• Gaps and errors in the 2009-2017 dataset are attributed to: the limited number of mandated (compulsory) fields; the fragmentation and complexity of procurement platforms in Member States; lack of compatibility between TED and Member State systems and the need to enter data twice – once in Member State public procurement systems and then in TED; lack of compliance with and enforcement of EU legislation; the structure of the TED platform, which is for publication, not data analysis; and the limited mandate of the POEU.

2.1 DIFFERENT SINGLE-BID CONTRACTING FIGURES

Why are the single-bid contracting figures presented in the previous study different from the DG REGIO figure discussed on 26 November 2018?

The previous study concluded that, overall, ‘single-bid contracting accounted for approximately 12% of all contracts (by value) from 2009 to 2017’.9 It noted that single-bid contracting appeared to be less common where contracts involved EU funding, compared with contracts that did not involve EU funding.

During the discussion following the presentation of the previous study to the European Parliament Committee on Budgetary Control on 26 November 2018, reference was made to a single-bid contracting rate of 35%, which came from a DG REGIO report.10 A concern was expressed that the overall figure given in the previous study was significantly lower than figures in the DG REGIO report.

Firstly, the DG REGIO single-bid figures refer specifically to TED data from 2014 (‘all figures from the last two rows [of Table 2] are taken from DG GROW’s TED Structured dataset from 2014’), 11 whereas the previous study analysed TED data from 2009 to 2017 inclusive. Secondly, the DG REGIO report does not appear to include an overall (EU-wide) figure that can be directly compared with the overall figure

9 It is important to note here that the methodology used in the previous study eliminated a large number of records from the analysis due to missing data, and the use of threshold values to exclude contracts with small values and contracts with doubtfully large values. Thus, when referring to ‘all contracts’ it was referring to all contracts remaining after the records had been filtered. 10 European Commission/ PWC (January 2016), Stock-taking of administrative capacity, systems and practices across the EU to ensure the compliance and quality of public procurement involving European Structural and Investment (ESI) Funds, p103-104, https://publications.europa.eu/en/publication-detail/-/publication/d1082259-0202-11e6-b713-01aa75ed71a1 . 11 Ibid. p59.

https://publications.europa.eu/en/publication-detail/-/publication/d1082259-0202-11e6-b713-01aa75ed71a1


10

presented in the previous study. The analysis in the previous study showed an overall spike (at EU level) in the rate of single-bid contracting recorded in TED 2014.12 For Hungary and Poland, the previous study showed single bid contracting rates in 2014 of approximately 42% and 52% respectively, 13 compared with 36% and 45% respectively in the DG REGIO report. Thus, the figures given in the previous study for these two Member States are actually higher than in the DG REGIO report. Although the two sets of figures are different, they do nevertheless both indicate that rates of single bid contracting in Hungary and Poland were relatively high in 2014 compared with the EU overall.

It is not possible to analyse the specific reasons for the differences between the two sets of figures for 2014, as it unclear what processing operations were applied to the data to generate the results in the DG REGIO report. However, it is possible to provide general indications as to how different analyses of the single-bid contracting phenomenon can reach different conclusions. For example:

• The use of different sources of data (TED, national data, or a combination of the two); • Analysis of different time periods; • The use of contract values versus number of contracts; • Where contract values are used, the use of different minimum and maximum thresholds; • Focus on different types of contracts; • Focus on different financial instruments; • Different approaches to handling missing and/ or doubtful data.

It is understood that DG REGIO is currently preparing a study on single-bid contracting but this was not available at the time of writing and DG REGIO was unable to share any data relating to this study.

A 2017 academic analysis on TED data for 2009-2014, which focused on roads, railways, airports, water transport, power plants, water distribution and sewer system, and sewage processing, found that single-bid contracting ranged between approximately 37% and 1% of contracts awarded. The rate varied according to the region of Europe and the sector, and within certain Member States it varied significantly between regions.14

12 Blomeyer & Sanz (12/09/2018), EU-funded large-scale infrastructure: deficient project preparation and procurement processes?, Figure 9, p52, https://bit.ly/2CQ3orK 13 Ibid., Figure 14, p57 and Figure 12, p55. 14 Mihály Fazekas and Bence Tóth (March 2017), Infrastructure for Whom? Corruption Risks in Infrastructure Provision Across Europe, https://www.researchgate.net/publication/316338624_Infrastructure_for_Whom_Corruption_Risks_in_Infrastructure_Provision_Across_Europe


https://www.researchgate.net/publication/316338624_Infrastructure_for_Whom_Corruption_Risks_in_Infrastructure_Provision_Across_Europe



11

2.2 DEFICIENCIES OF TED DATA

What are the deficiencies of TED data that limit its usefulness as a tool for analysis of public procurement in the EU?

2.2.1 Missing data

The analysed literature refers to ‘missing, incomplete and erroneous data as one of the fundamental problems of the European public procurement data system’15. During the period 2009-2015, 15% of the TED mandatory fields were empty.16 However, this only presents part of the picture, since even where all of the mandatory information is correctly provided, the information provided is still likely to be insufficient to carry out proper analysis. This is because various important pieces of information, such as the number of bids and the value of awarded contracts, are not mandatory.

The previous study noted that approximately 42% of the 4,630,484 downloaded records covering the years 2009 to 2017 are missing data that is important for analysis of single-bid contracting. Three fields were analysed in particular. These are explained in Table 1 below.

Table 1: Explanation of selected TED fields

FIELD NAME USED HERE

FIELD NAME IN TED DATASET

EXPLANATION

EU Funds B_EU_FUNDS The contract is related to a project and / or programme financed by European Union funds

Amount of Award

VALUE_EURO_FIN_1 CAN value, in EUR, without VAT. If a value variable is missing, this variable looks for it in all other fields from which it could be taken.

Number of Offers

NUMBER_OFFERS Number of tenders received

Source: Blomeyer & Sanz17

Table 2 below presents the percentage of records missing values in one or more of these three fields. It should be noted that the downloaded data consists of approximately 90 fields (some of these fields are generated by DG GROW processing operations) and it is probable that even where records include values in the three fields considered here, they are missing values in one or more of the other fields.

15 Mara Mendes Open Knowledge Foundation Germany, Mihály Fazekas University of Cambridge (31/03/2017), Recommendations for the Implementation of Open Public Procurement Data An Implementer‘s Guide, p13, http://digiwhist.eu/publications/digiwhist-recommendations-for-the-implementation-of-open-public-procurement-data-an-implementers-guide/ 16 Mara Mendes Open Knowledge Foundation Germany, Mihály Fazekas University of Cambridge (31/03/2017), Recommendations for the Implementation of Open Public Procurement Data An Implementer‘s Guide, p13, http://digiwhist.eu/publications/digiwhist-recommendations-for-the-implementation-of-open-public-procurement-data-an-implementers-guide/ 17 Blomeyer & Sanz (12/09/2018), EU-funded large-scale infrastructure: deficient project preparation and procurement processes? https://bit.ly/2CQ3orK

http://digiwhist.eu/publications/digiwhist-recommendations-for-the-implementation-of-open-public-procurement-data-an-implementers-guide/






12

Table 2: Percentage of TED Contract Award Notice records missing values from three specific fields

% OF ENTRIES WITH MISSING VALUES

YEAR ENTRIES MISSING

ONE OR MORE ITEMS

ENTRIES MISSING EU

FUNDS

ENTRIES MISSING AMOUNT OF

AWARD

ENTRIES MISSING NUMBER OF OFFERS

2009 46 24 26 20

2010 46 25 27 20

2011 45 25 27 19

2012 44 23 27 19

2013 45 24 27 18

2014 44 23 27 19

2015 45 23 28 19

2016 38 17 22 17

2017 31 10 13 21

Source: Blomeyer & Sanz18

This analysis is echoed by DG GROW, which notes gaps ‘especially regarding the number of bids as well as the final value of an award’.19

This varied between 46% (2009 and 2010), and 31% (2017). Although the records for 2017 appear to be more complete than in previous years, this apparent improvement is not linear (not consistent from year to year), and it may be illusory. For example:

• The downloaded data has been processed by DG GROW to address gaps and anomalies and this processing is based on certain assumptions.20

• While fewer records are missing data, this does not guarantee the quality (accuracy) of the data; • Improvements in the completeness of records may, in part, be attributable to an increase in the

number of records that are missing altogether (i.e. a relative increase in the non-use of TED by contracting authorities in one or more Member States);

• Improvements may be due to some other factor(s), rather than to improvements in the TED system, and thus such improvements may be transient;

18 Blomeyer & Sanz (12/09/2018), EU-funded large-scale infrastructure: deficient project preparation and procurement processes? https://bit.ly/2CQ3orK 19 European Commission (Benedikt Herz and Xosé-Luís Varela-Irimia) (June 2017), Border Effects in European Public Procurement, p3, http://ec.europa.eu/docsroom/documents/26201 20 DG GROW (23/02/2018) ‘TED CSV open data NOTES & CODEBOOK Version 3.1 (2018-02-23)’, http://data.europa.eu/euodp/repository/ec/dg-grow/mapps/TED(csv)_data_information.pdf Note that the document does not explicitly name DG GROW as the author, but this is inferred from the contact email address.


http://ec.europa.eu/docsroom/documents/26201

http://data.europa.eu/euodp/repository/ec/dg-grow/mapps/TED(csv)_data_information.pdf


13

• The analysis of missing data in the previous study focused on just three data fields that were central to the analysis of single-bid contracting. However, there are approximately 90 data fields in the downloaded data, and improvements in the completeness of the three fields of interest does not automatically imply improvements in the completeness of all fields. Indeed, it is possible that completeness in other fields may have deteriorated.

DG GROW points out another important gap, namely that non-winning bidders are not identified in TED records.21

Figure 1 below presents an analysis of data missing from three TED fields in downloaded TED data for 2009-2017.22 When considering this analysis, it is important to bear in mind that there is great variation between Member States in the extent to which they actually published public procurement on TED during these years. This is discussed further in section 2.2.4.

With regard to value of contract awards, the analysis considers only if a value is present or not. However, it is not uncommon to find the value entered is ‘0’.

21 European Commission (Benedikt Herz and Xosé-Luís Varela-Irimia) (June 2017), Border Effects in European Public Procurement, p3, http://ec.europa.eu/docsroom/documents/26201 22 The downloaded dataset comprised 4,630,484 records. Records relating to framework contracts were excluded from the analysis (see 2.2.2). The analysis was carried out on the following three fields: B_EU_FUNDS; AWARD_VALUE_EURO_FIN_1, NUMBER_OFFERS.



14

Figure 1: Percentage of TED records (2009-2017) missing data in three fields (by country)

Source: author analysis of downloaded TED data for 2009-2017

2.2.2 Framework contracts

Data on framework contracts in the downloaded TED data is particularly problematic. There appear to be two general approaches to entering framework contract data into the system, and both limit the usefulness of TED data for analysis of public procurement. There are variations on these two approaches:

0% 10% 20% 30% 40% 50% 60% 70% 80%

AT

BE

BG

CY

CZ

DE

DK

EE

ES

FI

FR

GR

HR

HU

IE

IT

LI

LT

LU

LV

MT

NL

PL

PT

RO

SE

SI

SK

UK

Records (contract awards) missing number of bids

Records (contract awards) missing award amount

Records (contract awards) missing indication of involvement of EU funds


15

a) The entire value of the framework is entered against the name of each economic operator included in the framework. This is understood to take place when economic operators have been selected to be included in the framework, and thus possibly before any contracts have been awarded within the framework:

o This gives the appearance of inflating the value of the framework by many times; o It gives the appearance that contracts have been awarded within the framework, when this

is not the case; o It greatly distorts the value of contracts awarded to individual economic operators;

b) In other cases, the total value of the framework is included once, and zeros or null values may be entered against the name of each economic operator included in the framework. This correctly shows that no contracts have been awarded when the results of the selection of the framework contractors is announced. However, the results of subsequent analysis may be distorted later when the contracting authority enters the values of each contract awarded within the framework – this is desirable from the perspective of transparency, but when combined with the overall value of the framework already entered into the system, the recorded total value must presumably greatly exceed the actual total value of the framework, which undermines transparency;

c) It is possible that in either of the two examples described above (a or b), having initially identified the economic operators that have been selected for inclusion in the framework, the contracting authority does not subsequently enter the values of individual contracts awarded to economic operators within the framework.

It is theoretically possible to address some of these anomalies with further processing, provided that a consistent pattern can be clearly identified. However, in practice, this is likely to be impractical due to lack of consistency in how framework contracts are recorded, the large number of records involved, and potential gaps and errors in the data recorded in other fields, for example any of the three fields identifying whether or not the record relates to a framework contract – and two of these appear to be already ‘estimated’ on the basis of other information during processing by DG GROW.

2.2.3 Errors and doubtfully high contract values

The previous study noted that there were 782 entries in the original dataset where the ‘Amount of Award’ exceeds EUR 1,000,000,000.23 The sum of these awards is EUR 200,003,034,035,733,000,000, and they had a median value of EUR 2,755,428,193. The maximum value is EUR 100,000,000,000,000,000,000 – the year of the notice is 2017, the contract awarding entity is given as ‘Министерство на транспорта’ (Ministry of Transport) in Sofia, while the country is given as ‘UK’. This example combines an impossibly high contract value with a simple factual error - a conflict between the identity of the contracting authority (the Bulgarian Ministry of Transport) and the indicated country (United Kingdom). It is not known if these errors originated during data entry or subsequent processing. Nevertheless, the fact that such errors exist suggests that there may be many more amongst the 4,630,484 records relating to contract awards during the period 2009 to 2017.

23 Blomeyer & Sanz (12/09/2018), EU-funded large-scale infrastructure: deficient project preparation and procurement processes?, p52, https://bit.ly/2CQ3orK



16

Indeed, DG GROW notes that ‘The information on monetary values published in TED notices often suffers from missing values and reporting mistakes (particularly in the form of nonsensically large values)’.24

Further analysis of this issue has been undertaken in the context of this briefing paper at the level of contract award notices (CAN) (as opposed to individual contract awards/ records). This identified 501 CANs with values above EUR 1 billion. Three of these CANs have even more highly improbable figures (see Table 3 below) with an aggregate value of EUR 51,640,00,000,000,000,000,000.

Table 3: CANs with most impossibly high award amount

ID_NOTICE_CAN VALUE_EURO_FIN_1 YEAR CAE_NAME MS B_FRA_AGREEMENT

2011228511 4,964,000,000,000,000,000,000 2011 EC, European Administrative School

BE Y

201230105 100,000,000,000,000,000,000 2012 Ville d'Ivry-sur-Seine FR N

2013239635 100,000,000,000,000,000,000 2013 Ville d'Ivry-sur-Seine FR Y

Source: author, based on analysis of downloaded TED data for 2009-2017

2.2.4 Limited publication of public procurement on TED

ANTICORRP’s ‘Government Favouritism in Europe’ states that ‘only a minority of awards are published in TED. Few countries publish sufficient procurement data online to allow comparison across EU-28’.25 This is emphasised by DG GROW reports on procurement indicators, which show significant variations in publication rates between Member States. According to three different indicators, Germany had consistently the lowest publication rate of all Member States from 2012 to 2015, the most recent year for which indicators are available.26 For example, Germany’s mean publication rate over these four years in terms of percentage of total general government expenditures on works, goods, and services (excluding utilities and defence) was 5.7%. According to the same measure, the Netherlands also had a low publication rate of 8.8%. On the other hand, Latvia and Bulgaria had publication rates of 59.1% and 47.1% respectively. It is understood that DG GROW is currently studying this issue.

24 European Commission DG GROW (19/12/2016), Public Procurement Indicators 2015, p2, https://ec.europa.eu/docsroom/documents/20679 25 Alina Mungiu-Pippidi and Roberto Marti�nez B. Kukutschka (28/10/2015), Government favoritism in Europe, The ANTICORRP project: Anticorruption report, volume 3, chapter 1, p9, http://anticorrp.eu/publications/volume-3-government-favouritism-in-europe/ 26 European Commission DG GROW (19/12/2016), Public Procurement Indicators 2015, p14-16, https://ec.europa.eu/docsroom/documents/20679

https://ec.europa.eu/docsroom/documents/20679

http://anticorrp.eu/publications/volume-3-government-favouritism-in-europe/




17

2.3 ORIGINS OF TED GAPS AND ERRORS

Why/ how do these issues arise?

2.3.1 Limited number of mandated fields

The older standards covering the 2009-2017 TED data were less demanding in terms of the fields that had to be filled in (mandatory fields). This has meant that, even where contracting authorities have filled in all mandatory fields, important data could still be omitted, such as contract award amounts and the number of bids. The POEU notes that, in practice, many fields were left blank.

2.3.2 National procurement platforms

In some Member States, there are several platforms publishing electronic tenders, and this contributes to confusion, complexity and inefficiency. This is the case for Belgium, France, Germany, Italy, Malta, Spain, and the UK.27 Sometimes, this is due to the administrative federal organisation of the country. In three countries, Austria, Luxembourg and Sweden, there is no public portal at all, and ‘various private companies step in to fill the informational void, however in return for levying an access fee while effectively preventing governments from controlling data quality and content’.

2.3.3 Complexity of systems

The templates and the format of the information provided for electronic public procurement are available in many different formats in every Member State as well as in TED. Mendez and Fazekas state that ‘several countries as well as the EU-wide TED portal use a large number of different templates – more than 15 or even 20 - which adds an unnecessary layer of complexity to understanding and reusing the data’.28 The current Commission implementing regulation includes 25 standard forms.29 In terms of accessibility, it seems that TED, together with Belgium, Poland and the United Kingdom, are organised in a way that is fully machine readable, which facilitates better understanding and reuse of data. In general terms, the newer Member States offer more accessible and better quality information (see Figure 3 below).30

Figure 2 below shows the number of types of procurement announcements in use in various countries inside and outside the EU.

27 Mara Mendes Open Knowledge Foundation Germany, Mihály Fazekas University of Cambridge (31/03/2017), Recommendations for the Implementation of Open Public Procurement Data An Implementer‘s Guide, p3, http://digiwhist.eu/publications/digiwhist-recommendations-for-the-implementation-of-open-public-procurement-data-an-implementers-guide/ 28 Mara Mendes Open Knowledge Foundation Germany, Mihály Fazekas University of Cambridge (31/03/2017), Recommendations for the Implementation of Open Public Procurement Data An Implementer‘s Guide, p3, http://digiwhist.eu/publications/digiwhist-recommendations-for-the-implementation-of-open-public-procurement-data-an-implementers-guide/ 29 European Commission (11 November 2015), Commission Implementing Regulation (EU) 2015/1986 of 11 November 2015 establishing standard forms for the publication of notices in the field of public procurement and repealing Implementing Regulation (EU) No 842/2011 (Text with EEA relevance), https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32015R1986 30 Mara Mendes Open Knowledge Foundation Germany, Mihály Fazekas University of Cambridge (31/03/2017), Recommendations for the Implementation of Open Public Procurement Data An Implementer‘s Guide, p11, http://digiwhist.eu/publications/digiwhist-recommendations-for-the-implementation-of-open-public-procurement-data-an-implementers-guide/





https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32015R1986




18

Figure 2: Number of national announcement types in use in Member States

Source: Mendes and Fazekas31 ‘Number of national announcement types used for call for tenders, contract awards, and contract modifications on the main national portal is optimal only in about half of European countries’

2.3.4 Double entry and lack of compatibility between national systems and TED

Although the need for double entry is being eliminated with the ongoing reform, the downloaded TED data for 2009-2017 was still subject to the old regime, which required public procurement information to be entered twice - once in Member State public procurement systems, and then into any one of 25 different standard forms for transmission to the POEU and publication in TED. The transfer of data between systems in this way significantly increases the risk of gaps and errors in TED due to (a) differences in the content and organisation of national systems, and (b) the process of transferring the data. Moreover, the fact that data has to be entered twice is an additional and unnecessary administrative burden and it would be unsurprising if contracting authorities limited the resources allocated to the publication of TED data. Indeed, the 2011 regulatory impact assessment acknowledges the high time and cost burdens for contracting authorities of completing standard forms,32 and it states that ‘A new generation of standard forms would be expected to be (to the extent possible) automated so that a vast majority of information introduced in TED would be reusable by the IT

31 Mara Mendes Open Knowledge Foundation Germany, Mihály Fazekas University of Cambridge (31/03/2017), Recommendations for the Implementation of Open Public Procurement Data An Implementer‘s Guide, p11, http://digiwhist.eu/publications/digiwhist-recommendations-for-the-implementation-of-open-public-procurement-data-an-implementers-guide/ 32 European Commission (20/12/2011), Impact Assessment Accompanying the Document Proposal for a Directive of the European Parliament and of the Council On Public Procurement and the Proposal for a Directive of the European Parliament and of the Council On Procurement by Entities Operating in the Water, Energy, Transport and Postal Sectors, p30, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52011SC1585






19

systems (e.g. information introduced in PIN would be re-used in a CN and a CAN). Such a new generation of standard forms would also be principally based on tick-boxes (to reduce the number of text fields), so that the time spent by the different parties on completing these procedural obligations could be limited to an absolute minimum’.33

2.3.5 Lack of compliance with and enforcement of EU legislation

As noted above (2.2.4), it appears that Member States generally publish limited public procurement data on TED, and it was reported to the study team that one significant national entity 34 has not published procurement information on TED for a number of years.35 Assuming that this is correct, the issue here seems to be lack of compliance with EU legislation and inadequate enforcement by national authorities and/ or the EC. In the context of non-compliance, it is interesting to note that an EC press release of 24 January 2019 states:36

The Commission decided today to send letters of formal notice to 15 Member States (Bulgaria, Croatia, Cyprus, Czechia, Denmark, Finland, Germany, Hungary, Italy, Malta, the Netherlands, Poland, Romania, Sweden, and the United Kingdom) regarding the conformity of their national legislation with EU rules on public procurement and concessions. The new rules (Directive 2014/24/EU, Directive 2014/25/EU and Directive 2014/23/EU) had to be transposed by Member States into national law by 18 April 2016. Today's letters are the result of a compliance check conducted by the Commission to see whether the transposed national rules comply with the EU Directives.

2.3.6 Structure of the TED platform

TED is a publication platform that has evolved over many years. The POEU notes that while almost all notices are now submitted to the POEU electronically in XML format, approximately 1% are still submitted on paper. Although 99% of forms are submitted electronically, the system can be described as electronic implementation of paper-based forms. Among other things, this has meant that the system was not designed from the ground up as a tool for validating and analysing large volumes of data. While it is no longer paper-based, it is only now moving from a ‘paper logic’ to a fully ‘IT logic’ – a process that will take several more years to complete.

2.3.7 Limited mandate of the Publications Office of the European Union

The POEU is responsible for the TED platform, but its role is limited to that of publisher. Some 2,300 notices are published daily. It is therefore impossible for the POEU to check them all. It undertakes quality control of a random sample of 2% of published documents. This quality control covers only the mandatory fields. While it is theoretically possible to automate checking on a larger scale, this would

33 European Commission (20/12/2011), Impact Assessment Accompanying the Document Proposal for a Directive of the European Parliament and of the Council On Public Procurement and the Proposal for a Directive of the European Parliament and of the Council On Procurement by Entities Operating in the Water, Energy, Transport and Postal Sectors, p73, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52011SC1585 34 The terms ‘entity’ and ‘entities’ refer to organisations and institutions that undertake public procurement. This includes central government departments, state agencies, local governments and any other body that undertakes procurement with public funds. European Commission documentation refers to these as ‘contracting authorities or entities’. See for example European Commission DG GROW (23/02/2018) ‘TED CSV open data NOTES & CODEBOOK Version 3.1 (2018-02-23), p2, http://data.europa.eu/euodp/repository/ec/dg-grow/mapps/TED(csv)_data_information.pdf 35 It is understood that DG GROW is currently studying this issue. 36 European Commission (24 January 2019), January infringements package: key decisions, http://europa.eu/rapid/press-release_MEMO-19-462_EN.htm




http://europa.eu/rapid/press-release_MEMO-19-462_EN.htm



20

not work in practice due to the large number of exceptions, which would require the development of impossibly complicated algorithms. The POEU informs contracting authorities about gaps and anomalies it finds but ultimately it must publish according to the requirements of contracting authorities, even where the POEU has significant doubts about the accuracy of the information submitted to it – it has no power to reject notices, even where mandated data is missing or anomalous.

2.4 PATTERNS OF TED GAPS AND ERRORS

Do the issues arise mainly in specific Member States or institutions?

Figure 3 below shows that countries in Northern and Western Europe tend to provide the least complete TED data (when considering only mandatory fields), whereas countries in Central and Eastern Europe tend to provide the most complete TED data. The reasons for this split are unclear. Possible explanations include out of date IT and/ or management systems, resource constraints, institutional culture, other cultural factors.

Figure 3: Extent of missing information in TED data in 2009-2015

Source: Mendes and Fazekas37 ‘Extent of missing information[:] Data quality is low throughout Europe with 15% of mandatory fields empty in the EU-wide TED data in 2009-2015’

37 Mara Mendes Open Knowledge Foundation Germany, Mihály Fazekas University of Cambridge (31/03/2017), Recommendations for the Implementation of Open Public Procurement Data An Implementer‘s Guide, p21, http://digiwhist.eu/publications/digiwhist-recommendations-for-the-implementation-of-open-public-procurement-data-an-implementers-guide/




21

2.5 AREAS FOR ENHANCEMENT OF TED

How might the reliability of TED data be enhanced?

The Commission is currently in the process of implementing a long-term, major public procurement reform intended, among other things, to improve the reliability of TED data, compared with the data for 2009-2017 that was used for the single-bid analysis in the previous study. This briefing paper therefore does not provide detailed recommendations on this issue but instead briefly reflects on future challenges and how these might be addressed (see 3.2 below)


22

3 RECENT DEVELOPMENTS AND FUTURE CHALLENGES

• The 2009-2017 TED dataset is covered by an old standard. The Commission is in the process of implementing a major reform of public procurement rules and processes that is expected to improve the quality and reliability of TED data.

• The new standard is based on a series of directives of 2014. The Commission is planning to publish a new implementing regulation in May-June 2019 that will fully operationalise the 2014 directives by 2023, the deadline for Member states to implement its requirements. Thus, the entire reform process will have taken approximately 12 years from when the impact assessment was published in 2011.

• Key improvements include a move from ‘paper logic’ to a full ‘IT logic’ and the elimination of the need for contracting authorities to enter procurement data twice – once in Member State public procurement systems, and then again in TED.

• Nevertheless, the new TED standard still allows much data to be marked as ‘not to be published’, with the result that much data may still missing. Moreover, some indicators suggest that much public procurement is not published in TED at all and it is unclear to what extent ongoing reforms will address this unless they are accompanied by political pressure and effective enforcement actions against non-compliant contracting authorities.

3.1 RECENT DEVELOPMENTS

The 2009-2017 TED dataset, which was downloaded from the EU Open Data Portal,38 is covered by directives of 2004,39 40 which have since been superseded by directives of 200941 and 2014.42 43 44 This data is also covered by three successive Commission Regulations on standard forms to be used to

38 Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs, Tenders Electronic Daily (TED) (csv subset) – public procurement notices, https://data.europa.eu/euodp/data/dataset/ted-csv 39 (31 March 2004), Directive 2004/18/EC of the European Parliament and of the Council of 31 March 2004 on the coordination of procedures for the award of public works contracts, public supply contracts and public service contracts. Superseded on 18 April 2016, https://eur-lex.europa.eu/legal-content/en/ALL/?uri=celex%3A32004L0018 40 (31 March 2004), Directive 2004/17/EC of the European Parliament and of the Council of 31 March 2004 coordinating the procurement procedures of entities operating in the water, energy, transport and postal services sectors. Superseded on 18 April 2016, https://eur-lex.europa.eu/legal-content/en/ALL/?uri=celex%3A32004L0017 41 (13 July 2009), Directive 2009/81/EC of the European Parliament and of the Council of 13 July 2009 on the coordination of procedures for the award of certain works contracts, supply contracts and service contracts by contracting authorities or entities in the fields of defence and security, and amending Directives 2004/17/EC and 2004/18/EC (Text with EEA relevance), https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32009L0081 42 (26 February 2014), Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement and repealing Directive 2004/18/EC Text with EEA relevance, https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32014L0024 43 (26 February 2014), Directive 2014/23/EU of the European Parliament and of the Council of 26 February 2014 on the award of concession contracts Text with EEA relevance, https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32014L0023 44 (26 February 2014), Directive 2014/25/EU of the European Parliament and of the Council of 26 February 2014 on procurement by entities operating in the water, energy, transport and postal services sectors and repealing Directive 2004/17/EC Text with EEA relevance, https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32014L0025

https://data.europa.eu/euodp/data/dataset/ted-csv

https://eur-lex.europa.eu/legal-content/en/ALL/?uri=celex%3A32004L0018









23

submit data to TED.45 46 47 The 2005 Regulation provided 13 standard forms, and this increased to 19 in the 2011 Regulation, and 25 in the 2015 Regulation. A new implementing regulation (the ‘eForms regulation’) is currently being prepared by DG GROW and is expected to be published in May-June 2019.

The new standard forms under the 2014 directives entered into force on 03 December 2015. The 2014 directives required contracting authorities to ‘implement exclusive electronic public procurement via dedicated e-procurement platforms’48 by 18 October 2018.49 Most Member States started to send data according to the new standard by the end of 2017. It is ultimately intended that national procurement systems and TED will be electronically integrated, thus eliminating duplication of data entry and reducing gaps and errors. ‘The ’once-only’ principle aims at creating an eprocurement ecosystem, which facilitates the seamless interaction of businesses with public buyers.…Beyond reducing administrative burden, this simplification will also result in higher traceability and transparency of procurement procedures.’50 The implementation of this integration will be operationalised by the ‘eForms regulation’ to be published in May-June 2019.51

Changes in the standard introduced with the 2014 directives means that TED data from 2018 onwards should already be significantly improved (e.g. among other things, the POEU can now reject notices where mandatory fields have not been completed, although this is not so clear in Directive 2014/24/EU), 52 and further improvements are expected with the introduction of the forthcoming ‘eForms regulation’:

• There will be a single point of data entry, with data entered into Member State systems being automatically transferred to TED;

• The number of mandatory fields will be increased; • The number of standard forms will be reduced from 25 to six; • Overall, the system will be easier to use;

45 European Commission (07 September 2005), Commission Regulation (EC) No 1564/2005 of 7 September 2005 establishing standard forms for the publication of notices in the framework of public procurement procedures pursuant to Directives 2004/17/EC and 2004/18/EC of the European Parliament and of the Council (Text with EEA relevance), https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32005R1564 46 European Commission (19 August 2011) Commission Implementing Regulation (EU) No 842/2011 of 19 August 2011 establishing standard forms for the publication of notices in the field of public procurement and repealing Regulation (EC) No 1564/2005 Text with EEA relevance, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32011R0842 47 European Commission (11 November 2015), Commission Implementing Regulation (EU) 2015/1986 of 11 November 2015 establishing standard forms for the publication of notices in the field of public procurement and repealing Implementing Regulation (EU) No 842/2011 (Text with EEA relevance), https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32015R1986 48 In the present context, a ‘platform’ can be described as a ‘basic foundation for the development and support of hardware and software’, Stanley Goodner (13 December 2018), What Is a Platform?, https://www.lifewire.com/what-is-a-platform-4155653 49 European Commission (February 2018) Public Procurement Guidance for Practitioners, p14, https://ec.europa.eu/regional_policy/sources/docgener/guides/public_procurement/2018/guidance_public_procurement_2018_en.pdf 50 European Commission (24 April 2016), EU Public Procurement reform: Less bureaucracy, higher efficiency, p4, http://ec.europa.eu/docsroom/documents?documentsIds=16412_native&iframe=true&locale=en&languagesExpanded=true&autoHeight=true 51 European Commission DG GROW (11/02/2019), Commission Implementing Regulation establishing standard forms for the publication of notices in the field of public procurement ("eForms") – public feedback explanatory note, https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2019-797630_en 52 Article 85 of Directive 2014/24/EU states that ‘Where the quality and completeness of the data referred to in the first subparagraph of this paragraph is not compliant with the obligations stipulated in Article 48(1), Article 49, Article 50(1), Article 75(2) and Article 79(3), the Commission shall request complementary information from the Member State concerned. Within a reasonable time, the Member State concerned shall supply the missing statistical information requested by the Commission.’ Article 51 enumerates the role and obligations of the POEU, but does not appear to indicate the possibility for notices to be rejected.





https://www.lifewire.com/what-is-a-platform-4155653

https://ec.europa.eu/regional_policy/sources/docgener/guides/public_procurement/2018/guidance_public_procurement_2018_en.pdf


http://ec.europa.eu/docsroom/documents?documentsIds=16412_native&iframe=true&locale=en&languagesExpanded=true&autoHeight=true




24

• It will enable more automatic validation of data submitted by contracting authorities, with the possibility of rejecting publication where notices do not comply with legislation;

• There will be greater clarity and consistency in the handling of framework contracts, with one field for the maximum value of the entire framework, and one field for the actual value awarded in each case. However, the new standard does not require publication of information at both stages i.e. when contractors are selected to participate in the framework and when they are actually awarded contracts within the framework.

The following key dates are envisaged for the rollout of eForms:53

• June 2020: Publication of the technical standard for electronic exchange of notices by the Publication Office of the EU. Start of implementation in Member States;

• June 2022: Optional use in Member States; • June 2023: Mandatory use in Member States.

Overall, the package of reforms should significantly reduce gaps and errors in TED and it should eliminate differences between the information included in TED and the information in Member State public procurement systems. The Commission expects that by 2023, when eForms are implemented by Member States, TED will be a reliable source of data for analysis of public procurement in the EU.

3.2 FUTURE CHALLENGES

Despite these improvements, the utility of TED as an analytical tool is likely to remain constrained for various reasons.

3.2.1 Data ‘not to be published’

For example, DG GROW notes that, while the new implementing regulation, which is currently under development, makes most fields mandatory by default, contracting authorities can nevertheless opt out of providing the data for many fields by marking them as ‘not to be published’. For example, while values will generally be mandatory under the forthcoming regulation, they can be left out. Article 50(4) of Directive 2014/24/EU states that ‘Certain information on the contract award or the conclusion of the framework agreement may be withheld from publication where its release would impede law enforcement or otherwise be contrary to the public interest, would harm the legitimate commercial interests of a particular economic operator, public or private, or might prejudice fair competition between economic operators.’ 54 The latest version (at the time of writing) of the eForms consultation codelist spreadsheet indicates the following reasons for not publishing data:55

• Law enforcement; • Other public interest; • Commercial interests of an economic operator; • Fair competition.

53 European Commission DG GROW (11/02/2019), Commission Implementing Regulation establishing standard forms for the publication of notices in the field of public procurement ("eForms") – public feedback explanatory note, p1, https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2019-797630_en 54 26 February 2014), Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement and repealing Directive 2004/18/EC Text with EEA relevance, https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32014L0024 55 European Commission (accessed 14/02/2019), 5_eForms_feedback_codelists_draft.xlsx, https://github.com/eForms/eForms




https://github.com/eForms/eForms


25

This document also lists approximately 60 fields which may be marked as ‘not to be published’, including ‘Winning Tender Value (BT-720)’.

As noted above (see 2.4), there is significant divergence between Member States in how much data they currently include in TED procurement forms. Member States in Eastern Europe tend to provide the most complete data, while Member States in Western Europe tend to provide the least complete data.56 Certain types of entity are also more likely to leave out important information.

ANTICORRP’s ‘Government Favouritism in Europe’ recommends the introduction of fines for countries that provide incorrect, late or incorrect submission of data to TED. 57 In some countries, such as Slovakia, if the data are not properly submitted, the tenders can be suspended until the issue is solved. Some other countries send automatic questions asking for clarifications to the competent institutions. The introduction of centralised control mechanisms by the Member States could also contribute to the fact that all the necessary data are correctly published.

3.2.2 Non-publication of public procurement in TED

Besides the quality of data entered in TED, another major concern is lack of compliance with EU legislation. Some indicators suggest that a limited percentage of public procurement is published in TED and there are significant variations between Member States. It is unclear to what extent ongoing EU public procurement reforms will address this unless they are accompanied by political pressure and effective enforcement.

3.2.3 Complexity of requirements

The ongoing reform is intended to ‘significantly improve the usefulness of data on European procurement for all users: companies, governments and citizens’ and simplify the legal framework and forms to be used.58 Nevertheless, this remains a very complicated area and we suggest that understanding of the requirements and the available data could be significantly enhanced with clearer explanation and guidelines.

56 Research undertaken by the Open Contracting Partnership identified 10 common arguments for not publishing contracting information but they ‘found surprisingly little evidence that backed up the harm proposed by the arguments and quite a lot of evidence that does not support them’. It also suggests that ‘Concern around the confidentiality of information in the contracts themselves is arguably one of the greatest barriers to more openness.’ Open Contracting Partnership (2018), Mythbusting Confidentiality in Public Contracting, https://www.open-contracting.org/wp-content/uploads/2018/07/OCP18-Mythbusting.pdf 57 Alina Mungiu-Pippidi and Roberto Marti�nez B. Kukutschka (28/10/2015), Government favoritism in Europe, The ANTICORRP project: Anticorruption report, volume 3, chapter 1, p18, http://anticorrp.eu/publications/volume-3-government-favouritism-in-europe/ 58 European Commission DG GROW (11/02/2019), Commission Implementing Regulation establishing standard forms for the publication of notices in the field of public procurement ("eForms") – public feedback explanatory note, https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2019-797630_en

https://www.open-contracting.org/wp-content/uploads/2018/07/OCP18-Mythbusting.pdf





26

4 CONCLUSIONS

The accessibility of TED data has significantly improved in recent years. However, the quality and reliability of TED data for 2009-2017 is undermined by large amounts of missing data, inconsistency in the way that data is entered, and errors in the data entered, including impossibly high figures for the value of some contract awards. Gaps and errors are attributed to: the limited number of mandated (compulsory) fields; the fragmentation and complexity of procurement systems in Member States; lack of compatibility between TED and Member State systems and the need to enter data twice – once in Member State public procurement systems and then in TED; lack of compliance with and enforcement of EU legislation; and the limited mandate of the Publications Office of the European Union. Underlying this all, is the structure of the TED platform, which has been described as an electronic implementation of a paper-based system, and which is for publication, not data analysis. Gaps appear to be more prevalent in the data submitted by Member States in Western and Northern Europe, while those in Central and Eastern Europe appear to provide more complete data.

Perhaps more significantly, the value of TED as a tool to analyse public procurement in the EU is constrained by the fact that there is apparently great variation in the extent to which Member States publish public procurement information on TED.

The 2009-2017 TED dataset is covered by an old standard. The Commission is in the process of implementing a major reform of public procurement rules and processes that is expected to improve the quality and reliability of TED data. The full impact of these changes will take several years to materialise.

Nevertheless, the new TED standard still allows much data to be marked as ‘not to be published’, with the result that much data may still missing. It is unclear to what extent ongoing reforms will address the issue of non-publication in TED, unless they are accompanied by political pressure and effective enforcement actions against non-compliant contracting authorities/ entities.


27

REFERENCES

• Blomeyer & Sanz (12/09/2018), EU-funded large-scale infrastructure: deficient project preparation and procurement processes? https://bit.ly/2CQ3orK

• Directive 2004/17/EC of the European Parliament and of the Council of 31 March 2004 coordinating the procurement procedures of entities operating in the water, energy, transport and postal services sectors https://eur-lex.europa.eu/legal-content/en/ALL/?uri=celex%3A32004L0017

• Directive 2004/18/EC of the European Parliament and of the Council of 31 March 2004 on the coordination of procedures for the award of public works contracts, public supply contracts and public service contracts https://eur-lex.europa.eu/legal-content/en/ALL/?uri=celex%3A32004L0018

• Directive 2009/81/EC of the European Parliament and of the Council of 13 July 2009 on the coordination of procedures for the award of certain works contracts, supply contracts and service contracts by contracting authorities or entities in the fields of defence and security, and amending Directives 2004/17/EC and 2004/18/EC (Text with EEA relevance) https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32009L0081

• Directive 2014/23/EU of the European Parliament and of the Council of 26 February 2014 on the award of concession contracts Text with EEA relevance https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32014L0023

• Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement and repealing Directive 2004/18/EC Text with EEA relevance https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32014L0024

• Directive 2014/25/EU of the European Parliament and of the Council of 26 February 2014 on procurement by entities operating in the water, energy, transport and postal services sectors and repealing Directive 2004/17/EC Text with EEA relevance https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32014L0025

• European Commission DG GROW, Tenders Electronic Daily (TED) (csv subset) – public procurement notices https://data.europa.eu/euodp/data/dataset/ted-csv

• European Commission (07 September 2005), Commission Regulation (EC) No 1564/2005 of 7 September 2005 establishing standard forms for the publication of notices in the framework of public procurement procedures pursuant to Directives 2004/17/EC and 2004/18/EC of the European Parliament and of the Council (Text with EEA relevance) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32005R1564



https://eur-lex.europa.eu/legal-content/en/ALL/?ur





https://data.europa.eu/euodp/data/dataset/ted-csv



28

• European Commission (11 November 2015), Commission Implementing Regulation (EU) 2015/1986 of 11 November 2015 establishing standard forms for the publication of notices in the field of public procurement and repealing Implementing Regulation (EU) No 842/2011 (Text with EEA relevance) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32015R1986

• European Commission (19 August 2011) Commission Implementing Regulation (EU) No 842/2011 of 19 August 2011 establishing standard forms for the publication of notices in the field of public procurement and repealing Regulation (EC) No 1564/2005 Text with EEA relevance https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32011R0842

• European Commission (20 December 2011), Impact Assessment Accompanying the Document Proposal for a Directive of the European Parliament and of the Council On Public Procurement and the Proposal for a Directive of the European Parliament and of the Council On Procurement by Entities Operating in the Water, Energy, Transport and Postal Sectors https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52011SC1585

• European Commission/ PWC (January 2016), Stock-taking of administrative capacity, systems and practices across the EU to ensure the compliance and quality of public procurement involving European Structural and Investment (ESI) Funds https://publications.europa.eu/en/publication-detail/-/publication/d1082259-0202-11e6-b713-01aa75ed71a1

• European Commission (24 April 2016), EU Public Procurement reform: Less bureaucracy, higher efficiency http://ec.europa.eu/docsroom/documents?documentsIds=16412_native&iframe=true&locale=en&languagesExpanded=true&autoHeight=true

• European Commission (24 January 2019), January infringements package: key decisions http://europa.eu/rapid/press-release_MEMO-19-462_EN.htm

• European Commission (accessed 14/02/2019), 5_eForms_feedback_codelists_draft.xlsx, https://github.com/eForms/eForms

• European Commission (Benedikt Herz and Xosé-Luís Varela-Irimia) (June 2017), Border Effects in European Public Procurement http://ec.europa.eu/docsroom/documents/26201

• European Commission (February 2018) Public Procurement Guidance for Practitioners https://ec.europa.eu/regional_policy/sources/docgener/guides/public_procurement/2018/guidance_public_procurement_2018_en.pdf

• European Commission DG GROW (23/02/2018) ‘TED CSV open data NOTES & CODEBOOK Version 3.1 (2018-02-23)’ http://data.europa.eu/euodp/repository/ec/dg-grow/mapps/TED(csv)_data_information.pdf Note that the document does not explicitly name DG GROW as the author, but this is inferred from the contact email address.














29

• European Commission DG GROW (11/02/2019), Commission Implementing Regulation establishing standard forms for the publication of notices in the field of public procurement ("eForms") – public feedback explanatory note https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2019-797630_en

• European Commission DG GROW (19/12/2016), Public Procurement Indicators 2015 https://ec.europa.eu/docsroom/documents/20679

• Mihály Fazekas and Bence Tóth (March 2017), Infrastructure for Whom? Corruption Risks in Infrastructure Provision Across Europe https://www.researchgate.net/publication/316338624_Infrastructure_for_Whom_Corruption_Risks_in_Infrastructure_Provision_Across_Europe

• Stanley Goodner (13 December 2018), What Is a Platform? https://www.lifewire.com/what-is-a-platform-4155653

• Mara Mendes Open Knowledge Foundation Germany, Mihály Fazekas University of Cambridge (31/03/2017), Recommendations for the Implementation of Open Public Procurement Data An Implementer‘s Guide http://digiwhist.eu/publications/digiwhist-recommendations-for-the-implementation-of-open-public-procurement-data-an-implementers-guide/

• Alina Mungiu-Pippidi and Roberto Martínez B. Kukutschka (28/10/2015), Government favoritism in Europe, The ANTICORRP project: Anticorruption report, volume 3, chapter 1 http://anticorrp.eu/publications/volume-3-government-favouritism-in-europe/

• Open Contracting Partnership (2018), Mythbusting Confidentiality in Public Contracting https://www.open-contracting.org/wp-content/uploads/2018/07/OCP18-Mythbusting.pdf

• Publications Office of the European Union (13/02/2019), TED tenders electronic daily Supplement to the Official Journal of the EU https://ted.europa.eu/TED/main/HomePage.do




https://www.lifewire.com/what-is-a-platform-4155653






The accessibility of TED public procurement data has improved in recent years. However, downloadable TED data up to 2017 is complex and includes numerous gaps and errors. Some indicators suggest that much public procurement in the European Union is not published on TED at all. The Commission is in the process of implementing a major reform of public procurement rules and processes which is expected to improve the quality of TED data. However, the new rules still permit much data to be marked as ‘not for publication’.

DISCLAIMER This document is prepared for, and addressed to, the Members and staff of the European Parliament as background material to assist them in their parliamentary work. The content of the document is the sole responsibility of its author(s) and any opinions expressed herein should not be taken to represent an official position of the Parliament.

Print ISBN 978-92-846-4645-6 | doi: 10.2861/49499 | QA-01-19-226-EN-C PDF ISBN 978-92-846-4644-9 | doi: 10.2861/06472 | QA-01-19-226-EN-N

gaps and errors in the ted database - european parliament€¦ · gaps and errors in the ted...

Documents