Evaluating stakeholder engagement: Practices in OECD countries Laura Seiffert, Junior Policy Analyst, Regulatory Policy Division, OECD Breakout session 1 6th Expert Meeting on Measuring Regulatory Performance Evaluating Stakeholder Engagement in Regulatory Policy

- Stakeholder engagement is essential to enhance regulatory quality, compliance with regulation, and trust in government - Evaluation helps to: - Highlight successes and areas for improvement - Guide policy makers in their efforts to improve stakeholder engagement Why evaluate?

OECD Framework for Regulatory Policy Evaluation applied to stakeholder engagement Have the good practices in been implemented? E.g. % of consultations that comply with formal requirements, such as minimum periods Are requirements for good practices in stakeholder engagement in place? E.g. requirements to consult on draft regulation, requirement to make relevant material of regulatory dossier available to inform consultation (impact assessment, explanation of reasons to regulate ect.) What resources are committed to stakeholder engagement? E.g. budget, staffing, expertise Have good practices helped to improve quality regulation? E.g. % of those involved in the regulatory process that think stakeholder engagement has improved the quality of regulation as opposed to being a tick-the box-exercise; comparison of cost/benefit of initial regulatory proposals with those of regulations that were actually passed Have strategic objectives for stakeholder engagement been achieved? Improved compliance rate Reduced enforcement costs Trust in government Inclusive growth Economic: Net benefits (possibly in comparison with counterfactual and alternatives) Reduced compliance costs and burdens Improved perception of regulatory quality in general Transparency and accessibility Objective of regulation achieved easy hard Regulatory system Regulatory impacts Other factors that influence outcomeFeedback loop Input Process Output Intermediate outcome Outcome Set strategic objectives for stakeholder engagement Design Implementation Strategic outcome I II III IV V

Public consultation is seldomly evaluated compared to other regulatory management tools Current country practices 8 6 20 14 21 20 0 5 10 15 20 25 Reports Indicators Numberofcountries Number of countries publishing performance reports or indicators on consultation, RIA or administrative burden reduction programmes Consultation on draft regulation Regulatory Impact Analysis (RIA) on draft regulation Administartive burden reduction programmes Source: OECD survey on Regulatory Policy Evaluation (2012-13)

Current country practices Countries publishing performance reports: Czech Republic, European Union, Japan, Mexico, the Netherlands, Slovenia, Switzerland, Turkey Countries publishing performance indicators: Estonia, European Union, Japan, Mexico, Slovenia, Turkey

Who publishes reports and indicators? Independent or central government agency/department Research institutes For what purpose are indicators used? To adjust the design and implementation of stakeholder engagement processes To monitor quality and compliance with formal requirements To communicate with internal or external stakeholders Current country practices

Output indicators Percentage of consultations that comply with minimum periods (e.g. EU, Japan, Slovenia) Number of different stakeholders participating in consultation (e.g. Mexico, Estonia) Number of languages in which consultation documents are published (EU) Total numbers of consultations (e.g. EU, Switzerland, Turkey) Type of performance indicators used

OECD Practitioner s Guide to Perception Surveys (2012)

European Union: Review of the Commission Consultation Policy (2012) Switzerland: Evaluation of practices with regard to hearing and consultation procedures (2011) in French Examples of performance reports

There is consensus on the importance of effective stakeholder engagement Few countries evaluate how consultation is functioning in practice Systematic evaluation is within reach for all OECD countries OECD work provides guidance and many practical examples on how to evaluate and improve stakeholder engagement Key messages

1. What are key good practices in evaluating government systems to engage with stakeholders in the regulatory policy cycle? 2. How can countries assess : The inclusiveness of consultation? The transparency of the consultation process? Whether stakeholders views are adequately considered and integrated into the decision-making process? 3. What are key challenges in assessing the engagement with stakeholders and how can they be overcome? Questions for discussion

OECD (2014), OECD Framework for Regulatory Policy Evaluation Radaelli, Claudio and Oliver Fritsch (2012), Measuring Regulatory Performance - Evaluating Regulatory Management Tools and Programmes, Expert Paper, No. 2, Paris. OECD (2012), Measuring Regulatory Performance: A Practitioners Guide to Perception Surveys OECD (2005), Evaluating Public Participation in Policy Making Coglianese, Cary (2003), Is satisfaction success? Evaluating public participation in regulatory policy-making Relevant material

Christiane Arndt, Programme Co-ordinator Measuring Regulatory Performance, E-mail: christiane.arndt@oecd.org, Tel. +33 (0) 1 45 24 76 56 Daniel Trnka, Policy Analyst, E-mail: daniel.trnka@oecd.org, Tel.: +33 (0) 1 45 24 18 68 Antonia Custance Baker, Economist/Policy Analyst, E- mail: antonia.custancebaker@oecd.org, Tel.: +33 (0) 1 45 24 93 22 Laura Seiffert, Junior Policy Analyst, E-Mail: laura.seiffert@oecd.org, Tel.: +33 (0) 1 45 24 13 37 Contacts