what makes indicators successful? lessons from practitioners
DESCRIPTION
BRAINPOoL (Bringing alternative indicators into policy) is an EU-funded project aimed at identifying and overcoming the barriers to ‘Beyond GDP’ indicators being used in policyTRANSCRIPT
What makes indicators successful? Lessons from practitioners
Funded by:
The Project
BRAINPOoL (Bringing alternative indicators into policy) is an EU-funded project aimed at identifying and overcoming the barriers to ‘Beyond GDP’ indicators being used in policy.
During the project we are carrying out research and interviews, conducting workshops and knowledge-brokerage seminars and carrying out various action research case studies to explore ways to improve uptake of Beyond GDP indicators.
The Project (work packagestructure)
This presentation
Understanding the supply of Beyond GDP indicators
• Work Package 1 outcomes:– Catalogue of over 100 Beyond GDP indicator initiatives– Understanding of intentions of indicator
producers/promoters– Documenting of impact, including media impact– Fact sheets for 16 indicator initiatives– Understanding of success factors for Beyond GDP
indicators.
Report available at: www.brainpoolproject.eu/research
This presentation
The initiatives studied
Domestic Material Consumption
Happy Life Years
OECD Handbook of Subjective Well-Being
UN Commission for Sustainable Development
User factors
User factors
Indicator factors
Salience
Legitimacy
Credibility
Indicator factors
Salience
Legitimacy
Credibility
UserIndicator
Success factors
Policy / Context factors
Policy / Context factors
Relationship factors
Relationship factors
The public
Policy-makers
Scientific community
From the perspectives of:
The factors that determine the extent to which an indicator is used by users cluster into four categories (the light blue spheres).Of these, indicator factors need to be understand from three different perspectives.
Indicator Factors – salience for policy makers
aenimation
Indicator Factors – salience for policy makers
• Fit with a vision or organisational strategy – this is particularly relevant for those initiatives promoting new indicators so as to shift priorities or assess progress differently.
• Measure things that can be influenced by policy – this can be problematic for alternative indicators seeking to measure overarching concepts such as progress or well-being.
• Low-cost or money-saving – e.g. minimal expensive data collection, providing clues for low cost policies or ways to save money.
Indicator Factors – salience for policy makers
• Links with other outcomes– links between what the indicator measures and other outcomes (e.g. subjective well-being being related to reducing staff turnover).
• Reaching multiple audiences – this can ensure indicators do not sit within particular silos and can achieve cross-cutting outcomes.
• Perceived need – this is particularly important where initiatives are bringing together data rather than creating new measures.
Indicator Factors – salience for public/broad audience
Knivesout
Indicator Factors – salience for public/broad audience
• Simplicity – initiatives are effective when they allow one to produce a simple and attractive message.
• Ease of understanding – while what they measure may be complicated, successful indicators manage to illustrate a complex reality using understandable concepts.
• Engagement with communications experts – close collaboration, rather than simply handing over data, can ensure that both communicability and accuracy are maintained.
• Avoiding taboo words – in the UK ‘happiness’ is considered ‘woolly’ or ‘unscientific’, while in the USA, practitioners have avoided the mention of ‘climate change’ instead referring to ‘air quality’.
Indicator Factors – Credibility
GuySie
Indicator Factors – Credibility
• Data quality – a particular concern was whether subjective well-being data changes over time.
• Concerns regarding composite indicators – concerns over methodology and the weighting of different components of composite indicators can elicit strong resistance. The OECD’s Better Life Initiative has managed to secure acceptance of a composite indicator by allowing users to decide for themselves how to weight the different dimensions of the measure
Indicator Factors – Legitimacy
United Nations Photo
Indicator Factors – Legitimacy
• Being (or appearing) neutral – some indicator initiatives work within a framework of simply providing ‘neutral’ information, while others are clearly connected to political agendas, such as social cohesion or respecting environmental limits. Mechanisms used to ensure neutrality included monitoring funding mix and barring staff involvement in political parties.
• Institutional power – governmental bodies or supra-governmental bodies like the Council of Europe and the OECD often carry greater legitimacy than NGOs.
Relationship and process factors
Rabanito
Relationship and process factors
Relationship and process factors
• Engage one’s audience from the start – fundamental to the success of local initiatives, it was also seen in terms of getting policy-makers involved in large-scale initiatives.
• Direct contact with audiences – while not all initiatives can or want to engage their audience from the beginning, all the most successful initiatives had direct contact with the people they were trying to influence.
• Small is beautiful – to date, local initiatives have been able to achieve more impact than larger/national ones, with local bodies tending to be more ‘flexible’ and responsive.
Relationship and process factors
• Partnership working – aside from allowing a greater network to be reached and a greater skill base to be marshalled, partnerships allow different organisations to take on different roles. This can ensure an initiative is not too associated with a particular agenda.
• Picking one’s audience – On the one hand, some initiatives worked with individuals within organisations who could be seen as ‘allies’, or organisations who are overall supportive. On the other hand, several initiatives highlighted the need to reach those bodies potentially least sympathetic to their initiative – ministries of finance, treasuries or economic departments.
AEN Foto
User factors
User factors
• Users’ capacity to use social and environmental indicators – Beyond GDP initiatives typically involve a rebalancing towards social and environmental indicators and away from economic ones. This is not just a matter of calculating different things but of grasping different disciplines, and valuing different academic perspectives.
• The OECD’s approach of using economic techniques with subjective well-being may be a fruitful technique for convincing economists, by using their own language.
Policy and context factors
Ryan Fitton
Policy and context factors
• The Stiglitz/Sen/Fitoussi Commision – Seen as the biggest positive factor.
• The economic crisis - On the contrary, the financial crisis is seen by many as hindering Beyond GDP efforts, leading people to view well-being as a distraction. Others, however, see an opportunity in highlighting the role of the fixation on GDP in causing the crisis.
Policy and context factors
• Ideology as a barrier – For example, subjective well-being has been criticised from a libertarian perspective as not being something government should influence.
• Vested interests – might Beyond GDP efforts have a negative effect for certain groups?
• Public pressure – support for the idea of alternative indicators required from the bottom up.
• Indicator initiatives take time – a last sobering lesson is that it can take generations for an indicator to become sufficiently embedded in the system to maximise its impact.
For the full report, visit:
www.brainpoolproject.eu/research
For more information please contact:
Alistair Whitby, World Future Council
Saamah Abdallah, nef (the new economics foundation)
Tomas Hak, Charles University Environment Centre
[email protected] James Jordan