what makes indicators successful? lessons from practitioners

What makes indicators successful? Lessons from practitioners

Funded by:

The Project

BRAINPOoL (Bringing alternative indicators into policy) is an EU-funded project aimed at identifying and overcoming the barriers to ‘Beyond GDP’ indicators being used in policy.

During the project we are carrying out research and interviews, conducting workshops and knowledge-brokerage seminars and carrying out various action research case studies to explore ways to improve uptake of Beyond GDP indicators.

The Project (work packagestructure)

This presentation

Understanding the supply of Beyond GDP indicators

• Work Package 1 outcomes:– Catalogue of over 100 Beyond GDP indicator initiatives– Understanding of intentions of indicator

producers/promoters– Documenting of impact, including media impact– Fact sheets for 16 indicator initiatives– Understanding of success factors for Beyond GDP

indicators.

Report available at: www.brainpoolproject.eu/research

This presentation

The initiatives studied

Domestic Material Consumption

Happy Life Years

OECD Handbook of Subjective Well-Being

UN Commission for Sustainable Development

User factors

User factors

Indicator factors

Salience

Legitimacy

Credibility

Indicator factors

Salience

Legitimacy

Credibility

UserIndicator

Success factors

Policy / Context factors

Policy / Context factors

Relationship factors

Relationship factors

The public

Policy-makers

Scientific community

From the perspectives of:

The factors that determine the extent to which an indicator is used by users cluster into four categories (the light blue spheres).Of these, indicator factors need to be understand from three different perspectives.

Indicator Factors – salience for policy makers

aenimation

Indicator Factors – salience for policy makers

• Fit with a vision or organisational strategy – this is particularly relevant for those initiatives promoting new indicators so as to shift priorities or assess progress differently.

• Measure things that can be influenced by policy – this can be problematic for alternative indicators seeking to measure overarching concepts such as progress or well-being.

• Low-cost or money-saving – e.g. minimal expensive data collection, providing clues for low cost policies or ways to save money.

Indicator Factors – salience for policy makers

• Links with other outcomes– links between what the indicator measures and other outcomes (e.g. subjective well-being being related to reducing staff turnover).

• Reaching multiple audiences – this can ensure indicators do not sit within particular silos and can achieve cross-cutting outcomes.

• Perceived need – this is particularly important where initiatives are bringing together data rather than creating new measures.

Indicator Factors – salience for public/broad audience

Knivesout

Indicator Factors – salience for public/broad audience

• Simplicity – initiatives are effective when they allow one to produce a simple and attractive message.

• Ease of understanding – while what they measure may be complicated, successful indicators manage to illustrate a complex reality using understandable concepts.

• Engagement with communications experts – close collaboration, rather than simply handing over data, can ensure that both communicability and accuracy are maintained.

• Avoiding taboo words – in the UK ‘happiness’ is considered ‘woolly’ or ‘unscientific’, while in the USA, practitioners have avoided the mention of ‘climate change’ instead referring to ‘air quality’.

Indicator Factors – Credibility

GuySie

Indicator Factors – Credibility

• Data quality – a particular concern was whether subjective well-being data changes over time.

• Concerns regarding composite indicators – concerns over methodology and the weighting of different components of composite indicators can elicit strong resistance. The OECD’s Better Life Initiative has managed to secure acceptance of a composite indicator by allowing users to decide for themselves how to weight the different dimensions of the measure

Indicator Factors – Legitimacy

United Nations Photo

Indicator Factors – Legitimacy

• Being (or appearing) neutral – some indicator initiatives work within a framework of simply providing ‘neutral’ information, while others are clearly connected to political agendas, such as social cohesion or respecting environmental limits. Mechanisms used to ensure neutrality included monitoring funding mix and barring staff involvement in political parties.

• Institutional power – governmental bodies or supra-governmental bodies like the Council of Europe and the OECD often carry greater legitimacy than NGOs.

Relationship and process factors

Rabanito

Relationship and process factors

Relationship and process factors

• Engage one’s audience from the start – fundamental to the success of local initiatives, it was also seen in terms of getting policy-makers involved in large-scale initiatives.

• Direct contact with audiences – while not all initiatives can or want to engage their audience from the beginning, all the most successful initiatives had direct contact with the people they were trying to influence.

• Small is beautiful – to date, local initiatives have been able to achieve more impact than larger/national ones, with local bodies tending to be more ‘flexible’ and responsive.

Relationship and process factors

• Partnership working – aside from allowing a greater network to be reached and a greater skill base to be marshalled, partnerships allow different organisations to take on different roles. This can ensure an initiative is not too associated with a particular agenda.

• Picking one’s audience – On the one hand, some initiatives worked with individuals within organisations who could be seen as ‘allies’, or organisations who are overall supportive. On the other hand, several initiatives highlighted the need to reach those bodies potentially least sympathetic to their initiative – ministries of finance, treasuries or economic departments.

AEN Foto

User factors

User factors

• Users’ capacity to use social and environmental indicators – Beyond GDP initiatives typically involve a rebalancing towards social and environmental indicators and away from economic ones. This is not just a matter of calculating different things but of grasping different disciplines, and valuing different academic perspectives.

• The OECD’s approach of using economic techniques with subjective well-being may be a fruitful technique for convincing economists, by using their own language.

Policy and context factors

Ryan Fitton

Policy and context factors

• The Stiglitz/Sen/Fitoussi Commision – Seen as the biggest positive factor.

• The economic crisis - On the contrary, the financial crisis is seen by many as hindering Beyond GDP efforts, leading people to view well-being as a distraction. Others, however, see an opportunity in highlighting the role of the fixation on GDP in causing the crisis.

Policy and context factors

• Ideology as a barrier – For example, subjective well-being has been criticised from a libertarian perspective as not being something government should influence.

• Vested interests – might Beyond GDP efforts have a negative effect for certain groups?

• Public pressure – support for the idea of alternative indicators required from the bottom up.

• Indicator initiatives take time – a last sobering lesson is that it can take generations for an indicator to become sufficiently embedded in the system to maximise its impact.

For the full report, visit:

www.brainpoolproject.eu/research

For more information please contact:

Alistair Whitby, World Future Council

alistair.whitby@worldfuturecouncil.org

Saamah Abdallah, nef (the new economics foundation)

saamah.abdallah@neweconomics.org

Tomas Hak, Charles University Environment Centre

tomas.hak@czp.cuni.cz James Jordan