The Use and Abuse of IndicatorsThe Use and Abuse of Indicators

Joachim NahemJoachim NahemUNDP Oslo Governance CentreUNDP Oslo Governance Centre

World Forum on Statistics, Knowledge and World Forum on Statistics, Knowledge and Policy: Istanbul 28-6-2007Policy: Istanbul 28-6-2007

"Economists have come to feel what can’t be measured, isn’t real. The truth is always an amount. Count numbers, only numbers count.“

“Not everything that can be counted counts, and not everything that counts can be counted.”

-Albert Einstein

Who Uses Governance Indicators and for What Purposes?

Plethora of governance indicators used by: – governments – development agencies– civil society– media – academic institutions– private sector

Indicators used for:– policy decisions– aid allocation– advocacy– research – business investment

Governance Indicators A Users’ Guide (2nd Edition 2006)

• Provides guidance on governance indicators; What are they? What are they used for? How to use and not to use indicators

• Intended for non-specialist users• Contains 35 sources of inter-country

governance indicators

Users Guide:‘The Normative Assumption’

• Every indicator will have an underlying normative assumption.

• Ex. Number of persons detained without charge – The assumption is that a lower number is better.

• Ex. Perceived state of corruption-The assumption is that less corruption is better.

Governance Indicators A Users’ Guide

Golden Rule 1: Use a range of indicators

• The single governance indicator which captures the subtleties and intricacies of national situations, in a manner which enables global, non-value laden comparison does not exist.

• Using just one indicator could very easily produce perverse assessments of any country and will rarely reflect the full situation.

• …at the other extreme, having too many indicators results in a different range of problems, including a lack of focus and burdensome data collection and analysis. The key is a balanced set with sufficient but not superfluous information.

Golden Rule 1: Use a Range of Indicators

Example : Voter turnout used as an indicator of democracy• Voter turnout figure – 85%• Clarify the definition: voter turnout is the number of those

casting votes as a proportion of those eligible. Who are eligible?

• Who were the 15%• Who is not eligible?• Who did vote?• Was there a choice not to vote?• What are the barriers to eligibility?• Are there any elements of compulsion?

Golden Rule 2: Use an Indicator as a first question-not a last

• Often indicators can be developed step by step as more information becomes available. In using an indicator one might go through the following steps.

• Yes/No – Asking the basic question, does a problem exist?• Number – After determining that a problem exists,

determining the size of the problem.• Percentage – Put the problem into context.• Significance – Use a significance test to examine whether the

problem is evenly spread or certain groups face more of a problem.

Golden Rule 3: Understand an Indicator Before You Use it

• By using an indicator you can be considered to be implicitly endorsing it, including its methodology and underlying values (normative assumptions)

Example: Voter Turnout - how is this defined? • It could either be votes cast as a percentage of voters

registered, or votes cast as a percentage of voting age population. Are there any other assessments which are included in the data – for example does the election have to be considered ‘free and fair’ before the data is included in the publication. How, who and where is the data compiled?

UNDP Project

Inter-country sources inadequate for governance and poverty reduction at national level

Need for governance indicators that are:• poverty and gender focused • policy-action oriented • nationally owned and participatory • methodologically sound