Measuring human rights

Purpose of measurementLevels of measurementCategories and dimensionsObjects of measurementProblems of measurement

Purposes of measurement

Contextual description Monitoring Documentation

Classification Types of regime Types of governance Types of rights violations

Mapping (time and space) Global trends Regional Local

Secondary analysis Academic research Policy research Political dialogue

Levels of social scientific measurement

L e v e l 4S c o re s fo r U n its

T h e sc o res fo r u n its o f o b se rva tio n (e .g. in d iv id ua ls , c o u n trie s , re g ion s) g e n e ra te d b y a p a rtic u la r in d ic a to r.Q u a n titativ e a n d q u a litativ e d a ta .

L e v e l 3In d ic a to rs

A lso re ferre d to as 'm e a su re s ', 'o p e ra tio n a lisa tio n s ', an d c la ssific a tio nsE v e n ts -b a se d, s ta n d a rd s-b a se d (o rd ina l, in te rv a l, no m in a l), su rv e y -b a se d (ord in a l, in te rv a l, n o m in a l)

L e v e l 2S y stem a tiz e d C o nc e pt

A sp e c ific fo rm u la tio n o f a c o nc e p t u se d b y sch o la r, IG O , N G OD im e n sio n s an d c o m p o n e nts o f c o n c e pt

L e v e l 1B a c k g ro u nd C o n c e pt

T h e b roa d c o n ste lla tion o f m ea n in g s a n d u n de rs ta n din g s a sso c ia ted w ith a g iv e n c o n c e ptN o rm a tiv e a n d e m p ir ic al th e o ry

Adapted from: Zeller and Carmines 1980; Munck and Verkuilen 2000; Adcock and Collier 2001; Ball and Spirer 2000

Measuring Example

L e v e l 4S c o re s fo r U n itsB R A Z IL 1 9 8 5

L O W E X E C U T IV E C O N S T R A IN T ; H IG H C IV IL R IG H T S V IO L A T IO N S

L e v e l 3In d ic a to rs

E X E C U T IV E C O N S T R A IN T ; C IV IL R IG H T S V IO L A T IO N S

L e v e l 2S y s tem a tiz e d C o nc e pt

L IB E R A L D E M O C R A C YIN S T IT U T IO N A L D IM E N S IO N ; R IG H T S D IM E N S IO N

L e v e l 1B a c k g ro u nd C o n c e pt

D E M O C R A C Y

Categories and dimensions of human rights

Categories Civil rights Political rights Economic rights Social rights Cultural rights Solidarity rights

Dimensions Protect Respect Fulfil

Categories

Dimensions

I

Respect (no interference in the exercise of the

right)

II Protect

(prevent violations from third parties)

III Fulfil

(provision of resources and the

outcomes of policies)

Civil and political

Torture, extra-judicial

killings, disappearance,

arbitrary detention, unfair trials, electoral

intimidation, disenfranchisement

Measures to

prevent non-state actors from committing

violations, such as

Investment in

judiciaries, prisons, police forces, and

elections, and resource allocations

to ability

Cat

egor

ies

of h

uma

n rig

hts

Economic, social, and cultural

Ethnic, racial, gender,

or linguistic discrimination in

health, education, and welfare and

resource allocations below ability.

Measures to

prevent non-state actors from engaging in

discriminatory behaviour the limits

access

Progressive realisation

Investment in health, education, and

welfare, and resource allocations to ability

Objects of measurementPrinciple (de jure)

International legal National legal

Practice (de facto) Events-based Standards-based

Dichotomous categories Polychotomous scales

Survey-based Hybrid measures

Policy Input Process Output Outcome

Principle (de jure) measurement

Code treaty participation (scale) No signature (0) Signature (1) Ratification (2) Ratification with reservations (weighting)

Code national constitutions (n or scale) Articles on civil rights Articles on political rights Articles on economic rights Articles on social rights Articles on cultural rights

International de jure human rights

0

20

40

60

80

100

120

140

160

180

200

1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000

Year

Num

ber

of C

ount

ries

Rat

ifyi

ng I

nstr

umen

ts

ICESCR ICCPR OPT1 OPT2 CERD CEDAW CAT CRC

Landman (2005) Protecting Human Rights: A Comparative Study, Washington DC: Georgetown University Press

International de jure rights: coding reservations

Rewarding the absence of reservations Countries with no reservations with regard to said treaty that do not

modify obligations, or non-substantial declarations (score = 4) Countries whose reservations could have some but not major impact on

their obligations (score = 3) Countries whose reservations have noticeable effect on the obligations

(score = 2) Countries whose reservations can have significant and severe effects on

treaty obligations (score = 1)

The ratification score No signature (0) Signature (1) Ratification (2)

Weighting the ratification score Weighted Ratification = [Ratification score (0,1,2) * Reservations score

(1,2,3,4)] High score = ratification with fewer substantial reservations Low score = ratification with more substantial reservations

International de jure rights: ICCPR with and without reservations

Year

2000

1998

1996

1994

1992

1990

1988

1986

1984

1982

1980

1978

1976

Me

an

6

5

4

3

2

1

0

ICCPR (1966)

CCPRRESW

Landman (2005) Protecting Human Rights: A Comparative Study, Washington DC: Georgetown University Press

International de jure rights: ICCPR with reservations

Regional groupingsM

iddl

e E

ast

Pos

t-C

omm

unis

t Eur

op

Com

mun

ist E

urop

e

Eur

ope

Pac

ific

Asi

a

Am

eric

as

Afr

ica

Me

an

CC

PR

RE

SW

8

6

4

2

0

Landman (2005) Protecting Human Rights: A Comparative Study, Washington DC: Georgetown University Press

Domestic de jure civil and political rights

Civil and Political Rights

0

20

40

60

80

100

120

Lif

e

Sla

very

/ser

vitu

de

Tor

ture

/cru

eltr

eatm

ent

Equ

alit

y be

fore

the

law

Arb

itra

ry d

eten

tion

Pri

vate

lif

e

Fre

edom

of

mov

emen

t

Asy

lum

Tho

ught

/opi

nion

Exp

ress

ion

Ass

embl

y/as

soci

atio

n

Vot

e

Par

tici

pati

on

Types of Rights

Per

cena

tge

of N

atio

nal

Con

stit

utio

ns

1788-1948 1949-1957 1958-1966 1967-1975

Source: van Maarseveen and Tang (1978)

Domestic de jure economic and social rights

Economic and Social Rights

0

10

20

30

40

50

60

70

80

Soc

ial

secu

rity

Wor

k

Fre

e ch

oice

of

job

Fai

r/eq

ual

pay

Tra

de u

nion

s

Res

t/le

isur

e

Sta

ndar

d of

liv

ing

Edu

cati

on

Types of Rights

Per

cent

age

of N

atio

nal

Con

stit

utio

ns

1788-1948 1949-1957 1958-1966 1967-1975

Source: van Maarseveen and Tang (1978)

Practice (de facto) measurement

Events-basedStandards-basedSurvey-basedHybrid

De facto measurement: events-based methodology for human rights

Disaggregated events (‘who did what to whom’) Act Violation(s) Perpetrator Victim Context

When Where

Controlled vocabulariesAggregated event countsMultiple sources of information

De facto measurement: events-based data model

Source: http://shr.aaas.org/hrdag/idea/datamodel/index.html

Measuring de facto rights: events-based example in Kosovo

Source: Patrick Ball and Jana Asher

Estimated total refugee migration and killings over time, in Kosovo

Source: http://shr.aaas.org/hrdag/project-38.php

Measuring de facto rights: events-based example in Peru, 1980-2000

Measuring de facto rights: events-based example for abuse against Human Rights Defenders, 1997-2000

Source: Landman (2006)’Holding the Line: Human Rights Defenders in the Age of Terror’, British Journal of Politics and International Relations, Vol. 8..

0

100

200

300

400

500

600

700

Alg

eria

Arg

enti

naA

rmen

iaA

zerb

aija

nB

angl

ades

hB

elar

usB

oliv

iaB

osni

a H

erze

gove

nia

Bot

swan

aB

urki

na F

aso

Bur

undi

Cam

bodi

aC

amer

oon

Cha

dC

hile

Chi

naC

olom

bia

Con

go,

Dem

ocra

tic

Con

go,

Rep

ubli

c of

the

Cos

ta R

ica

Cot

e d'

Ivoi

reC

roat

iaC

uba

Cze

ch R

epub

lic

Dji

bout

iE

ast T

imor

Les

teE

cuad

orE

gypt

El

Sal

vado

rE

thio

pia

Gam

bia

Geo

rgia

Gre

ece

Gua

tem

ala

Gui

nea-

Bis

sau

Hai

tiH

ondu

ras

Indi

aIn

done

sia

Iran

Isra

elIt

aly

Jord

anK

azak

hsta

nK

enya

Kor

ea,

Rep

ubli

c of

Kyr

gyzs

tan

Lao

sL

eban

onL

iber

iaM

aced

onia

Mal

aysi

aM

auri

tani

aM

auri

tius

Mex

ico

Mor

occo

Nam

ibia

Nep

alN

icar

agua

Nig

eria

Pak

ista

nP

anam

aP

eru

Phi

lipp

ines

Rus

sia

Rw

anda

Sen

egal

Spa

inS

ri L

anka

Sud

anS

yria

Tan

zani

aT

hail

and

Tog

oT

unis

iaT

urke

yT

urkm

enis

tan

Ukr

aine

Uni

ted

Kin

gdom

Uzb

ekis

tan

Ven

ezue

laV

ietn

amY

emen

Yug

osla

via

post

Zam

bia

Zim

babw

e

Country

HR

D a

buse

(N

act

s)

De facto measurement: standards-based methodology

Ideal standards Legal instruments Conceptual definition

Democracy Good governance Human rights

Empirical information Monitoring bodies

Human rights treaty bodies NGOs (e.g. Amnesty International/Human Rights Watch) Governments (e.g. US State Department)

Newspapers Historical accounts/narratives

Coding Dichotomous categories

ACLP Doorenspleet

Polychotomous scales Freedom House Political Terror Scale Torture Scale

De facto measurement: standards-based scales of rights

1

1.5

2

2.5

3

3.5

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

Year

Rig

hts

Sca

le

FH Political Rights

FH Civil Rights

Torture Scale (Hathaway)

Amnesty PIR

State Department PIR

Landman (2005) Protecting Human Rights: A Comparative Study, Washington DC: Georgetown University Press

De facto rights:Standards-based measures across space

REGION

Eas

t A

sia

Pac

ific

So

uth

Asi

a

ME

NA

Su

b S

ah A

fric

a

Lat

in A

mer

ica

No

rth

Am

eric

a

Po

st-C

om

m E

uro

pe

Co

mm

Eu

rop

e

Eu

rop

e

Mea

n Sc

ore

(Hig

h =

wor

se)

5

4

3

2

1

0

PTS (AI)

PTS (SD)

Torture Scale

Freedom House CR

Freedom House PR

Landman (2005) Protecting Human Rights: A Comparative Study, Washington DC: Georgetown University Press

De facto measurement: standards-based scales of political and civil rightsCingranelli and Richards (CIRI) data set

YEAR

2003

2001

1999

1997

1995

1993

1991

1989

1987

1985

1983

1981

Mea

n1.6

1.4

1.2

1.0

.8

.6

.4

POLPAR

SPEECH

ASSN

KILL

MOVE

TORT

POLPRIS

www.humanrightsdata.com

De facto measurement: standards-based scales of women’s and workers’ rightsCingranelli and Richards (CIRI) data set

YEAR

2003

2001

1999

1997

1995

1993

1991

1989

1987

1985

1983

1981

Mea

n2.0

1.8

1.6

1.4

1.2

1.0

.8

.6

.4

WECON

WOPOL

WORKER

WOSOC

www.humanrightsdata.com

De facto measurement: survey-based methodology

Sample of the population ‘VIPs’ Quota sample Random

Standardised questionsReponses

Open Closed

De facto measurement: survey-based measure of human rights

82

84

86

88

90

92

94

96

France UK Germany Netherlands USA Mexico Brazil Chile

Country

Per

cent

age

of r

espo

nden

ts

World Values Survey (1994) question on support for the idea of human rights in 1990 across eight countries (1002 N 2095).

De facto measurement: survey-based measure of human rights

Physicians for Human Rights (2002); N = 991 IDPs in Sierra Leone

Policy indicators

InputProcessPerformanceOutputOutcomePerception

Policy indicators: input

Provision of resources Spending in education Spending on health service Spending on housing

Policy indicators: process

Health Number of patients seen per day Waiting lists Average journey time to hospital

Water Time it takes to access clean water Number of trips to water source

needed per day

Policy indicators: performance

Health Time it takes to build new hospitals Time it takes to deliver new beds Time it takes to train and recruit new

doctors

Water Time it takes to provide a water

connection Time it takes to build a sewerage system

Policy indicators: output

Health Number of doctors per 100,000 Number of hospital beds per 100,000 Number of hospitals per geographical

area

Water Households with access to water

within 200m of dwelling Increase in quality of water

Policy indicators: outcome

Health Infant mortality rates Longevity rates New HIV/AIDs cases

Water Level of water born diseases Infant mortality rate

Policy indicators: perception

Attitudinal dataSurveysFeedback questionnairesCan evaluate inputs, process, and outputs

Problems of measurement

ValidityReliabilityMeasurement biasLack of transparencyVariance truncationInformation biasAggregation problems