Screen 1 of 27
TargetingMonitoring and Evaluation of Targeting
LEARNING OBJECTIVES
Understand the basic measures to monitor and evaluate targeting activities.
Use qualitative information to integrate measures of targeting accuracy.
Screen 2 of 27
TargetingMonitoring and Evaluation of Targeting
INTRODUCTION
Good feedback information can be very important in fine-tuning or correcting the targeting during an ongoing operation.
Which types of information and analysis are useful in monitoring and evaluating how well
targeting works in practice?
Food security information systems can make an important contribution.
Screen 3 of 27
TargetingMonitoring and Evaluation of Targeting
INTRODUCTION
KEY QUESTIONS:
• Has the targeting system identified the right people?
• How many of the target group have been assisted?
• Has the targeting system succeeded in concentrating resources on the intended beneficiaries?
• Have any of the resources intended for the target group been diverted elsewhere?
Screen 4 of 27
TargetingMonitoring and Evaluation of Targeting
CAUSES OF TARGETING “ERROR”
In the real world, no targeting system is perfect.The imperfections are called targeting “errors”.
Some potential causes of targeting error:
Technical problems
• Inaccurate needs or vulnerability assessment
• Gaps in data or information
• Poor design
• Flawed implementation
Governance problems
• Malpractice, especially diversion or misuse of resources• Weak administrative capacity• Poor accountability • Poor analysis of power structures and interest groups • Clash of values • Cultural issues
Screen 5 of 27
TargetingMonitoring and Evaluation of Targeting
THREE APPROACHES TO MEASURING ACCURACY
1. How many people were correctly or incorrectly targeted?
2. How much aid reached the right people?
3. Who are the beneficiaries (and the non-beneficiaries)?
Key questions:
Screen 6 of 27
TargetingMonitoring and Evaluation of Targeting
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
The accuracy is often measured in terms of how many people are correctly or incorrectly receiving benefits.
The diagram shows how the target group and the beneficiaries may not exactly
overlap
Target population
Beneficiaries (population receiving assistance)
TOTAL POPULATION
CORRECT TARGETING
EXCLUSION ERROR
INCLUSION ERROR
1. How many people?
Screen 7 of 27
TargetingMonitoring and Evaluation of Targeting
Correct targeting is usually measured in terms of coverage.
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Coverage is the percentage of the target group who receive benefits.
Coverage and exclusion error are complementary.
TARGET GROUP
Exclusion Error
Coverage
Example – Coverage and exclusion error
Screen 8 of 27
TargetingMonitoring and Evaluation of Targeting
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Example – Perfect targeting
Community of 100 people.
20 meet the targeting criteria.
Perfect targeting
Beneficiaries Non-beneficiaries
Screen 9 of 27
TargetingMonitoring and Evaluation of Targeting
Example – Perfect targeting
total Beneficiaries Non-beneficiaries
total 100 20 80
Target group
20 0Exclusion error = 0/20
= 0
Non-target group
80 0Inclusion error = 0/20
= 0 %
80
20Coverage = 20/20
= 100%
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Screen 10 of 27
TargetingMonitoring and Evaluation of Targeting
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Example – Imperfect targeting (with adequate resources)
Imperfect targeting (with adequate resources)
Beneficiaries Non-beneficiaries
Total Beneficiaries Non-beneficiaries
Total 100 20 80
Target group
20 5Exclusion error = 5/20
= 25%
Non-target group
80 5Inclusion error = 5/20
= 25%
75
15Coverage = 15/20
= 75%
Screen 11 of 27
TargetingMonitoring and Evaluation of Targeting
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Example – Imperfect targeting (with under-resources)
Imperfect targeting B (with under-resourcing)
Beneficiaries Non-beneficiaries
Total
Beneficiaries Non-beneficiaries
Total 100 20 80
Target group
30 11
Exclusion error (or under-coverage) = __%
Non-target group
70 1
Inclusion error =__%
69
19
Coverage =__%
Screen 12 of 27
TargetingMonitoring and Evaluation of Targeting
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Example – Imperfect targeting (with under-resources)
Imperfect targeting B (with under-resourcing)
Beneficiaries Non-beneficiaries
Total
Beneficiaries Non-beneficiaries
Total 100 20 80
Target group
30 1111 / 30
Exclusion error (or under-coverage) = 37 %
Non-target group
70 11 /20
Inclusion error = 5 %
69
1919 /30
Coverage = 63 %
Screen 13 of 27
TargetingMonitoring and Evaluation of Targeting
2. How much aid?
Another approach to measuring the success of targeting is by assessing the quantity or proportion of aid actually received by the
target group.
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Screen 14 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Beneficiaries
Beneficiaries (20)
Target group (15) Non-target group (5)
(25% inclusion error)
Target group Non-target group
Example – Imperfect targeting (with adequate resources)
Screen 15 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Targeting Effectiveness = the percentage of transfer resources actually delivered to the intended target group.
Target group Non-target group
targeting effectiveness
= 75%
Effectiveness Leakage
Effectiveness and leakage are complementary.
Screen 16 of 27
TargetingMonitoring and Evaluation of Targeting
IDENTIFYING TARGET GROUPS – METHODS OF TARGETINGEFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
The 20 sacks have been distributed as follows:
• 18 to the correctly targeted households; and • only 2 to the non-target group.
Example – Imperfect targeting (with adequate resources) - 2
Target group Non-target group
Effectiveness = ?
Leakage = ?
Screen 17 of 27
TargetingMonitoring and Evaluation of Targeting
IDENTIFYING TARGET GROUPS – METHODS OF TARGETINGEFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
The 20 sacks have been distributed as follows:• 18 to the correctly targeted households; and • only 2 to the non-target group.
Example – Imperfect targeting (with adequate resources) - 2
Target group Non-target group
Effectiveness = 18/20 = 90%
Leakage = 2/20 (or 100-90)
= 10%
…of course, the opposite can also happen...
Screen 18 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Example – Imperfect targeting (with adequate resources) - 3
Target group Non-target group
?? Missing resources
Effectiveness = ?
Leakage = ?
Target group -> 12 sacks in total. Non-target group beneficiaries -> 4 sacks. Four sacks are unaccounted for.
Screen 19 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Example – Imperfect targeting (with adequate resources) - 3
Target group Non-target group
?? Missing resources
Effectiveness = 12/20 = 60%
Leakage = 100-60 = 40%
Target group -> 12 sacks in total. Non-target group beneficiaries -> 4 sacks. Four sacks are unaccounted for.
Screen 20 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
TOTAL POPULATION
TARGET GROUP
Targeting Performance = % of resources received by the target group / % of target group in population.
Targeting effectiveness can be compared with the results of an untargeted distribution.
The accuracy of the targeting can be measured in terms of how much more the target group receive from the targeted programme.
Screen 21 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
It is worse than no targeting.
PERFORMANCE < 1
PERFORMANCE = 1
PERFORMANCE > 1
The targeting has had no effect .
Targeting has improved the outcome for the target group.
Screen 22 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Dilution = the amount of aid received by each targeted beneficiary, as a percentage of the planned amount.
Dilution is the opposite of concentration (the aim of targeting).
It is a common targeting problem in which aid is shared among a larger number of people than the programme was designed for.
Screen 23 of 27
TargetingMonitoring and Evaluation of Targeting
BENEFICIARY & NON-BENEFICIARY PROFILES – WHO’S IN, WHO’S OUT?
3. Who are the beneficiaries?
The third group of targeting indicators falls under the question of who is included and who is
excluded by the targeting system.
It is very useful to compare the characteristics of the beneficiary and non-beneficiary populations.
If there is a clear difference between them it means that the targeting has been effective.
Screen 24 of 27
TargetingMonitoring and Evaluation of Targeting
BENEFICIARY & NON-BENEFICIARY PROFILES – WHO’S IN, WHO’S OUT?
It is also useful to look at the profiles of people in the exclusion and inclusion errors.
Exclusion Inclusion
Do the excluded target group members have any factor in common that might suggest systematic marginalization?
Who are the people wrongly included among the beneficiaries? Why and how the error is happening?
Screen 25 of 27
TargetingMonitoring and Evaluation of Targeting
THE BOTTOM LINE
Standard of accuracy and targeting priorities are different in different contexts.
Targeting is more accurate in countries with higher incomes, good administrative capacity, and high levels of inequality.
In emergency operations, the priority is often to maximize coverage , while in other situations, the priority may be to eliminate inclusion error and leakage.
Screen 26 of 27
TargetingMonitoring and Evaluation of Targeting
THE BOTTOM LINE
Contextual information and qualitative assessment of how the targeting system is working help to interpret the error measurements in a meaningful way.
• Why are problems or errors happening? What can be changed in order to reduce them?
• What do beneficiaries and their community think about the targeting?
• How are targeting decisions actually made, and by whom?
• How transparent and accountable is the process?
• Does the targeting match the programme’s objectives?
• Could the targeting design have been better?
Screen 27 of 27
TargetingMonitoring and Evaluation of Targeting
SUMMARY
No targeting system is perfect. Usually some degree of inaccuracy must be accepted to make the programme workable and affordable. A judgement has to be made about which kinds of error, and what degree of error, are acceptable given the programme’s objectives and context.
The most useful indicators for measuring the accuracy of targeting fall under 3 key questions:
1. How many people were correctly or incorrectly identified as beneficiaries?
• Coverage = % of target group receiving benefits
• Exclusion = % of target group not receiving benefits
• Inclusion = % of beneficiaries who are not target group members
2. How much of the resources for distribution reached the right people?
• Effectiveness = % of transfer resources received by the target group
• Leakage = % of transfer resources not received by the target group
• Performance = % of transfer resources received by the target group, divided by the % they would have received from an untargeted distribution
3. Who is included, who is excluded?