screen 1 of 27 targeting monitoring and evaluation of targeting learning objectives understand the...
TRANSCRIPT
![Page 1: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/1.jpg)
Screen 1 of 27
TargetingMonitoring and Evaluation of Targeting
LEARNING OBJECTIVES
Understand the basic measures to monitor and evaluate targeting activities.
Use qualitative information to integrate measures of targeting accuracy.
![Page 2: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/2.jpg)
Screen 2 of 27
TargetingMonitoring and Evaluation of Targeting
INTRODUCTION
Good feedback information can be very important in fine-tuning or correcting the targeting during an ongoing operation.
Which types of information and analysis are useful in monitoring and evaluating how well
targeting works in practice?
Food security information systems can make an important contribution.
![Page 3: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/3.jpg)
Screen 3 of 27
TargetingMonitoring and Evaluation of Targeting
INTRODUCTION
KEY QUESTIONS:
• Has the targeting system identified the right people?
• How many of the target group have been assisted?
• Has the targeting system succeeded in concentrating resources on the intended beneficiaries?
• Have any of the resources intended for the target group been diverted elsewhere?
![Page 4: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/4.jpg)
Screen 4 of 27
TargetingMonitoring and Evaluation of Targeting
CAUSES OF TARGETING “ERROR”
In the real world, no targeting system is perfect.The imperfections are called targeting “errors”.
Some potential causes of targeting error:
Technical problems
• Inaccurate needs or vulnerability assessment
• Gaps in data or information
• Poor design
• Flawed implementation
Governance problems
• Malpractice, especially diversion or misuse of resources• Weak administrative capacity• Poor accountability • Poor analysis of power structures and interest groups • Clash of values • Cultural issues
![Page 5: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/5.jpg)
Screen 5 of 27
TargetingMonitoring and Evaluation of Targeting
THREE APPROACHES TO MEASURING ACCURACY
1. How many people were correctly or incorrectly targeted?
2. How much aid reached the right people?
3. Who are the beneficiaries (and the non-beneficiaries)?
Key questions:
![Page 6: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/6.jpg)
Screen 6 of 27
TargetingMonitoring and Evaluation of Targeting
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
The accuracy is often measured in terms of how many people are correctly or incorrectly receiving benefits.
The diagram shows how the target group and the beneficiaries may not exactly
overlap
Target population
Beneficiaries (population receiving assistance)
TOTAL POPULATION
CORRECT TARGETING
EXCLUSION ERROR
INCLUSION ERROR
1. How many people?
![Page 7: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/7.jpg)
Screen 7 of 27
TargetingMonitoring and Evaluation of Targeting
Correct targeting is usually measured in terms of coverage.
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Coverage is the percentage of the target group who receive benefits.
Coverage and exclusion error are complementary.
TARGET GROUP
Exclusion Error
Coverage
Example – Coverage and exclusion error
![Page 8: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/8.jpg)
Screen 8 of 27
TargetingMonitoring and Evaluation of Targeting
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Example – Perfect targeting
Community of 100 people.
20 meet the targeting criteria.
Perfect targeting
Beneficiaries Non-beneficiaries
![Page 9: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/9.jpg)
Screen 9 of 27
TargetingMonitoring and Evaluation of Targeting
Example – Perfect targeting
total Beneficiaries Non-beneficiaries
total 100 20 80
Target group
20 0Exclusion error = 0/20
= 0
Non-target group
80 0Inclusion error = 0/20
= 0 %
80
20Coverage = 20/20
= 100%
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
![Page 10: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/10.jpg)
Screen 10 of 27
TargetingMonitoring and Evaluation of Targeting
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Example – Imperfect targeting (with adequate resources)
Imperfect targeting (with adequate resources)
Beneficiaries Non-beneficiaries
Total Beneficiaries Non-beneficiaries
Total 100 20 80
Target group
20 5Exclusion error = 5/20
= 25%
Non-target group
80 5Inclusion error = 5/20
= 25%
75
15Coverage = 15/20
= 75%
![Page 11: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/11.jpg)
Screen 11 of 27
TargetingMonitoring and Evaluation of Targeting
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Example – Imperfect targeting (with under-resources)
Imperfect targeting B (with under-resourcing)
Beneficiaries Non-beneficiaries
Total
Beneficiaries Non-beneficiaries
Total 100 20 80
Target group
30 11
Exclusion error (or under-coverage) = __%
Non-target group
70 1
Inclusion error =__%
69
19
Coverage =__%
![Page 12: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/12.jpg)
Screen 12 of 27
TargetingMonitoring and Evaluation of Targeting
COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?
Example – Imperfect targeting (with under-resources)
Imperfect targeting B (with under-resourcing)
Beneficiaries Non-beneficiaries
Total
Beneficiaries Non-beneficiaries
Total 100 20 80
Target group
30 1111 / 30
Exclusion error (or under-coverage) = 37 %
Non-target group
70 11 /20
Inclusion error = 5 %
69
1919 /30
Coverage = 63 %
![Page 13: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/13.jpg)
Screen 13 of 27
TargetingMonitoring and Evaluation of Targeting
2. How much aid?
Another approach to measuring the success of targeting is by assessing the quantity or proportion of aid actually received by the
target group.
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
![Page 14: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/14.jpg)
Screen 14 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Beneficiaries
Beneficiaries (20)
Target group (15) Non-target group (5)
(25% inclusion error)
Target group Non-target group
Example – Imperfect targeting (with adequate resources)
![Page 15: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/15.jpg)
Screen 15 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Targeting Effectiveness = the percentage of transfer resources actually delivered to the intended target group.
Target group Non-target group
targeting effectiveness
= 75%
Effectiveness Leakage
Effectiveness and leakage are complementary.
![Page 16: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/16.jpg)
Screen 16 of 27
TargetingMonitoring and Evaluation of Targeting
IDENTIFYING TARGET GROUPS – METHODS OF TARGETINGEFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
The 20 sacks have been distributed as follows:
• 18 to the correctly targeted households; and • only 2 to the non-target group.
Example – Imperfect targeting (with adequate resources) - 2
Target group Non-target group
Effectiveness = ?
Leakage = ?
![Page 17: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/17.jpg)
Screen 17 of 27
TargetingMonitoring and Evaluation of Targeting
IDENTIFYING TARGET GROUPS – METHODS OF TARGETINGEFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
The 20 sacks have been distributed as follows:• 18 to the correctly targeted households; and • only 2 to the non-target group.
Example – Imperfect targeting (with adequate resources) - 2
Target group Non-target group
Effectiveness = 18/20 = 90%
Leakage = 2/20 (or 100-90)
= 10%
…of course, the opposite can also happen...
![Page 18: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/18.jpg)
Screen 18 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Example – Imperfect targeting (with adequate resources) - 3
Target group Non-target group
?? Missing resources
Effectiveness = ?
Leakage = ?
Target group -> 12 sacks in total. Non-target group beneficiaries -> 4 sacks. Four sacks are unaccounted for.
![Page 19: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/19.jpg)
Screen 19 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Example – Imperfect targeting (with adequate resources) - 3
Target group Non-target group
?? Missing resources
Effectiveness = 12/20 = 60%
Leakage = 100-60 = 40%
Target group -> 12 sacks in total. Non-target group beneficiaries -> 4 sacks. Four sacks are unaccounted for.
![Page 20: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/20.jpg)
Screen 20 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
TOTAL POPULATION
TARGET GROUP
Targeting Performance = % of resources received by the target group / % of target group in population.
Targeting effectiveness can be compared with the results of an untargeted distribution.
The accuracy of the targeting can be measured in terms of how much more the target group receive from the targeted programme.
![Page 21: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/21.jpg)
Screen 21 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
It is worse than no targeting.
PERFORMANCE < 1
PERFORMANCE = 1
PERFORMANCE > 1
The targeting has had no effect .
Targeting has improved the outcome for the target group.
![Page 22: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/22.jpg)
Screen 22 of 27
TargetingMonitoring and Evaluation of Targeting
EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?
Dilution = the amount of aid received by each targeted beneficiary, as a percentage of the planned amount.
Dilution is the opposite of concentration (the aim of targeting).
It is a common targeting problem in which aid is shared among a larger number of people than the programme was designed for.
![Page 23: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/23.jpg)
Screen 23 of 27
TargetingMonitoring and Evaluation of Targeting
BENEFICIARY & NON-BENEFICIARY PROFILES – WHO’S IN, WHO’S OUT?
3. Who are the beneficiaries?
The third group of targeting indicators falls under the question of who is included and who is
excluded by the targeting system.
It is very useful to compare the characteristics of the beneficiary and non-beneficiary populations.
If there is a clear difference between them it means that the targeting has been effective.
![Page 24: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/24.jpg)
Screen 24 of 27
TargetingMonitoring and Evaluation of Targeting
BENEFICIARY & NON-BENEFICIARY PROFILES – WHO’S IN, WHO’S OUT?
It is also useful to look at the profiles of people in the exclusion and inclusion errors.
Exclusion Inclusion
Do the excluded target group members have any factor in common that might suggest systematic marginalization?
Who are the people wrongly included among the beneficiaries? Why and how the error is happening?
![Page 25: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/25.jpg)
Screen 25 of 27
TargetingMonitoring and Evaluation of Targeting
THE BOTTOM LINE
Standard of accuracy and targeting priorities are different in different contexts.
Targeting is more accurate in countries with higher incomes, good administrative capacity, and high levels of inequality.
In emergency operations, the priority is often to maximize coverage , while in other situations, the priority may be to eliminate inclusion error and leakage.
![Page 26: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/26.jpg)
Screen 26 of 27
TargetingMonitoring and Evaluation of Targeting
THE BOTTOM LINE
Contextual information and qualitative assessment of how the targeting system is working help to interpret the error measurements in a meaningful way.
• Why are problems or errors happening? What can be changed in order to reduce them?
• What do beneficiaries and their community think about the targeting?
• How are targeting decisions actually made, and by whom?
• How transparent and accountable is the process?
• Does the targeting match the programme’s objectives?
• Could the targeting design have been better?
![Page 27: Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities](https://reader035.vdocuments.us/reader035/viewer/2022062713/56649ce65503460f949b3c26/html5/thumbnails/27.jpg)
Screen 27 of 27
TargetingMonitoring and Evaluation of Targeting
SUMMARY
No targeting system is perfect. Usually some degree of inaccuracy must be accepted to make the programme workable and affordable. A judgement has to be made about which kinds of error, and what degree of error, are acceptable given the programme’s objectives and context.
The most useful indicators for measuring the accuracy of targeting fall under 3 key questions:
1. How many people were correctly or incorrectly identified as beneficiaries?
• Coverage = % of target group receiving benefits
• Exclusion = % of target group not receiving benefits
• Inclusion = % of beneficiaries who are not target group members
2. How much of the resources for distribution reached the right people?
• Effectiveness = % of transfer resources received by the target group
• Leakage = % of transfer resources not received by the target group
• Performance = % of transfer resources received by the target group, divided by the % they would have received from an untargeted distribution
3. Who is included, who is excluded?