social policy evaluation: concepts, methods and limits. titre

Social policy evaluation: concepts , methods and limits.

• Titre

• Titre


1. Introduction

Socio-political aspect of evaluation: concerns all of us: beneficiary, taxpayer, voter and decision maker

Economic aspects: efficiency, costs and benefits analysis

Question:What can be obtained at what cost and at what probability

Quantitative methods

Micro simulation: ex ante and counterfactual ex post reform valuation.

Micro economic methods – ex post program evaluation:

Qualitative methods (interviews, experts…)

Socio-political dimension: beneficiary, taxpayer, voter and decision makers

Citizen – Voter

Taxpayer Beneficiary

Parliament

Government Administration and NGOs

Social policies: an interactive process

- Socio-economic needs: labor market policies (unemployment - work incentive policies) social sector financing (retirement founding, social VAT,

redistribution policies).- Reform projects (programs, expected results )

- Ex ante: expected effects’ evaluation.

- Controlled experiments (pilot programs: regions, populations)

- Implementation (legal and administrative procedures, financing rules)

- Ex post evaluation: proportion of positive responses, result-expectation analysis, take up evaluation

- Feedback (continuation, correction, abandonment)

- Risks of political sanction (election) evaluation

Evaluation of social and fiscal policies simulation and micro econometric methods

Unemployment expenditure (directs, actif and tax reductions)billions

euros

2000 2001 2002 2003 2004 2005

Unemployment compensation 20659,72 21837,71 26124,22 29242,85 30441,89 29 817,23

Activation policies 26526,85 27362,48 27776,65 26859,13 26198,36 26 168,25

Tax and contribution reductions 11576,14 14416,01 15424,75 16090,39 16275,48 17 193,43

Total unemployment expenditure 58 762,71 63 616,19 69 325,62 72 192,37 72 915,72 73 178,91

GDP % 4,34 4,48 4,65 4,66 4,51 4,35

source DARES, 2007

Economic challenges of policy evaluation:

Costs of implementation and costs of non implementation !


Evaluation criteria of programs and reforms:

Adequacy : well defined and well targeted

Efficiency: economically rational equitably founded

Efficacy : fulfills the objectives


Main difficulties:

The lack of the appropriate data: statistics (surveys and administrtive files), experimental data (voluntary or natural experiments)

The absence of the control group . (program participants versus non participants)

The complexity and interdependence of different segments of socio-economic and tax-benefit systems: sédimentation process and internal policy contradictions)

Unobserved hetérogénéité of individual situations and individual behaviours



Microsimulation methods

Origin and interest : Welfare state reforms in 1980ies (tax –benefit systems, pension systems, redistribution rules modifications). It was highly political issue generating the need for independent control and monitoring (equity, redistribution problems)

Opportunity: the development of micro econometric methods and microeconomic data bases

Micro - because based on the individual observations which allows the measure the program and reform impacts by modelling the individual behavior.

Simulation - because many variants of changes in socio-economic systems rules can be simultaneously or sequentially introduced generating numerous predicted outputs in terms of new behaviors or new system states

Social and fiscal policies simulation and micro econometric evaluation methods Microsimulation

Micro-simulation model structure

- Exogenoeus rule unit (tax benefit système: income, consumption taxes, tax, contributions, benefits, with founding flows)

- Individual data base (surveys, administrative data, with updating models, with transition probabilities, matching procedures…)

- Behavioural models : individual reactions as a consequence of the (new) rules’ application (labour supply , consumption, tax evasion, informal market participation…).

Social policy evaluation: concepts , methods and limits.Microsimulation

The rules can be deterministic (tax burden) or stochastique (demographic events like marriages,

births, divorces)

The effects are measured in global level terms (tax expenditure for example)distributional terms (distribution of income effects).

Frequently used indicators inequality (redistribution) measuresequity work incentive measures (marginal tax rates)


Data base

Database – the essential element for micro simulation. As exhaustive as possible on both individual characteristics and socio-economic environment description. The need for permanent updating (time and coverage).

This is somewhat utopist postulate. No survey or administrative file can provide all needed information. The lacking information of interest can be completed by indirect methods – matching or imputation (Family budgets survey with tax files for example or imputing demographic events’ probabilities.

Evaluation des politiques sociales et fiscales Modèles de microsimulation: un outil d’aide à la décision et d’évaluation ex ante

Microsimulation la base de données


Microsimulation

Exemple imputationModèle INES (INSEE)

ERF97 ERF97

BBBaaassseee cccaaasss---tttyyypppeeesss

génération des cas-types

A01

A02

A03

APE

AF

APJE

ASF

AAH

CAAH

MINV

AES

Bourses

API

RMI1

RMI2

Loyers imputés

AL

APL

ARENT

BASE

itérations

IRPP

CF

CAAH

API

RMI2

APL

ARENT

cotisations

Actualisation

PPE

Passage du foyer fiscal à la famille

CAF

EEEnnnqqquuuêêêttteee LLLooogggeeemmmeeennnttt


typology

Static models (no time dimension)

Typical household sets

Artificially built different socio-economic family structures (couple one child, median income…) to which the tax benefit system rules are applied.

Frustrating because non representative they can give a good idea of tax-benefit system interactions (effective marginal tax rate (EMTR) for example), and more generally threshold effects)

Simple static models (without behavioural adjustment)

The typical household set is replaced by an individual data base. The policy impacts are observed on representative sample of the total population. The policy changes effects are computed for every individual comparing the situation after and before the policy change. Typically the lost and gains in term of disposable incomes can be computed for every individual or for group of individuals (different types of households, income groups). The results is often given as a change in relative individual’s position in terms of well being or income distribution.

Social policy evaluation: concepts , methods and limits.MicrosimulationSocial policy evaluation: concepts , methods and limits.

Microsimulationtypology

Static models (no time dimension)

Static Models with behavioral responses:

The individual response behavioral model is added to the simple static model (labour force supply model, consumption behavior model). Then the obtained individual behavioural parameters (elasticities) are used to correct the results for the effect of individual adaptation to the new situation. Typically for the change in VAT taxation reform, a Consumer Demand System is estimated and all elasticities are derived.


typical household analysis exemple

(Tax reform effect on the marriage versus partnership union after PPE ( low wage workers allowance)

0

5 000

10 000

15 000

20 000

25 000

30 000

35 000

40 000

45 000

50 000

0 10 000 20 000 30 000 40 000 50 000 Salaires déclarés par Monsieur (en Euros)

Sala

ires d

écla

rés p

ar

Mad

am

e (

en

Eu

ros)

Double décote pour les concubins

Effet de la PPE

avantage aux couples mariés

équivalent

avantage aux couples en concubinage

couple without children,, S. Guérin INSEE, Et. Sociales


typology

Dynamic micro simulation models

Similar structure than those of static ones (typical households, models with behavioral response, models without behavioral response.

The time dimension is introduced with appropriate changes especially as far as population evolution. The data base individuals “are moving” in the time (get married, have children , divorce, get a job, become unemployed… by associaton of the estimated probabilities of all these events).

Simultanoeusly to these events the changing socio-fiscal rules are applied over the life cycle and ll outcomes are added.


typology

Dynamic typical household analysis typical households situation over the life cycle Typical households are multiplied in time dimensions making their structure vary in the with respect to their hypothetical life cycle events. (Madinier, Sahut d’Izarn, 1992).

Dynamic models with behavioral response, without “feedback”

first – econometric simulation of data base demographic evolution using the transition probabilities of state changing (birth, marriage, divorce, retirement, unemployment)

second –simulation of life cycle income evolution

third – the income effects different variants of evolution of the tax benefit system, and the labor market are estimated.

Dynamic models with behavioral response and with "feed-back”

Inter temporal choice individual model are added with possible response on the change in socio-economic environment.


INES model (INSEE)simulation examples

Scénario 1,2,3

Social security founding reforms : lowering work taxation by substituting health contribution (paid as proportion of wages) by a general, flat rate tax on all incomes.


MicrosimulationMicrosimulation Model (INSEE)

simulation examples



-8

-6

-4

-2

0

2

4

6

vari

ati

on

en

%

5 15 25 35 45 55 65 75 85 95 100 touscentiles du revenu disponibles en %

prélèvement cotisation csg impôt

SCENARIO N°1

Scénario 1

Suppression des cotisations sociales maladie financée par CSG :+ 4,5 points et augmentation de l’impôts par la non déductibilité de la CSG

Social policy evaluation: concepts , methods and limits.MicrosimulationINES model (INSEE)simulation examples

Tableau II-1-2 SCENARIO 1, variation en % rapportée au revenu déclaré initialdu prélèvement total selon la Cs du chef du ménage et le type de famille

personnes

homme femme famille couple tous

en % sans lien de famille

seul seule mono-parentale

sans enfant

avec 1 ou 2 enfants

avec 3 enfants ou plus

ménages

agriculteurs exploitants

0.2 1.5 0.9 -0.1 0.7 0.4 -0.1 0.4

artisans, commerçants et chefs d'entreprise

0.9 1.6 2.5 0.7 1.5 1.4 1.7 1.5

cadres supérieurs et professions libérales

0.3 0.7 0.5 1.0 0.5 0.2 0.3 0.4

professions intermédiaires

-0.6 -0.3 -0.3 -0.9 -0.7 -0.9 -1.2 -0.8

employés -0.7 -0.6 -0.7 -1.2 -0.8 -0.9 -1.4 -0.9

ouvriers -1.0 -1.3 -1.3 -1.9 -1.2 -1.8 -2.4 -1.7

retraités et inactifs 0.6 1.7 1.6 -0.2 1.6 0.7 -1.1 1.4

toutes CS 0.2 0.4 0.9 -0.7 0.8 -0.4 -0.7 0.0


Microsimulation Modèle INES (INSEE)

Exemples de simulations

GINI

de l’impôt sur le revenu

du revenu disponible

du revenu disponible par UC

1996 référence 0.77967 0.33917 0.29765

Scénario 1 0.76382 0.33605 0.29180



TABLEAU II-1-9

TAUX MOYEN D’IMPOSITION SELON LE CENTILE D’IMPOT

centiles d’impôt référence Scénario 1 Scénario 2 Scénario 3 5 0 0 0 0 10 0 0 0 0 20 0 0 1.2 0.5 30 0 0.4 2.8 1.5 40 1.4 2.0 4.0 2.5 50 3.3 3.9 5.3 3.3 60 5.0 5.7 6.7 4.6 70 6.9 7.6 8.6 6.1 80 9.0 9.7 11.1 7.9 90 12.8 13.6 15.4 11.8 95 17.9 18.9 21.3 17.0 99 25.3 26.6 28.3 23.19 100 33.3 34.1 34.6 29.0 Tous 7.9 8.5 10.0 7.4



Scenario 1-

- by income level

Advantageous for low income increase in taxes, but lower contributions, finally lower global tax burden

For 6-th-7th decile global tax burden is unchanged .

For _8th and higher deciles reform is disadvantageous. Direct tax increase is higher then the decrease in social contribution.

- by family type

Large families from lower social classe are beneficiaries of the reform.

Retired and selfemployed are loosing independently on their family situation

Middle class professions improve their situation proportionally to the family

Globaly: no change in inequality, but another shape of redistribution



TABLEAU II-1-10

NOMBRE DE MENAGES IMPOSES

référence Scénario 1 Scénario 2 Scénario 3 Nombre de ménages imposés en millions

13.51 14.17 17.85 17.23

proportion de ménages imposés en %

61.7 64.7 81.5 78.6



TABLEAU II-1-11

CONCENTRATION DE L’IMPOT : MASSE CUMULEE D’IMPOT PAR CENTILE D’IMPOT

centiles d’impôt référence Scénario 1 Scénario 2 Scénario 3 5 0 0 0 0 10 0 0 0 0 20 0 0 0 0 30 0 0 1.1 0.7 40 0.1 0.4 3.2 2.3 50 1.8 2.6 6.4 5 60 5.5 6.6 11 9 70 11.5 12.9 17.5 14.9 80 20.9 22.7 27.2 24.1 90 36.8 38.6 42.6 39.1 95 50.5 52.2 55.8 52.2 99 73.8 75.1 77.7 75.4 100 100 100 100 100



Conclusions are incomplete: the lack of essential response: what impact on the unemployment ? In order to answer that question behavioral the model of labor supply is necessary.

The behavioral response model is needed.

In the case of employment the situation is difficult: many problems to obtain coherent labor supply and labor demand elasticities.

Instead the example of consumption.



Behavioural component example : value added tax reform

Reform projects: « Social »VAT, lowering VAT as an incentive for legal employment in some sectors: hotel- restauration, construction.

A behavioural madel is needed to compute –income and price elasticities

Rise in VAT=rise in prices implying income and substitution effects.



Behavioural responses VAT reform

Théoretical model Linear Expenditure System (LES)

With very well known direct and indirect utility functions:

u q Log qi i ii

n

( ) . ( )

1

v y p y p pi ii

n

ji

nj( , ) ( . ) / ( )

11

Model estimated on grouped data from matched fiscal and consumption surveys

The demand system

.

p q p y pi i i i i k kk

n

( )

1

avec ii

n

1

1

Social polcy evaluation: concepts , methods and limits.Microsimulation






.

élasticités à la dépense totale avec le modèle I élasticités sur la

cellule moyenne moyenne minimum maximum écart-type

alim. à domicile (A1)

0.497 0.522 0.246 1.912 0.184

alcools & tabacs (A2)

0.693 0.938 0.170 31.937 1.441

alim. hors domicile (A3)

1.231 3.147 0.339 121.087 7.994

effets, soins personne (B1)

1.256 1.529 0.411 15.912 0.843

logement (C1) 0.775 0.784 0.344 1.569 0.207

chauffage & éclairage (C2)

0.462 0.486 0.162 1.474 0.186

équip. services dom. (D1)

0.263 0.370 0.072 6.721 0.307

santé, hygiène (E1)

1.099 1.378 0.205 12.091 0.851

auto, moto (F1) 1.343 2.698 0.526 328.605 11.365

autre transport (F2)

0.869 2.383 0.113 197.427 7.371

télécommunications (F3)

0.593 0.619 0.214 1.675 0.206

loisirs, culture (G1)

1.362 1.750 0.538 11.617 0.954

autres biens, services (H1)

1.571 4.932 0.247 199.331 12.333




.

Distribution des élasticités prix-propre non compensées avec le modèle I élasticités sur la

cellule moyenne moyenne minimum maximum écart-type

alim. à domicile (A1)

-0.398 -0.410 -1.667 -0.102 0.162

alcools & tabacs (A2)

-0.494 -0.649 -23.537 -0.050 0.986

alim. hors domicile (A3)

-0.869 -1.947 -65.879 -0.197 4.550

effets, soins personne (B1)

-0.890 -1.004 -8.290 -0.212 0.472

logement (C1) -0.620 -0.608 -1.332 -0.213 0.147

chauffage & éclairage (C2)

-0.336 -0.346 -1.191 -0.051 0.162

équip. services dom. (D1)

-0.183 -0.232 -3.961 -0.054 0.174

santé, hygiène (E1)

-0.787 -0.932 -6.214 -0.208 0.558

auto, moto (F1) -0.951 -1.524 -119.582 -0.368 4.272

autre transport (F2)

-0.614 -1.534 -128.098 -0.064 4.705

télécommunications (F3)

-0.421 -0.425 -1.523 -0.050 0.170

loisirs, culture (G1)

-0.959 -1.107 -5.575 -0.331 0.442

autres biens, services (H1)

-1.096 -2.868 -99.886 -0.159 6.253




.

Simulated reform: unification of several VAT levels 1%, 5.5% et 20.6 into one:15%

Hypothesis

- the change of VAT is entirely integrated into retail prices

-The total expenditure remain unchanged – only substitution effects, no change in saving behaviour

- tax revenue reamains constant.




.

Variation de la TVA suite à l’harmonisation des taux normaux et réduits, selon le type et le niveau de vie du ménage

en francs centiles de revenu disponible par uc

revenu disponible par uc

seule, autre couple sans enfant

couple 1 enfant

couple 2 enfants

couple 3 enfants

famille mono-parentale

tous

5 24213 159 267 493 653 1050 230 389 10 44120 214 323 497 615 1058 213 392 20 53297 176 429 281 442 812 198 364 30 63226 134 407 203 269 694 40 273 40 72897 52 400 55 94 585 -103 174 50 82393 26 326 63 25 412 -154 123 60 92986 -24 172 -27 -30 308 -314 25 70 106262 -156 79 -236 -147 192 -413 -102 80 123715 -245 5 -384 -264 -258 -333 -203 90 152387 -384 -206 -620 -456 -461 -266 -381 95 196847 -492 -412 -1172 -1236 -367 -493 -639

100 351461 -443 -544 -1134 -1336 -122 -579 -667 tous 105542 -53 88 -203 -79 505 -47 1




.

Effets comparés de l’harmonisation du taux normal et du taux réduit de TVA dans le cadre du modèle statique et du modèle L.E.S.

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

% r

ev

en

u d

isp

on

ible

1*2 3 4 5 6 7 8 9 10* 1*2 3 4 5 6 7 8 9 10* 1*2 3 4 5 6 7 8 9 10* 1*2 3 4 5 6 7 8 9 10*déciles de revenu disponible par uc

L.E.S. statique

harmonisation du taux normal et dutaux réduit de la tva

couple sansenfant

couple 1enfant

couple 2enfants

couple 3enfants & +

* le 1er vingtile et le dernier vingtile sont éliminés




.

Conclusion: the behavioural effect introduction did not change considerably the results despite of rather large scale of the reform.


DESTINIE model (INSEE)dynamic model

simulation examples

.

Destinie microsimulation model was built to analyse different scenarios in retirment systems.

Individual life trajectories are simulated 50 years ahead

Annual and individual base- every year the probabilities of change in individual demographic marriage, divorce, situaitions…and economic (employment, unemployment, wage evolution , retirment) are estimated.

(Remember in static models the population does not change!)

The fertility rate is supposed to influence the retirment schemes.

Data base- Family History survey and Census sample (250 000)



simulation examples

.



simulation examples

.

Essential estimated parameters:

Demographic transition probabilities

Labour market transition probabilities (employment survey)

Labour participation probabilities evolution (especially women)

Wage evolution model (with individual fixed effects, exogenous chocs and and productivity evolution

Wae dynamics in public secotor

Several specific surveys and administrative files have to be used to obtain these parameters.



simulation examples, public administration pension reform

.

2003 reform

General objectif – later retirment for cicvil servants

Longer working period shift from 37,5 to 40 years for full pension.

Bonus malus system for longer or shorter working period

Minimum guarantee rules changed.




.

Effects on the number of retired

Effects on the total civil pension expenditure




.


Ex ante evaluation final remarks

.

Very useful to learn about possible interactions effects of existing systems in Good prediction of reform consequences but on relatively high aggregation level. Not very adapted to regional studies, when specific longitudinal data not available or particular sub- population . Limits: available data implying the necessity of combining many different sources, updating difficult especially in changing Individual behaviour modeling limits – heterogeneity of individual situations,

Ex post policy and programs evaluation Methods and measures for program participation and outputs

Question: how to measure the program or reform impact? The « output » of a program is often defined in terms of “additionnality”:

Difference between the outcomes under the new program and the outcomes which would have occurred without the program.

Ex: tax credit for unemployed who would accept a low paid job:To what extent the decreasing unemployment the effect of the program, or the economic growth,

demographic evolution. Problem how distinguish between the two effects.

The main difficulty: estimate the outcomes which would have occurred without the program. This is called the counterfactual.

All ex post evaluation methods try to estimate this figure directly or indirectly, as an necessary element to obtain the additionnality.


More formally the program’s output can be written : (Rubin’s model)

Denoting Y the output variable T the program participation ( 0 for participation 1 for not participation)

Y1 Y0 - potential results - participation and not participation respectively. They are never observed simultaneously for the same individual. He cannot be in two states at the same time.

Y = T Y1 + (1-T)Y0

In the case of participation Y1, the Y0 represents the counterfactual

Impact or “additionnality”

I = Y1 - Y0


The problem of the counterfactual:

How to divide the eligible population into treatment ( or intervention group which participates in the program), and control (or comparison group which do not participate.

This is essentially the control group which is difficult to identify – it should be identical to treatment group except from program participation.

In reality, many individual (often unobservable) factors influence the decision to participate.

Ex: The monetary incentive is not the only condition to participate (others – qualification, opportunity costs (distance, domestic production, informal markets availability…)

Thus, choosing a control group for the counterfactual estimation is exposed to a high risk of selectivity bias

Different methods to minimize that risk


Different methods to minimise that risk

Randomised trials

if not possible

Quasi experimental methods

Before-after design

Difference in differences

One-to one matched comparison group design

Matched area comparison design

Statistical micro econometric modelin


Randomised trials

« Golden standard »

Eligible population is divided at random into two groups: a program (treatment) group and a control group .

(Like in medical research quasi single blind experiment but not double or « triple » blind) )

Both group are balanced as far as all charcteristcs which can influence the outcome are concerned, except for program participation.

Advantages:

The only observed differences are random differences and the program impact

Difficulties

Moral: the program is refused to the control group

Administrative: high administrative costs managing the selection between participants and non participants (two systems must be run)

Conclusion: very good design but difficult to implement. Very rich administrative data can facilitate the design (no specific survey costs).


Quasi experimental designs

Before-After design

Used to evaluate programs that are to be introduced nationally

The comparison group is drawn from the eligible population before the program is implemented,

The program group is drawn from the eligible population post- program implementation.

The main disadvantage of the before-after design is that change, or additionality, due to the program can not be separated from change that might naturally occur between any two points in time. Different factors can affect outcomes in different periods (change in labor market conditions)

Different other programs interference possibility.

Conclusion: the most often used, but the identification of particular programs’ effects can be difficult.

Can be improved, if a long time series is available.



Ex post micro-simulation

A variant of the previous method (before and after) is the use of the micro simulation model (Constant population structure analysis)

A set of output indicators is built (inequalities, poverty, unemployment, effective marginal and average tax rate…).

The introduced program are applied to the same population sample generating the hypothetical « after » outputs .

The treatment and control (comparison group) are the same and are compared before and after program application.

Problems: there is no control for population structure evolution impact and the influence of economic situation change



Matched area comparison design

First: selecting pilot areas to run a new program

Second : these areas are matched to a set of control areas (not necessarily on the one to one basis;

The eligible population is followed up in both areas and the outcomes compared

Problem: controlling for observable or not observable differences in areas

Advantage: no administrative problems when running different programs.



One to one matching methods (individuals, groups, areas)

Post program implementation design:

Individuals or more often groups are selected first among participants,

Then the « similar » ones are selected among non participants.

“Similar” means matching the closest possible observable characteristics of interest except from program participation.

This is one-to one matching i.e. to every participating group (individual) another if possible identical is associated from the non participating population.

Main problems: the weaknesses of matching methods especially when many unobservable characteristics influencing program participation or few common characteristics between observations.

.



Différence in différences

Two groups are compared before and after program implementation

Two groups are selected from eligible population :

participants (intervention group)

non participants in the program (control or comparison group)

Both groups are observed over the time and outcomes of the variable of interest (for example unemployment) are calculated as differences « before » and « after ».

For « non participants « natural change” is observed,

For participants for intervention group both « natural and program impact change are observed”.

The second difference between change in the treatment and control group evolution gives the estimate of program impact effect.

main hypothesis:« natural evolution is identical for both groups.


Statistical modeling to estimate the counterfactual

It is a variant of one-to one matching schemes associating an appropriate control group (non-participant in the program) to the treatment group (participants in the program).

If matching is difficult (usually it is) it is interesting to get the non participant group much larger then participants’ group allowing a more precise reliable results when estimating the counterfactual from a a larger data set taking into account a large number of potential candidate for match.

Several matches can be realized among non participants to correspond to one participant observation.

Ex post policy and programs evaluation Methods and measures for program participat

Statistical matching models and estimatorsi

Matching on observable characteristics

The estimation principle:

Use all information available on non all non participants to build for every participant a counterfactuel.

1.Matching estimator on observable characteristics (Rubin 1977).

2.The potential outcome for all non participants is estimated as a prediction based on the same characteristics and real outcomes of the variable of interest (unemployment) for program participants.

3.The set of identical characteristics between participants and non participants can be difficult or even impossible to identify Then it can be replaced the closest possible individuals in sense of a defined distance mesure (Mahalanobis for exemple).

4.The program result is as usual the average difference of outcomes between participant et non participants



Propensity score matching estimation


Probability of treatment (participation) is estimated conditionally on the observed individual characteristics.

Then the participants and non participants are matched on the basis of the propensity score proximity.

Propensity score with kernel weighting

The basic idea is that every non treated individual is participating in building of the counterfactual of an treated individual with the weight varying with respect to the propensity score distance between both individuals



The instrumental variable estimator


First find a variable correlated with participation (treatment) but not correlated with variables (observed and non observed) related to outcomes Comparing outcomes with this variable gives the information how outcomes relate to participation and allows additionnality estimation.

Problem: difficult to find such a variable; Propensity score is often used.

Heckman selection estimator

Allows the correlation of the instrument with outcome equation errors by explicit estimation of both (instrument and errors). Specification of that relationships depends however on strong hypothesis.

Ex post policy and programs evaluation Final remarks

What are the recommended evaluation schemes

1.Both ex ante and ex post policy evaluations bring an information about programs –reforms perspectives in terms of expected results.

2.The reliability of this information depend on the use of the appropriate models but essentially on the quality of existing or created data sets

3.Ex ante methods treat the general population effects, but need the development in the sense of macro-economic general equilibrium

4.Ex post methods are adapted to treat rather small populations and suffer also from the lack of the general population or macro-economic feedback

social policy evaluation: concepts, methods and limits. titre

Documents

social policies

expected effects evaluation

social vat

micro econometric methods

redistribution policies

ngos slide

titre slide

qualitative methods