Raj K Pandey

PROJECT EVALUATION

1. Definition• 2 aspects 1) Value Judgement of evaluated subject

- A process of judgement of achievement of evaluated subject- A judgement of outcome from activity to achieve a certain target- value determination of something

Raj K Pandey

2) Information for decision - data collection and interpretation of data- A process of determination, procurement, presentation of information that useful for decision making - Information gathering and use that

information for decision making of projects, i.e., output, process of project.

Raj K Pandey

2. Evaluation Research- Base on research design- Data collection, analyze and process of data- Not aim to look for new theory - Aiming to test the application of new

knowledge in management and action- Evaluation is a process of information

formulation to judge something that leads to a decision making as systematically and with criteria.

Raj K Pandey

- Evaluation is composed of :1) Measurement 2) Comparison

- Measurement is a process of determining status and amount - Comparision is a process of using the results from measurement to judge according to standard criteria.- Compare between before and after project.- Judgement between actual and plan

Raj K Pandey

4 . Evaluation Process

Measurement Comparison

Description Decision

= Evaluation+

Raj K Pandey

Purpose of Evaluation

- Identify objective and use of information

- Users of evaluation results, evaluation team, planners, implementation team, committee, executive, public, lawyer

- The purpose of evaluation is to improve the project management and implementation

Raj K Pandey

Good Evaluation must have the following information:

- How much project objective is achieved.

- Cost effective

- Effects of the projectWhat should be decided for the project

Raj K Pandey

6 Functions of Evaluation 1) Information function

- Feedback to improve project 2) Professional function

- Increase understanding of project objectives and process- Success, failure, strength, weakness and recommendation

3) Organizational Function - Analysis of goal, purpose, strength,

weakness of organization structure

Raj K Pandey

4) Political Function- Encourage the participation and benefit between planners and project managers

5) Socio-Psychological Function- build up the security of stakeholders

6) Historical Function - Record direction and lesson learn for future project implementation

Raj K Pandey

Scope and Coverage of Evaluation1) Internal/External

- Internal : by project staffs- External : increase the confidence of objectivity in

evaluation process

Raj K Pandey

2) Formation/Summative- Ongoing evaluation : information to improve process - Post-evaluation : aims to sum thing up to look at the achievement of project objectives

* Impact* Cost-effectiveness

* Outcomes3) Based on the unit of analysis

- participants- project- impact on community

Raj K Pandey

3 Phases of evaluation1) pre-implementation phase - emphasizes on project design and development - questions to be addresse

* target and objectives* resources planing * planning on activities and time frame* required personnel * report writing

Raj K Pandey

2) During the programme implementation phase- emphasizes on fact finding and effectiveness of project under various conditions- trend analysis

* process evaluation * experimental study

- questions:* before fully implementation * on-going implementation

Raj K Pandey

11. Before fully implementation- project verification V.S. sample of target group - questions:

1) operation is according to the plan2) quality of output is as expected3) problems, recommendation for improvement

12. On-going evaluation- check the output/outcome as plan- questions :

1) any changes according to objectives2) should continue or terminate

3) any improvement required

Raj K Pandey

3) At the programme phase

- evaluate outcome, impact and outputs of project- summative evaluation/cost-benefit analysis- questions:

1) innovation and dissemination 2) operation in the broad sense

* reasonal benefit/cost* should terminate, improve, correct or continue

* sustainable?

Raj K Pandey

11 steps in evaluation planning1) review project details2) set up objectives and indicators

SMARTS = SpecificM = MeasurableA = AchievableR = Result Oriented

T = Time bound

Raj K Pandey

3) Develop evaluation schedule, when and whom data is collected

4) Sampling – for large project5) Type of data – look at objectives 6) Select and innovate instrument for data

collection7) Data collection plan 8) Data analysis plan – look at objectives9) Cost estimation10) Report and discloses information schedule11) Scope of evaluation

Raj K Pandey

Scope of evaluation depends on the purpose of evaluation

- To improve process or system 1) Emphasize on input and process

- To look at outcomes and effects1) Accomplishment and outcomes

Raj K Pandey

Full Project Evaluation is composed of 5 aspects:

- Context

- Input

- Process

- Product

- Effect/Impact

- Which aspect depends on time frame of

evaluation

Raj K Pandey

Pre implementation evaluation

- For project planning : compose of1. Context evaluation

- economic, social, politic- problem and community’s needs- target group

2. Input evaluation- check adequacy of input and quality to be used in project operation- appropriate technology- a feasibility of work plan, financial plan and staffs

Raj K Pandey

Formatives/Ongoing Evaluation- Evaluate process- Is the operation progress?- Is input appropriate?- Does activity contribute to the achievement of objective?- How much is the outcome/effects?- What is the strength and weakness?- What is the obstacle?- The results will lead to the activity, process improvement

Raj K Pandey

Post/Summative Evaluation- Look at product after project completion

* whether objective is achievable, as well as target

* whether the project is success according to the plan

- Evaluate after a certain period of project completion

* Impact evaluation* Output – outcomes* Both positive and negative impact

- Evaluation models : CIPP, CIPOO, CIPPI

Raj K Pandey

Evaluation Components : 2 aspects1. Contents of evaluation

- scope or components - objectives- questions- indicators

2. Methodology of evaluation- evaluation design- source of data- data collection, mean of verification, tools- data analysis

- evaluation criteria

Raj K Pandey

Essential components of evaluation design

- purpose of evaluation- users- objectivity- time bound- resource available

Raj K Pandey

Evaluation of policy, plan, project- 6 aspects

1) Effectiveness : want to know- policy, plan or project is success base on the objectives - output, outcomes are according to objective, target and expected results both quantitative and qualitative

2) Efficiency : want to know- How much is resource used in implementing the policy, plan or project?- What and how much are the effects?

Raj K Pandey

Efficiency of policy, plan or project is 2 folds 1) Technological efficiency

- Least cost to produce outputs or outcomes

2) Economic efficiency- The relationship between cost and benefit- Look at the increase of outcome

per unit of investment- Decrease the cost per unit of

outcome- Look at the effort/input and output

- Measure in financial term

Raj K Pandey

3) Sufficiency- policy, plan, project meet the

demand of beneficiaries at the level of

problem alleviation

4) Equality- who are the potential

beneficiaries of the project

5) Justice- The project is determine for the

people in the society

6) Appropriateness - Whether the activities, outputs

are complied to the need of

beneficiaries- Appropriate to area and time of project

implementation