M.Phil. Education Minhaj University Lahore

Date: 28 Jan, 2012 Supervisor:

Ijaz Ahmad Tatla

Asma Anis AlviAyesha sheikh

Monitoring & Evaluation Techniques

Governance

Objectives of Presentation

Introduction of Monitoring and EvaluationDifference between M & EUnderstanding of Techniques of Monitoring and Evaluation

Definiti

on of M

onitorin

g

It carries with it very hierarchical, authoritative and evaluative-com-directional connotations. Sarfraz Khawaja(2001)

Definitio

n of Eva

luation The systematic

collection and interpretation of evidence, leading to a judgment of value intended to produce action”.

Worthen and Sanders (1996)

Characteristics Evaluation Monitoring

Subject: Usually focused on strategic aspects addresses operational management issues

Character: incidental, flexible subject & methods continuous, regular, systematic

Primary client: stakeholders and external program management

Approach:

 

objectivity, transparency utility

Methodology: demanding research methodologies, refined tools

rapid appraisal methods

 

Primary focus: focus on relevancy, outcomes, impact and sustainability

focus on operational efficiency and effectiveness

Objectives: to check outcomes / impact, verify developmental hypothesis

to identify and resolve implementation problems

  to document successes and lessons learned

to assess progress towards objectives

MONITORING & EVALUATION – COMPARATIVE CHARACTERISTICS

Techniques of Monitoring

William Gantt was an engineer who, during World War 1, developed a method to keep track of major activities, depict the time and sequence of activities in order to monitor a project .

Construct with the placement of vertical and horizontal axis on a page. The various functions and tasks to be completed in a project are listed in sequence from first to last, along the axis. A timeline with measured raises is recorded along horizontal axis with beginning to end. Symbols can be added to the Gantt chart that represent report documents and specified activities.

Gantt Chart

Progra

m E

valu

atio

n

and R

evie

w

tech

nique

(PER

T)

The ProgramEvaluation and Review Technique (PERT) was introduced by space engineers (NASA) and originally use to help manage large projects in which literally thousands of tasks to perform. Desmond L. Cook took the concept and applied in education.

PERT is a network. The PERT network is used in graphic presentation to depict the activities to be accomplished in a project and to show the sequence and relationship of activities. Time estimates for each activity are included in the PERT.

Program Evaluation and Review technique (PERT)

EventsStage or check point in the plan. It defines the start or finish of one or more activities.

ActivitiesIt represented by arrows joining two events following the direction of the gap of time. It is symbol of work in hand of the consumption/labor.

NetworkA network is a diagrammatic representation of a project having a logical sequence of various activities which together makeup project.

Time estimateMinimum possible period of the activity is time estimate.

Elements of PERT

Event is a point in time. e.g.

Start Normal

Event Key or

milestone event

Stop

PERT

There may be many paths in a PERT network; the path that requires the greatest amount of time for completion is the Critical path.

PERT Network

CPM is the project network analysis technique for determining the minimum project duration.

useful approach to develop realistic schedules based on knowledge of resources

Main application of this technique is, planning, controlling and construction programs for the large infrastructural and industrial projects.

Critical Path Method (CPM)

Techniques of Evaluation

Response sheets Interviews Committees and Panels

Techniques of Evaluation

Observation

What ?How?Whom? Participants/non-participants. Adversary evaluation

To filter out personal views and prejudices. Goal-Free evaluation

Unexpected results with observation. Product evaluation

Its part of learning and traditional culture to conduct activity/ceremony. Social-impact methods

Gather data to seek social impact after need assessment before the program.

Participatory Approaches

A favorite method in Western nations is the response sheet handed out at the end of course. Having a yes/no response makes the analysis easy but scaled responses, based on Likert type options, produces more useful data.

Response sheets

Interviews can replace the written words, as participants pass on verbally their views and they are recorded, either in writing at that time by the interviewer or on audio tape for later analysis.

Interviews

Committees are one useful technique for the process of evaluation. It is elected internally or externally based on experts focus on comments, suggestions, and criticism. A committee is drawn from participants in a program whereas a panel is drawn from outsiders. The panel can provide information, or as it may be appropriately termed in these circumstances ‘feedback’, to both the learners, instructors and program managers.

Committees and Panels

It was originally developed by personnel at the

Rand Corporation in 1960s. The Delphi was originally designed to provide the predictions of experts. The Delphi method technique is planning and management technique of collecting and utilizing specific information from several experts as a basis for decision-making or planning. Involvement in the Delphi process is time-consuming.

The Delphi Method

Step One: Experts are involved and selected.

Two: Topics and issues are asked from experts/panel with time frame and best judgment.

Three: The survey containing the original topics for consideration and time predictions to the participants to measure “effectiveness and probability” with seven-point scale.

Process of Delphi

Four: The responses are complied, summarized, synthesized and feedback.

Five: Third questionnaire, the experts are asked to review their position of group of experts, to read the comments of other experts/opinion leaders, and reevaluate their own position.

Six: Received responses are tabulated thus becomes the source of information for constructing opinion.

Final: Results are analyzed and report is prepared.

Process….

It has been used in Russia and USA as mean of determining futures based on expert opinion. Both positive and negative features have been perceived to accompany such techniques as the Delphi process, as well as, such as brain storming techniques. This approach developed in an effort to minimize negative and highlight positive traits.

Qualitative Controlled Feedback (QCF)

Group members are not allowed to have face to face interaction.

Group members are not permitted to know the identities of participants regarding any issue or question publicized.

The responses or survey is presented in frequency and percentage. Questions are focused on “what” and “why”.

Continue

Administrators use models and simulations to “test” plans or ideas in many ways. A simulation may be applied to immediate planning efforts, or a simulation may become part of strategic planning efforts.

The scenario is a means of communication potential options for planners and decision makers to consider. Futurists use scenarios to depict their perceptions of what the future will look like.

Simulations and Scenarios

A method for determining what abilities are needed to do a particular job. The standards of success are established through actual incidents going on the job. Flangan used it to develop charts of the personal and social developments of elementary school children. Films or tapes are used to study cause and possible solutions of problem encountering in achieving success. (e.g.)

Critical Incident Method

PPBS is in effect an addition of a techniques in planning and budgeting process for identifying, costing and assigning a complexity of resources for establishing priorities and strategies in a major program and for forecasting costs, expenditure and achievements within the immediate financial year or over a longer period.

Planning, Programing, Budgeting System (PPBS)

Combining two or more courses of action when it is considered more effective rather than pursuing them individually.

 Synergy