part one first edition food security assessment facilitators

FOOD SECURITY ASSESSMENTFACILITATORS’ TOOL KIT

LEARNING EVENT DESIGN AND PLANNING GUIDE

PART ONE l FIRST EDITION

FOOD SECURITY ASSESSMENTFACILITATORS’ TOOL KIT

LEARNING EVENT DESIGN AND PLANNING GUIDE

VOLUME II l FIRST EDITION

ACKNOWLEDGEMENTS

The Facilitators’ Tool Kit was made possible by the generosity of the European Union, including itssupport of a series of food security assessment (FSA) workshops, development of distance learningmodules and implementation of on-the-job learning to provide a blended approach to enhancingassessment skills, knowledge and abilities.

The authors would like to thank everyone who provided comments and inputs to the tool kit: GeertBeekhuis, Andrea Berardo, Anne Callanan, Agnès Dhur, Mamadou Diouf, Simon Dradri, Steffenie Fries,Patrick Foley, Yvonne Forsen, Mark Gordon, Sheila Grudem, Nick Haan, Peter Horjus, Naouar Labidi,Margarita Lovon, Sylvie Montembault, Asif Niazi, Leah Richardson, Scott Ronchini, Issa Sanogo, JaneShaw, Nicole Steyer and Samir Wanmali.

Acronyms Used in this Document page 9

1 Introduction page 11

1.1 Purpose of the Tool Kit page 121.2 Structure of the Tool Kit page 121.3 Food Security Assessment Course Overview page 13

2 Defining Course Content page 15

2.1 Learning Objectives page 162.2 Learning Paths page 18

3 Training Needs Analysis page 35

3.1 Country Context page 363.2 Methods Review page 373.3 Stakeholder Analysis page 373.4 Skills Gap Analysis page 39

4 Session Adaptation Guidelines page 41

4.1 Adapting Content page 424.2 Adapting Exercises page 444.3 Adapting Case Studies page 444.4 Adapting the Agenda page 45

5 Pre-Workshop Preparation page 47

5.1 Facilitators and Resource People page 485.2 Administrative Arrangements page 485.3 Venue Concerns page 505.4 Arrival in the Country page 51

6 Workshop Implementation Logistics page 53

6.1 At the Start page 546.2 During the Workshop page 54

7 Running the Workshop page 55

7.1 Ensuring a Participatory Approach page 567.2 Workshop Methods page 57

8 Workshop Follow-up page 59

8.1 Workshop Test Results page 608.2 Learning Programme Final Report page 608.3 Certificates page 60

9 Supervised On-the-Job Learning page 63

9.1 On-the-Job Learning Activities page 649.2 Management and Implementation of Supervised On-the-Job Learning Events page 649.3 Duration page 659.4 Learning Objectives page 65

C O N T E N T S

10 M&E Framework for FSA Learning Events page 67

10.1 M&E Framework for FSA Workshops page 6910.2 M&E Framework for On-the-Job Learning page 7310.3 Evaluating Outcomes through the Annual Online Survey page 7410.4 Evaluating Impact through Cases Studies page 76

11 Regional Food Security Learning Strategies and Action Plans page 77

11.1 Responsibilities for Post-Learning Event Follow-up page 78

Tables, Figures and Boxes

Tables

Table 1 FAO’s Performance and Content Matrix page 17Table 2 Bloom Taxonomy - Cognitive Domain page 17Table 3 Categorization of Learning Objectives page 18Table 4 Bloom Taxonomy Cognitive Domain and FSA Learning Events page 18Table 5 Beginners, Basic and Intermediate FSA Learning Paths page 22Table 6 Thematic/ Technical Learning Paths page 27Table 7 Performance Expectations of Advanced Assessors page 33Table 8 Learner Data page 39Table 9 Schedule for Pre-Workshop Preparation page 49Table 10 Bloom Taxonomy Psychomotor Domain page 66Table 11 FSA Learning Strategy M&E Framework page 68Table 12 Workshop Evaluation Components and Possible Combinations page 70Table 13 Selecting a Combination of Components page 70Table 14 WFP Responsibilities for Post-Learning Event Follow-up page 80

Figures

Figure 1 Learning Paths page 19Figure 2 Required Skills and Experience Levels for Assessments page 20Figure 3 Sample Layout of a workshop venue page 50

Boxes

Box 1 Workshop Certificates – Best Practice page 61Box 2 Automated Data Entry and Analysis using QuestionMark Software page 71Box 3 Fitting the Learning Objectives to the Audience page 72Box 4 Learner Follow-up – Best Practice page 79

Annexes included on the CD-ROM and Intranet

Annex A: Food Security Facilitation Tools

A1 Curriculum Outline: Courses, Modules, Sessions and Learning ObjectivesA2 Workshop Session Slides, Exercises and Facilitation NotesA3 Procedures for Delivering and Reporting on Knowledge Assessment Tests (KATs)A4 Short Papers on Adult Learning and Instructional Design

Annex B: FSA Learning Events Data Collection and Reporting Tools

B1 Workshop Report Short FormatB2 Workshop Report Detailed FormatB3 Supervised On-the-Job Learning Pre and Post Learning Evaluation ToolsB4 FSA Learning Event M&E Data Collection ToolsB5 Terms of Reference Workshop Lead FacilitatorB6 Knowledge Assessment Test Formats

Annex C: Sample Learning Implementation and Planning Tools

C1 Sample Pre-workshop Self Assessment QuestionnaireC2 Sample Time Table for Detailed Training Needs Analysis and WorkshopC3 Ethiopia Country Level Capacity AnalysisC4 Sample JAM and FSA Participants WorkbookC5 Sample Workshop ReportsC6 Sample Supervised On-the-Job Learning Pre-Activity Evaluation

C O N T E N T S

ACRONYMS USED IN THIS DOCUMENT

CAP Consolidated Appeals ProcessCFSAM crop and food supply assessment missionCFSVA comprehensive food security vulnerability analysis DSA daily subsistence allowanceENA emergency needs assessmentFAO Food and Agriculture Organization of the United NationsFSA food security assessmentIDP internally displaced personJAM joint assessment missionKAT knowledge assessment testM&E monitoring and evaluationNAF Needs Analysis Framework NGO non-governmental organizationODA Assessment, Analysis and Preparedness Division ODAN Emergency Needs Assessment BranchODAV Vulnerability Analysis and Mapping BranchODK Regional Bureau Kampala (East and Central Africa)PACE Performance and Competency Enhancement ProgrammePPT PowerPointPRA participatory rural appraisalRAO Regional Assessment Officer RRA rapid rural appraisal SENAC Strengthening Emergency Needs Assessment ProjectSENAIP Strengthening Emergency Needs Assessment Implementation PlanSPSS Statistical Package for Social SciencesTOR Terms of Reference

9

F O O D S E C U R I T Y A S S E S S M E N T F A C I L I T A T O R S ’ T O O L K I T

INTR

OD

UC

TIO

N 1

12

1 . I N T R O D U C T I O N

WFP has developed the Food Security Assessment Learning Programme to improve the ability of WFPand partner staff to plan and implement sound food security assessments (FSAs). The learningprogramme is based on the Emergency Food Security Assessment Handbook and methodologicaldevelopments from the “Strengthen Emergency Needs Assessment Capacity Project” (SENAC) , whichspecify technical standards and provide guidelines on tools and methods derived from consultationsamong food-security practitioners. The approach to instruction and the contents are based on anassessment of learning needs that involved visits to country offices and regional bureaus, evaluation ofprevious assessment training, a review of practices and ongoing consultations with food securitypractitioners, facilitators and learners.

The programme was rolled out to the field through regional workshops and distance learning in 2005;in 2006, it was expanded to country workshops to address specific skills for WFP staff. The SupervisedOn-the-Job Learning Programme was also launched in 2006, and preparation of the Interactive DistanceLearning Programme was started.

A core assumption is that learning is an iterative process; each learning event or assessment helps toenhance individual capacity. The workshop element of the programme is one step in the learningprocess; continued hands-on and technical training - both through experience and in the classroom - isalso needed to reach the required standards.

1.1 PURPOSE OF THE TOOL KIT

The tool kit aims to assist facilitators and trainers with the design and delivery of FSA learning events.

It provides guidance, rather than rigid instructions, and users should adapt its contents to the specificneeds of their audience.

WFP will expand the tool kit to include training on crop and food supply assessment missions (CFSAMs)and other types of assessments.

1.2 STRUCTURE OF THE TOOL KIT

This toolkit provides suggestion on learning event design and planning, Session Guides (i.e. “trainer’snotes) for each workshop session, learning data collection and reporting tools and sample learningimplementation and planning tools.

The Learning Event Planning and Design Guide is made up of the following components:

• Learning Paths, Performance Expectations, and Learning Objectives;• Training Needs Analysis (Methods Review, Stakeholder Analysis and Gap Analysis);• Developing Workshop Agenda and Adapting Sessions;• Logistical Planning and Implementing the Workshops;• Evaluating Learning Events;• Supervised on-the-Job Learning;• Regional Food Security Learning Strategies and Actions

13


1.3 FOOD SECURITY ASSESSMENT COURSE OVERVIEW

The course is designed to prepare WFP and partner agency staff to undertake FSAs in response toexisting or anticipated crises. The learning programme includes workshops, distance learning andsupervised on-the-job learning.

The Facilitators’ Tool Kit includes the following modules and sessions for use in workshops:

• FSA overview and principles;• analysing food security;• establishing assessment objectives and plans;• designing an assessment;• data collection, processing and analysis;• drawing conclusions;• programming and reporting.

For each module and session in the tool kit, a session guide provides:

• a “session at a glance” table of the main activities;• learning objectives and participant prerequisites;• guidance on running the session;• facilitator’s notes, including the content of slides to be presented during the session; • exercises, with instructions on implementation;• suggested background reading;• equipment and supplies needed for the session.


DEF

ININ

G

CO

URS

E C

ON

TEN

T 2

16

2 . D E F I N I N G C O U R S E C O N T E N T

1. Question Mark Corporation, Instructional Design Process, 20012. Ibid3. Ghirardini, Beatrice and Andrew Nadeau. Instructional Design Methodologies: Information Management Resource Kit Presentation. FAO WAICENT Capacity

Building and Outreach Branch, Proc. Of ODAN Capacity Building Strategy Meeting, January 2006. 4. Information Management Resource Kit, Instructions for Authors Module on Community Building and Electronic Networking, January 2004.

2.1 LEARNING OBJECTIVES

Every learning activity must have realistic learning objectives. These are “statements that define whatthe learner must do to demonstrate learning has been accomplished.”1

Learning objectives should cover three major areas:2

• performance - actions that the learner should be capable of carrying out;• conditions - under which the performance should occur;• criteria - standards of performance that the learner is expected to achieve.

2.1.1 Identifying Learning Objectives

Learning objectives are identified through a rapid training needs analysis, based on responses to thefollowing questions:

• What is the learner’s role in the FSA process?• What skills, knowledge and abilities are required to perform this function?• Does the learner have sufficient of these skills, knowledge and abilities to perform the function

properly? What concerns do the learner and his/her supervisor have regarding the learner’s capacityto carry out FSAs?

• What gaps are there in the learner’s skills, knowledge and abilities? Can these be addressed throughworkshops, on-the-job learning or other activities?

Chapter 3 contains more details on conducting training needs analysis.

When planning supervised on-the-job learning, the learner and facilitator should also review thelearner’s profile, emergency needs assessment (ENA) questionnaire, learning assessments and worksamples. The learner’s supervisor can provide additional capacity-building priorities for the learner.

Identifying the learning objectives for workshops requires stakeholder analysis, review of methods andunderstanding of the country context. For each workshop, the facilitators and country office staffshould:

• identify relevant modules from the Facilitators’ Tool Kit; • consider whether each module’s learning objectives reflect the needs and capacities of the

workshop participants;• revise the learning objectives, when necessary.

Facilitators should base their revised learning objectives on participants’ existing knowledge, experience,needs and gaps, and the time available for the workshop.

2.1.2 Using Learning Objectives in Course Design

The performance and content matrix3 developed by the Food and Agriculture Organization of theUnited Nations (FAO) is a tool for ensuring that learning objectives, exercises and knowledge assessmenttests (KATs) are coherent. It divides performance into three categories - application, understanding andmemorization - and content into four: facts, concepts, procedures/processes, and principles/rules.4

All these categories are then presented in one matrix, as shown in Table 1.

175. Op. cit., footnote 36. Benjamin Bloom, S. Taxonomy of Educational Objectives. Ally and Beacon, Boston MA.Pearson. Eduction. 19847. Op. cit., footnote 28. Clark, Donald (1999, June 5). Learning Domains or Bloom’s Taxonomy [online]. Avalible: http://www.nwlink.com/donclark/hrd/bloom.html


Table 1. FAO’s Performance and Content Matrix5

Another tool, the Bloom taxonomy6 (Table 2), identifies six categories of intellectual activity: knowledge,comprehension, application, analysis, synthesis and evaluation.7 Key words help facilitators to identifythe depth of learning and place each learning objective into the appropriate category. The taxonomyalso provides examples of KATs and exercises that can be used in course design.

Table 2. Bloom Taxonomy Cognitive Domain8

APPLICATION

UNDERSTANDING

MEMORIZATION

PER

FOR

MA

NC

E

CONTENT

FACTS

Repeat the fact

CONCEPTS

Establish whetheror not the object ispart of the class

Reformulate thedefinition

List the characteristics of the concept

PROCEDURES/PROCESSES

Perform the procedure/applythe process to a new situation

Reformulate theprocedure/process

List the steps of theprocedure/process

PRINCIPLES/RULES

Solve the problem byapplying theprinciples

Reformulate theprinciples

List the principles

Knowledge:Recall of information.

Comprehension:Understanding andinterpretation of instructionsand problems. Rephrasing ofinstructions and problems.

Application:Use of concepts in new situations.Application of classroom learningto work situations.

Analysis: Separation of material orconcepts into components, toexamine their organizationalstructure. Distinction betweenfacts and inferences.

Synthesis:Creation of structures or patternsfrom diverse elements. Assemblyof parts to form a whole,particularly a new meaning orstructure. Use of facts andexperience to present arguments.

Evaluation:Judgement of the value of ideasor materials.

CATEGORY EXAMPLES OF KATS/EXERCISES, AND KEY WORDS

Examples: Recite a policy. Quote prices from memory. Know safety rules.Key words: define, describe, identify, know, label, list, match, name, outline, recall,recognize, reproduce, select, state.

Examples: Rewrite the principles of test design. Identify the steps in a complex task.Translate an equation on to a computer spreadsheet.Key words: comprehend, convert, defend, distinguish, estimate, explain, extend,generalize, give, infer, interpret, paraphrase, predict, rewrite, summarize, translate.

Examples: Use the manual to calculate an employee’s vacation time. Apply principlesof statistics to evaluate a written test.Key words: apply, change, compute, construct, demonstrate, discover, manipulate,modify, operate, predict, prepare, produce, relate, show, solve, use.

Examples: Use logical deduction to troubleshoot a piece of equipment. Recognizelogical fallacies. Identify a department’s training needs.Key words: analyse, break down, compare, contrast, deconstruct, differentiate,discriminate, distinguish, identify, illustrate, infer, outline, relate, represent, select,separate.

Examples: Write a manual for an operation or process. Design a machine for a specifictask. Integrate learning from several sources to solve a problem. Revise a process toimprove its outcome.Key words: categorize, combine, compile, compose, create, devise, design, explain,generate, modify, organize, plan, rearrange, reconstruct, relate, reorganize, revise,rewrite, summarize, tell, write.

Examples: Select the most effective solution. Select the most qualified candidate for ajob. Explain and justify a new budget.Key words: appraise, compare, conclude, contrast, criticize, critique, defend, describe,discriminate, evaluate, explain, interpret, justify, relate, summarize, support.

Table 3 categorizes two sample learning objectives according to both the performance and contentmatrix and the Bloom taxonomy.

Table 3. Categorization of Learning Objectives

As mention earlier in this text, the bloom taxonomy helps facilitators to identify the depth of learningand appropriately define learning objectives. The table below provides suggested targets for the depthof learning facilitators and learners can expect from each type of learning event.

Table 4. Bloom Taxonomy -Cognitive Domain- and FSA Learning Events

2.2 LEARNING PATHS

Learning events have to address the roles that learners are expected to have in an assessment, takinginto account their previous experience and technical skills, the methods they are familiar with, thecountry context, and staff development priorities.

Memorization

Application

Define the purpose and scope of an FSA

Classify the range of immediate and long-term response options according to type ofproblem

PERFORMANCE

Concepts

Procedures

Knowledge

Application

CONTENT

BLOOM

TAXONOMY

CATEGORY

LEARNING OBJECTIVE

PERFORMANCE AND CONTENT MATRIX

DistanceLearning

CountryOffice orsub-regionalWorkshop

Thematic/TechnicalWorkshops

SupervisedOn the-Job-Learning

DistanceCoaching

ActiveParticipationin anAssessments

√√ √√

√√ √

√

√√

√√

√√

√√

√√

√√ √√

√√

√

KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS

LEARNINGEVENTS

BLOOM TAXONOMY CATEGORY

18


The Food Security Assessment Learning Programme has three levels: beginners, basic and intermediate.The Assessment Analysis and Preparedness Division (ODA) has developed a learning path for each ofthese levels. These paths should be used during preparation of a learning event, together withstakeholder analysis and skills gap analysis. By focusing on learners’ performance expectations - theskills, knowledge and abilities they require to fulfil their FSA roles successfully - the learning paths helpfacilitators identify skill gaps and design appropriate combinations of workshops, on-the-job learningand distance learning. Learning paths can be adapted to meet each learners’ needs.

The learning paths are based on the assumption that country offices will need: (i) staff with skills inimmediate and medium-term FSAs; and (ii) strategies for building the abilities of staff and partners.Workshops, on-the-job learning and distance learning should therefore be tailored to individual countryrequirements.

Figure 1. Learning Paths9

Most learners will not need to meet all of the performance expectations listed in the learning paths, buttraining in specific categories will improve their performance. In skills gap analysis, performance targetsshould be used to clarify learning objectives and select learning activities and content.

Learning paths and performance expectations can also be used after a learning event to determineappropriate follow-up activities.

2.2.1 Matching the Performance Expectations to the FSA Category

The first edition of the Emergency Food Security Assessment Handbook provides three categories ofassessment: initial investigation, rapid assessment, and in-depth assessment. These categories aresubdivided according to the scale and complexity of the crisis. The technical skills and experiencerequired to conduct an assessment often depend on the category of the crisis. The learning pathstherefore link performance expectations to the type of assessment to be conducted. Figure 2 shows thelevels of learning path for the different assessment categories.

INDIVIDUAL OR HOMOGENOUS GROUP LEARNING PATH

Defining learners’ profiles allows the facilitation team to create learning paths, appropriate to the skill andexperience of participants and to define learning objectives and clear course attendance criteria.

DEFINING CONTENT OF AN FSA COURSE

Module ModuleModuleModuleModuleModule

Session SessionSessionSessionSessionSession




199. Diagram is based on a FAO Instructional Design Presentation and revised to meet the requirement of this publication. Ghirardini, Beatrice and Andrew Nadeau.

Instructional Design Methodologies: Information Management Resource Kit Presentation. FAO WAICENT Capacity Building and Outreach Branch, Proc.Of ODAN Capacity Building Strategy Meeting, January 2006.


Source: FAOWAICENT

Figure 2. Required Skills and Experience Levels for Assessments

Rapid assessments can be either simple or complex:

• Simple rapid assessments are at the local level, such as for a small-scale internally displaced person(IDP) or refugee crisis, or a recurrent natural disaster.

• Complex rapid or in-depth assessments are at the national level, such as for major ongoingemergency situations.

2.2.2 Course Levels and Targets

Learners will have diverse functions in FSAs, so facilitators combine different learning paths within thesame course. Tables 5, 6 and 7 refer primarily to FSAs, but they can be adapted to other types ofassessments - such as food security monitoring, joint assessment missions (JAMs) and CFSAMs.

Before using these tables to develop a regional learning strategy, for each type of assessment to becarried out it is essential to identify: (i) the composition of the assessment team, (ii) the functions/rolesof each team member, (iii) the skills required to perform each function/role and (iv) the functions/rolesand skills that are common to all assessments in the region.

Table 5 is based on the performance targets and content of Food security and analysis beginning, basicand intermediate courses. Table 6 includes details on technical courses, with additional functions andskills. Table 7 lists the performance expectations of advanced assessors.

20


Beginning

Basic

Intermediate

Advanced

Co

mp

lexi

ty a

nd

mag

nit

ud

e o

f th

e cr

isis

an

d E

FSA

Skills, Experience and Abilities

Complex / nationalAssessment

(Team Leader & Lead inthe design, analysis &

reporting)

Simple / LocalAssessment

(partecipate ininformation

collection & planning)Limited Extensive

2.2.3 Course Candidate Selection Criteria

Regional Bureau senior managers and technical staff should advocate for the application of well-definedcriteria to select candidates for the learning events. The following criteria can be adapted to specificlearning events:

(i) Sufficient prior background knowledge and experience

• basic understanding of, and experience in, of designing and implementing food securityinterventions.

• meets skills, knowledge and ability requirements of the previous course level prior to attendingthe subsequent higher level learning event. (i.e. beginning, basic and intermediate level EFSA orbasic and intermediate level sampling and data analysis);

• good mathematical ability and acceptable command of excel;• identified by Regional Technical staff and/or coach (on-the-job learning) for continued

assessment capacity development (candidates for thematic/specialized technical learningevents);

(ii) Availability and need for practical application

• confirmed by supervisor to have a specified, active role in the planned assessment/exercise;• assessment and analysis activities are included as part of routine responsibilities;• opportunity to apply the skills to be addressed in the learning event under supervision;• need to develop specific skills to be addressed in the learning event;• time available for continued learning;• approval of his/her supervisor(s) to dedicate time to learning events and follow-up – on-the-job

learning, participating in assessments, etc;• participation in country and TDY assessments and FSA learning events have been included in

work plans (PACE).

21


22


Fun

ctio

n/r

ole

in

an a

sses

smen

to

r re

late

dac

tivi

ty

Perf

orm

ance

exp

ecta

tio

ns

for

each

fun

ctio

n/r

ole

BEG

INN

ERS

BA

SIC

INTE

RM

EDIA

TE

•Pa

rtic

ipat

e in

an

initi

al in

vest

igat

ion

•C

ontr

ibut

e to

logi

stic

al p

lann

ing

and

info

rmat

ion

colle

ctio

n fo

r a

sim

ple

rapi

das

sess

men

t•

Con

trib

ute

to a

res

pons

e op

tions

ana

lysi

s

Lear

ner

s sh

ou

ld b

e ab

le t

o:

•pa

rtic

ipat

e in

ana

lysi

s, p

lann

ing,

man

agem

ent

and

repo

rtin

g fo

r an

initi

alas

sess

men

t;•

colle

ct a

nd p

roce

ss q

ualit

ativ

e da

ta;

•pl

an lo

gist

ics

- tim

ing,

tra

nspo

rt,

adm

inis

trat

ion;

•pa

rtic

ipat

e in

con

sulta

tions

with

exp

erts

to

eval

uate

res

pons

e op

tions

bas

ed o

n th

efo

od s

ecur

ity s

ituat

iona

l ana

lysi

s.

•Le

ad a

n in

itial

inve

stig

atio

n•

Part

icip

ate

in t

he m

anag

emen

t of

a s

impl

era

pid

asse

ssm

ent

•Pa

rtic

ipat

e in

a s

impl

e ra

pid

asse

ssm

ent

•Su

bsta

ntia

lly c

ontr

ibut

e to

a r

espo

nse

optio

ns a

naly

sis

In a

dd

itio

n t

o t

he

beg

inn

ers

cap

acit

ies,

lear

ner

s sh

ou

ld a

lso

be

able

to

:•

lead

the

pla

nnin

g, d

ata

anal

ysis,

des

ign,

man

agem

ent

and

repo

rtin

g fo

r an

initi

alas

sess

men

t at

the

ons

et o

f sm

all-s

cale

cris

is;

•pa

rtic

ipat

e in

the

man

agem

ent

of a

sim

ple

rapi

d as

sess

men

t -

prep

are

Term

s of

Ref

eren

ce(T

OR)

, det

erm

ine

staf

fing

need

s, c

olla

bora

tew

ith p

artn

ers;

•

lead

the

logi

stic

al p

lann

ing

for

a sim

ple

rapi

das

sess

men

t -

timin

g, t

rans

port

and

adm

inist

ratio

n;

•su

perv

ise e

num

erat

ors/

inte

rvie

wer

s in

a s

impl

era

pid

asse

ssm

ent;

•pa

rtic

ipat

e in

the

pre

para

tion

of s

ectio

ns o

fth

e fo

od s

ecur

ity r

epor

t fo

r a

simpl

e ra

pid

asse

ssm

ent;

•ca

rry

out

qual

ity c

ontr

ol o

f an

initi

al o

r sim

ple

rapi

d as

sess

men

t re

port

; •

iden

tify

and

eval

uate

res

pons

e op

tions

, bas

edon

the

FSA

situ

atio

nal a

naly

sis.

Non

-pro

babi

lity

sam

plin

g:•

part

icip

ate

in t

he d

esig

n of

a s

impl

e ra

pid

asse

ssm

ent,

usin

g an

ana

lytic

al p

lan,

sam

plin

gst

rate

gy a

nd d

ata

colle

ctio

n in

stru

men

ts;

•pa

rtic

ipat

e in

dat

a an

alys

is an

d pr

esen

tatio

n of

resu

lts in

the

FSA

rep

ort

of a

sim

ple

rapi

das

sess

men

t, us

ing

focu

s gr

oup

disc

ussio

ns a

ndke

y in

form

ant

inte

rvie

ws.

•Le

ad a

sim

ple

rapi

d as

sess

men

t•

Part

icip

ate

in a

com

plex

rap

id a

sses

smen

tor

an

in-d

epth

ass

essm

ent

•Ev

alua

te t

he q

ualit

y of

initi

al a

nd r

apid

asse

ssm

ent

repo

rts

•C

ondu

ct a

res

pons

e op

tions

ana

lysi

s

In a

dd

itio

n t

o t

he

bas

ic c

apac

itie

s,le

arn

ers

sho

uld

als

o b

e ab

le t

o:

•le

ad t

he p

lann

ing,

dat

a an

alys

is, d

esig

n,m

anag

emen

t an

d re

port

ing

for

an in

itial

asse

ssm

ent

at t

he o

nset

of

a sm

all-

or la

rge-

scal

e cr

isis;

•m

anag

e a

simpl

e ra

pid

asse

ssm

ent;

•pl

an lo

gist

ics

for

and

man

age

an in

-dep

th F

SAfo

r a

maj

or o

ngoi

ng e

mer

genc

y sit

uatio

n,in

clud

ing

a co

mpl

ex r

apid

ass

essm

ent;

•pa

rtic

ipat

e in

the

des

ign

of a

n in

-dep

th F

SAfo

r a

maj

or s

udde

n or

ong

oing

em

erge

ncy

situa

tion,

incl

udin

g a

com

plex

rap

idas

sess

men

t.

Non

-pro

babi

lity

sam

plin

g:•

part

icip

ate

in o

r le

ad d

ata

anal

ysis

for

a sim

ple

rapi

d as

sess

men

t.

Prob

abili

ty s

ampl

ing:

•w

ith o

vers

ight

, par

ticip

ate

in t

he p

repa

ratio

nof

a s

ampl

ing

plan

;•

with

ove

rsig

ht, p

artic

ipat

e in

dat

a en

try,

bas

icda

ta c

lean

ing

and

the

prod

uctio

n of

bas

icde

scrip

tive

outp

uts;

•w

ith o

vers

ight

, par

ticip

ate

in a

naly

sis o

fva

riabl

es a

nd a

sim

ple

corr

elat

ion

test

;•

part

icip

ate

in t

he in

terp

reta

tion

of o

utpu

tta

bles

, pre

sent

res

ults

in t

he F

SA r

epor

t fo

r a

simpl

e ra

pid

asse

ssm

ent;

Tab

le 5

.Be

ginn

ers,

Bas

ic a

nd In

term

edia

te F

SA L

earn

ing

Path

s

Lead

= w

ork

inde

pend

ently

. Pa

rtic

ipat

e =

wor

k w

ith s

uper

visi

on.

23


Sug

ges

ted

lear

nin

g p

ath

s

BEG

INN

ERS

BA

SIC

INTE

RM

EDIA

TE

•D

ista

nce

lear

ning

and

/or

FSA

tec

hniq

ues

wor

ksho

p10

•In

tegr

atio

n of

FSA

ski

lls in

to d

aily

wor

k •

Part

icip

atio

n in

loca

l ini

tial i

nves

tigat

ions

and

rapi

d as

sess

men

ts

Prob

abili

ty s

ampl

ing:

•pa

rtic

ipat

e in

the

des

ign

(ana

lytic

al p

lan,

data

col

lect

ion

inst

rum

ents

) of

a s

impl

era

pid

asse

ssm

ent,

usi

ng h

ouse

hold

ques

tionn

aire

s;•

Und

erst

and

sam

plin

g co

ncep

ts,

met

hodo

logi

es a

nd im

plic

atio

ns o

n su

rvey

desi

gn,

anal

ysis

and

rep

ortin

g.•

Lead

sur

vey

team

to

sele

ct h

ouse

hold

fro

ma

clus

ter;

•pa

rtic

ipat

e in

dat

a en

try

and

proc

essi

ng o

fda

ta t

able

s us

ing

the

Stat

istic

al P

acka

ge f

orSo

cial

Sci

ence

s (S

PSS)

- id

entif

y si

mpl

e da

tacl

eani

ng n

eeds

, pr

oduc

e m

eans

and

perc

enta

ges.

•D

ista

nce

lear

ning

and

/or

FSA

tec

hniq

ues

wor

ksho

p11

•Ba

sic-

leve

l lea

rnin

g ev

ents

/wor

ksho

ps in

core

ass

essm

ent

skill

s -

asse

ssm

ent

desi

gn,

asse

ssm

ent

man

agem

ent,

dat

a an

alys

is(q

uant

itativ

e an

d/or

qua

litat

ive)

, re

port

writ

ing,

res

pons

e op

tions

ana

lysi

s, e

tc.

•Pa

rtic

ipat

ion

in a

sim

ple

rapi

d as

sess

men

t,w

ith f

eedb

ack

on q

ualit

y of

out

puts

•

Whe

re p

ossi

ble,

inte

grat

ion

of s

kills

into

prog

ram

me

activ

ities

- p

roje

ct m

onito

ring,

bene

ficia

ry t

arge

ting,

etc

. •

Whe

re p

ossi

ble,

lead

ersh

ip -

with

sup

port

-of

the

initi

al a

sses

smen

t fo

r a

sudd

en-o

nset

emer

genc

y•

App

licat

ion

of s

kills

in a

n as

sess

men

t,fo

llow

ed b

y on

-the

-job

lear

ning

, di

stan

cele

arni

ng,

tech

nica

l bac

ksto

ppin

g an

d/or

addi

tiona

l wor

ksho

ps f

ocus

ing

on id

entif

ied

skill

gap

s

•pa

rtic

ipat

e in

the

des

ign

of d

ata

colle

ctio

nto

ols

for

quan

titat

ive

and

qual

itativ

e sim

ple

rapi

d as

sess

men

ts.

•D

ista

nce

lear

ning

and

/or

regi

onal

or

coun

try

offic

e w

orks

hop12

•Ba

sic-

or

inte

rmed

iate

-leve

l lea

rnin

gev

ents

/wor

ksho

ps in

cor

e as

sess

men

t sk

ills

-as

sess

men

t de

sign

, qu

alita

tive

and/

orqu

antit

ativ

e da

ta a

naly

sis,

mar

kets

,re

spon

se o

ptio

ns a

naly

sis,

etc

. •

On-

the-

job

lear

ning

and

dis

tanc

e le

arni

ngto

rei

nfor

ce s

kills

•Le

ader

ship

, w

ith s

uppo

rt,

of a

sim

ple

rapi

das

sess

men

t, w

ith f

eedb

ack

on q

ualit

y of

outp

uts,

incl

udin

g pr

epar

atio

n of

the

rep

ort

or s

ectio

ns o

f it

•C

ontin

ued

lead

ersh

ip o

f an

initi

alin

vest

igat

ion,

inte

grat

ion

of c

ore

skill

s in

tore

gula

r pr

ogra

mm

e pl

anni

ng,

impl

emen

tatio

n an

d m

onito

ring

activ

ities

•

Succ

essf

ul a

pplic

atio

n of

ski

lls,

follo

wed

by

addi

tiona

l lea

rnin

g fo

cusi

ng o

n id

entif

ied

skill

gap

s

10.

FSA

tec

hniq

ues

incl

ude

food

sec

urity

con

cept

s, f

ram

ewor

ks a

nd s

umm

ary

of k

ey a

sses

smen

t ta

sks

and

outp

uts.

11

. Ib

id.

12.

Op.

cit.

Foo

t N

ote

10.

24


Skill

an

dkn

ow

led

ge

targ

ets

BEG

INN

ERS

BA

SIC

INTE

RM

EDIA

TE

Ass

ist

the

plan

ning

and

impl

emen

tatio

n of

an

initi

al a

sses

smen

t of

cha

nges

in c

omm

unity

food

sec

urity

con

ditio

ns

•C

olle

ct a

nd r

evie

w s

econ

dary

dat

a•

Iden

tify

shoc

ks/e

mer

genc

ies

and

pote

ntia

lim

pact

s•

Col

lect

prim

ary

data

usi

ng p

artic

ipat

ory

rura

l app

rais

al (

PRA

) an

d ra

pid

rura

lap

prai

sal (

RRA

) te

chni

ques

•Su

mm

ariz

e co

nclu

sion

s fr

om f

ield

vis

its•

Eval

uate

the

nee

d fo

r a

follo

w-u

pas

sess

men

t, a

nd w

hy

Ass

ist

the

man

agem

ent

of a

rap

id F

SA

•A

sses

s ad

min

istr

atio

n an

d lo

gist

ics

requ

irem

ents

- t

rave

l, m

ater

ials

, tim

ing,

data

col

lect

ion

•D

isse

min

ate

findi

ngs

to s

take

hold

ers

•Su

gges

t re

spon

ses,

bas

ed o

n th

e th

ree

pilla

rs o

f fo

od s

ecur

ity

Ass

ist

the

plan

ning

and

des

ign

of a

sim

ple

rapi

d FS

A

In a

dditi

on t

o th

e be

ginn

ers

skill

and

know

ledg

e ta

rget

s:

•D

efin

e a

wor

k pl

an a

nd o

bjec

tives

•

Con

trib

ute

to T

OR

prep

arat

ion

(def

ine

obje

ctiv

es,

cont

ext

and

cons

trai

nt a

ndou

tput

s)

•Es

tabl

ish

and

mai

ntai

n pa

rtne

rshi

ps

•C

ontr

ibut

e to

iden

tifyi

ng in

dica

tors

and

anal

ytic

al m

etho

d(s)

, fo

r bo

th q

ualit

ativ

ean

d qu

antit

ativ

e da

ta

Non

-pro

babi

lity

sam

plin

g:

•C

ontr

ibut

e to

a s

impl

e sa

mpl

ing

plan

,in

clud

ing

crite

ria

•D

evel

op s

emi-s

truc

ture

d in

terv

iew

and

focu

s gr

oup

disc

ussi

on g

uide

s fo

r an

FSA

usin

g ra

pid

appr

aisa

l tec

hniq

ues

Plan

and

impl

emen

t an

initi

al a

sses

smen

t of

cha

nges

in c

omm

unity

foo

d se

curit

y co

nditi

ons

In a

dditi

on t

o th

e be

ginn

ers

skill

and

kno

wle

dge

targ

ets:

•D

evel

op a

pre

limin

ary

wor

king

sce

nario

•D

evel

op a

sses

smen

t ob

ject

ives

and

que

stio

ns

•Pr

epar

e TO

R fo

r fo

llow

-up

rapi

d FS

A

Lead

or

cont

ribut

e to

the

pla

nnin

g an

d de

sign

of a

sim

ple

rapi

d FS

A

In a

dditi

on t

o th

e ba

sic

skill

and

kno

wle

dge

targ

ets:

Non

-pro

babi

lity

sam

plin

g:

•Id

entif

y in

dica

tors

and

ana

lytic

al m

etho

d(s)

•

Dev

elop

dat

a co

llect

ion

tool

s an

d a

sam

plin

g pl

an,

incl

udin

g cr

iteria

for

purp

osiv

e sa

mpl

ing

Prob

abili

ty s

ampl

ing:

•A

ssum

e gr

eate

r re

spon

sibi

lity

for

asse

ssm

ent

desi

gn,

incl

udin

g th

eid

entif

icat

ion

of in

dica

tors

•Ev

alua

te p

opul

atio

n da

ta a

nd u

pdat

e if

nece

ssar

y, w

ith s

uppo

rt•

Cre

ate

a sa

mpl

ing

plan

- s

impl

e,sy

stem

atic

, cl

uste

r, st

ratif

ied

or,

with

supp

ort,

com

plex

•

Cal

cula

te s

ampl

e si

ze,

with

sup

port

•C

onsi

der

impl

icat

ions

of

sam

plin

g on

anal

ysis

of

(i) c

lust

er s

ampl

e ef

fect

s, (

ii)st

ratif

icat

ion

and

(iii)

wei

ghtin

g

25


BEG

INN

ERS

BA

SIC

INTE

RM

EDIA

TE

Ass

ist

the

impl

emen

tatio

n of

a s

impl

e ra

pid

FSA

•

Col

lect

dat

a in

the

fie

ld

Prob

abili

ty s

ampl

ing:

•U

nder

stan

d sa

mpl

ing

met

hodo

logi

es a

ndde

sign

s •

Part

icip

ate

in t

he c

olle

ctio

n of

rel

iabl

epo

pula

tion

data

•

Part

icip

ate

in in

form

atio

n ga

ther

ing

and

the

crea

tion

of a

sim

ple

sam

plin

g pl

an

•U

nder

stan

d ac

cura

cy a

nd p

reci

sion

•U

nder

stan

d w

hat

mak

es a

sur

vey

repr

esen

tatio

nal,

and

how

to

eval

uate

surv

ey r

esul

ts•

Und

erst

and

the

basi

cs o

f co

nfid

ence

inte

rval

s•

Part

icip

ate

in t

he s

elec

tion

of h

ouse

hold

sfr

om w

ithin

clu

ster

s

Ass

ist

the

impl

emen

tatio

n of

a s

impl

e ra

pid

FSA

In a

dditi

on t

o th

e be

ginn

ers

skill

and

know

ledg

e ta

rget

s:

•Tr

ain

enum

erat

ors/

inte

rvie

wer

s •

Man

age

a te

am o

fen

umer

ator

s/in

terv

iew

ers,

incl

udin

g re

view

and

qual

ity c

ontr

ol•

Con

trib

ute

to a

n as

sess

men

t re

port

Resp

onse

opt

ions

, pa

rtic

ipat

e in

:

•ev

alua

tion

of t

he n

eed

for

a re

spon

se,

and

clas

sific

atio

n an

d ev

alua

tion

of r

espo

nses

acco

rdin

g to

typ

e of

pro

blem

•SW

OT

anal

ysis

to

iden

tify

‘pro

s’ a

nd ‘

cons

’of

var

ious

res

pons

e op

tions

in t

he m

ost

likel

y sc

enar

io•

diss

emin

atio

n of

fin

ding

s to

sta

keho

lder

s

Lead

the

impl

emen

tatio

n of

a s

impl

e ra

pid

FSA

In a

dditi

on t

o th

e ba

sic

skill

and

kno

wle

dge

targ

ets:

•Pa

rtic

ipat

e in

sta

keho

lder

iden

tific

atio

n•

Part

icip

ate

in c

onsu

ltatio

ns w

ithst

akeh

olde

rs t

o de

sign

a f

ram

ewor

kad

apte

d fo

r th

e cr

isis

/cou

ntry

/are

a•

Prep

are

an a

sses

smen

t re

port

Resp

onse

opt

ions

:

•C

arry

out

the

bas

ic t

asks

with

less

or

nosu

ppor

t

Prob

abili

ty s

ampl

ing:

•D

esig

n im

prov

ed q

uest

ionn

aire

s•

Und

erst

and

impl

icat

ions

of

sam

plin

g on

data

ana

lysi

s•

Cle

an d

ata

•Ex

port

dat

a to

SPS

S

26

2 . D E F I N I N G C O U R S E C O N T E N TB

EGIN

NER

SB

ASI

CIN

TER

MED

IATE

Non

-pro

babi

lity

sam

plin

g:

•Pr

oces

s qu

alita

tive

data

usi

ng s

impl

e m

atrix

form

ats

•A

naly

se q

ualit

ativ

e da

ta -

iden

tify

patt

erns

and

rela

tions

hips

, su

mm

ariz

e re

sults

Prob

abili

ty s

ampl

ing:

•En

ter

surv

ey d

ata

into

com

pute

r us

ing

Exce

lor

Acc

ess

•Pa

rtic

ipat

e in

dat

a cl

eani

ng•

Part

icip

ate

in t

he e

xpor

ting

data

to

SPSS

•Pa

rtic

ipat

e in

sim

ple

data

tra

nsfo

rmat

ion

•Pa

rtic

ipat

e in

the

pro

duct

ion

of m

eans

and

perc

enta

ges,

usi

ng S

PSS

Part

icip

ate

in t

he m

anag

emen

t of

an

in-d

epth

FSA

:

•de

finiti

on o

f ob

ject

ives

•pr

epar

atio

n of

TO

R•

colla

bora

tion

with

exp

erts

in a

sses

smen

tpl

anni

ng a

nd d

esig

n •

sche

dulin

g re

port

ing

•C

arry

out

sim

ple

data

tra

nsfo

rmat

ion

•U

se S

PSS

to p

rodu

ce m

eans

and

perc

enta

ges

•Pa

rtic

ipat

e in

dat

a m

anag

emen

t•

Prod

uce

and

read

des

crip

tive

outp

uts

Lead

the

pla

nnin

g an

d m

anag

emen

t of

an

in-

dept

h FS

A

In a

dditi

on t

o th

e ba

sic

skill

and

kno

wle

dge

targ

ets:

Sche

dule

rep

ortin

g•

Dis

sem

inat

e fin

ding

s an

d re

port

s to

stak

ehol

ders

27


Part

icip

ant

crit

eria

/pre

req

ui

site

s

Skill

/kn

ow

led

ge

targ

ets

and

per

form

ance

exp

ecta

tio

ns

Co

nte

nt

LEA

RN

ING

PA

THSA

MPL

ING

•A

ccep

tabl

e co

mm

and

of E

xcel

: A

cces

s an

d SP

SS a

lso

usef

ul•

Trai

ned

in b

asic

qua

ntita

tive

data

ana

lysi

s •

Goo

d m

athe

mat

ical

abi

lity

•Ba

ckgr

ound

in s

tatis

tics

•Ex

perie

nce

of q

ualit

ativ

e da

ta c

olle

ctio

n te

chni

ques

•

Nee

d to

impr

ove/

deve

lop

sam

plin

g ab

ilitie

s•

Tim

e av

aila

ble

for

lear

ning

•

Ava

ilabi

lity

for

follo

w-u

p le

arni

ng -

on-

the-

job,

inte

rmed

iate

, ad

vanc

ed,

etc.

•A

ppro

val o

f su

perv

isor

(s)

to d

edic

ate

time

to le

arni

ng e

vent

s an

d fo

llow

-up

•U

nder

stan

d sa

mpl

ing

met

hodo

logi

es -

sim

ple,

sys

tem

atic

, cl

uste

r, st

ratif

ied

and

com

plex

- a

nd d

ata

need

ed t

o ca

lcul

ate

sam

ple

size

:co

nfid

ence

inte

rval

s, p

opul

atio

n si

ze,

sam

plin

g m

etho

dolo

gy,

prec

isio

n, e

tc.:

ÝC

olle

ct in

form

atio

n fo

r sa

mpl

ing

plan

s, in

clud

ing

sam

ple

size

Ý

Obt

ain

relia

ble

popu

latio

n da

ta

•U

nder

stan

d ac

cura

cy,

prec

isio

n an

d re

pres

enta

tion

ÝIn

terp

ret

conf

iden

ce in

terv

als

ÝU

nder

stan

d w

hat

a re

pres

enta

tive

surv

ey is

•Ev

alua

te s

urve

y re

sults

•U

nder

stan

d im

plic

atio

ns o

f su

rvey

des

ign

on a

naly

sis

•U

nder

stan

d sa

mpl

ing

stra

ta,

and

impl

icat

ions

on

repo

rtin

g•

Link

sam

plin

g to

dat

a co

llect

ion

met

hods

•

Dat

a co

llect

ion

tech

niqu

es f

or r

apid

app

rais

als:

(i) s

ampl

ing

crite

ria;

(ii)

whe

n to

use

con

veni

ence

sam

plin

g; a

nd (

iii)

appl

ying

res

ults

from

sam

ples

to

wid

er p

opul

atio

ns

•Le

ad s

urve

y te

am t

o se

lect

hou

seho

lds

from

a c

lust

erÝ

Cla

rify

repl

acem

ent

issu

esÝ

Cla

rify

who

to

incl

ude

in t

he s

ampl

e Ý

Sele

ct h

ouse

hold

s in

eac

h cl

uste

r -

for

segm

enta

tion,

sys

tem

atic

ran

dom

sam

plin

g, e

tc.

•Ba

sic

sam

plin

g th

eory

, de

finiti

ons

and

conc

epts

•Si

mpl

e ra

ndom

sam

plin

g, s

yste

mat

ic r

ando

m s

ampl

ing,

clu

ster

sam

plin

g, s

trat

ified

sam

plin

g, c

ompl

ex s

ampl

e de

sign

•

Cal

cula

tion

of s

ampl

e si

ze•

Sam

plin

g is

sues

to

be c

onsi

dere

d du

ring

anal

ysis

:Ý

Ava

ilabi

lity

and

relia

bilit

y of

dat

a Ý

Repr

esen

tatio

nÝ

Con

fiden

ce in

terv

als

ÝW

eigh

ting

Tab

le 6

.Th

emat

ic/T

echn

ical

Lea

rnin

g Pa

ths

BASIC SAMPLING

28


On

-th

e-jo

ble

arn

ing

Part

icip

ant

crit

eria

/pre

req

ui

site

s

Skill

/kn

ow

led

ge

targ

ets

and

per

form

ance

exp

ecta

tio

ns

Co

nte

nt

LEA

RN

ING

PA

THSA

MPL

ING

•C

ase

stud

ies

and

scen

ario

s•

Revi

ew o

f ex

istin

g sa

mpl

ing

plan

s an

d su

rvey

rep

orts

•A

ltern

ativ

e sa

mpl

ing

met

hodo

logi

es f

or R

RA:

ÝPu

rpos

ive

sam

plin

g Ý

Con

veni

ence

sam

plin

g Ý

”Rul

es o

f th

umb”

- t

riang

ulat

ion,

opt

imal

igno

ranc

e, m

inim

izat

ion

of b

ias,

etc

.•

May

ove

rlap

with

inte

rmed

iate

qua

ntita

tive

trai

ning

•C

riter

ia,

inpu

ts,

outp

uts

and

supe

rvis

ors

mus

t be

cle

arly

def

ined

•Th

is is

a k

ey s

tep

in p

rodu

cing

peo

ple

who

can

hel

p w

ith b

asic

dat

a an

alys

is

•C

ompl

etio

n of

bas

ic t

rain

ing

and

prio

r/co

ncur

rent

inte

rmed

iate

qua

ntita

tive

anal

ysis

, or

equ

ival

ent

as d

eter

min

ed b

ysu

perv

isor

s/fa

cilit

ator

s•

Expe

rienc

e of

app

lyin

g sk

ills

from

bas

ic t

rain

ing

in o

wn

wor

kpla

ce

•Id

entif

icat

ion

by a

naly

sis

supe

rvis

ors

for

cont

inue

d le

arni

ng

•G

ood

com

man

d an

d de

mon

stra

ted

rece

nt u

se o

f Ex

cel a

nd S

PSS:

Acc

ess

also

use

ful

•N

eed,

tim

e, a

vaila

bilit

y an

d ap

prov

al f

or le

arni

ng,

as f

or b

asic

crit

eria

•W

ith s

uppo

rt,

eval

uate

pop

ulat

ion

data

and

upd

ate

whe

n ne

cess

ary

•W

ith s

uppo

rt,

crea

te s

ampl

ing

plan

s -

sim

ple,

sys

tem

atic

, cl

uste

r, st

ratif

ied

and

com

plex

•W

ith s

uppo

rt,

calc

ulat

e sa

mpl

e si

zes

•W

ith s

uppo

rt,

acco

unt

for

impl

icat

ions

of

sam

plin

g du

ring

anal

ysis

: Ý

Clu

ster

sam

ple

effe

cts

ÝSt

ratif

icat

ion,

bot

h be

fore

and

aft

er a

sur

vey

ÝW

eigh

ting:

•C

alcu

late

and

app

ly w

eigh

ts•

Dem

onst

rate

impr

oved

und

erst

andi

ng o

f co

nfid

ence

inte

rval

s, s

tand

ard

devi

atio

n, e

tc.

in s

ampl

ing

•In

-dep

th r

evie

w o

f sa

mpl

ing

met

hodo

logi

es -

pla

ns,

sam

ple

size

s, e

tc.

•In

-dep

th e

xerc

ises

on

deve

lopi

ng s

ampl

ing

plan

s -

hous

ehol

d su

rvey

s an

d RR

A t

echn

ique

s•

Cal

cula

tion

of w

eigh

ting

varia

bles

•A

ccou

ntin

g fo

r im

plic

atio

ns o

f sa

mpl

ing

durin

g an

alys

is:

ÝPr

actic

al a

naly

sis

exer

cise

s, u

sing

SPS

S:•

Wei

ghtin

g•

Clu

ster

sam

ple

effe

cts,

cal

cula

ting

conf

iden

ce in

terv

al•

Stra

tific

atio

n

INTERMEDIATESAMPLING

29


Part

icip

ant

crit

eria

/pre

req

ui

site

s

Skill

/kn

ow

led

ge

targ

ets

and

per

form

ance

exp

ecta

tio

ns

Co

nte

nt

LEA

RN

ING

PA

THD

ATA

AN

ALY

SIS

•A

ccep

tabl

e co

mm

and

of E

xcel

, SP

SS,

Acc

ess

or o

ther

pro

gram

bas

ed o

n da

ta m

anag

emen

t ta

bles

•G

ood

mat

hem

atic

al a

bilit

y •

Back

grou

nd in

sta

tistic

s •

Nee

d to

impr

ove/

deve

lop

data

ana

lysi

s ab

ilitie

s•

Opp

ortu

nity

to

appl

y ne

w s

kills

, un

der

supe

rvis

ion

•Ti

me

avai

labl

e fo

r co

ntin

ued

lear

ning

•A

vaila

bilit

y fo

r fo

llow

-up

lear

ning

eve

nts

- on

-the

-job,

inte

rmed

iate

, ad

vanc

ed,

etc.

•A

ppro

val o

f su

perv

isor

(s)

to d

edic

ate

time

to le

arni

ng e

vent

s an

d fo

llow

-up

- on

-the

-job

lear

ning

, pa

rtic

ipat

ion

in a

sses

smen

ts,

repo

rtin

g, e

tc.

•O

pen

files

in S

PSS:

und

erst

and

basi

c la

yout

of

data

- v

aria

ble

view

, da

ta v

iew

, ca

ses,

var

iabl

es•

Impo

rt s

impl

e da

ta t

able

s in

to S

PSS,

with

sup

port

•

Add

labe

ls a

nd v

aria

bles

to

files

, w

ith s

uppo

rt

•Id

entif

y:

ÝSi

mpl

e da

ta c

lean

ing

need

sÝ

Out

liers

ÝM

isco

ding

sÝ

Mis

sing

dat

a

•Pe

rfor

m b

asic

dat

a tr

ansf

orm

atio

ns,

with

sup

port

•

Poin

t-an

d-cl

ick

to p

rodu

ce m

eans

and

per

cent

ages

- f

requ

enci

es,

cros

s ta

bs,

desc

riptiv

es a

nd m

eans

com

paris

onÝ

NO

TE: a

t th

e ba

sic le

vel,

part

icip

ants

’ pro

duct

ion

of o

utpu

t ta

bles

, dat

a tr

ansf

orm

atio

ns, e

tc. m

ust

be s

uper

vise

d by

an

expe

rienc

ed a

naly

st

•Ba

sic

read

ing

of o

utpu

t ta

bles

, w

ith s

uppo

rt

•Ty

pes

of s

oft

data

ÝEx

cel

ÝA

cces

sÝ

Oth

er p

rogr

ams

•Ba

sic

over

view

of

SPSS

•D

ata

impo

rtat

ion

into

SPS

S•

Dat

a la

belli

ng:

ÝVa

riabl

e na

mes

and

labe

lsÝ

Varia

ble

valu

es

•Ba

sic

iden

tific

atio

n of

dirt

y da

ta:

ÝM

issi

ng d

ata

ÝM

isla

belle

d da

taÝ

Out

liers

BASIC DATA ANALYSIS

30


On

-th

e-jo

ble

arn

ing

Part

icip

ant

crit

eria

/pre

req

ui

site

s

Skill

/kn

ow

led

ge

targ

ets

and

per

form

ance

exp

ecta

tio

ns

LEA

RN

ING

PA

THD

ATA

AN

ALY

SIS

•Si

mpl

e da

ta t

rans

form

atio

n:Ý

Add

ing

new

var

iabl

esÝ

Tra

nsfo

rmin

g co

ntin

uous

to

cate

goric

al d

ata

Ý R

ecod

ing

of c

ateg

oric

al d

ata

Ý C

ompu

ting

new

var

iabl

es -

sum

med

, m

ultip

lied,

etc

.

•Ba

sic

desc

riptiv

e an

alys

is:

Ý F

requ

enci

esÝ

Cro

ss t

abs

Ý M

eans

, de

scrip

tives

Ý C

ompa

rison

of

mea

ns•

Read

ing

SPSS

des

crip

tive

data

out

put

tabl

esM

ost

lear

ning

tim

e is

use

d fo

r ha

nds-

on p

ract

ice

to a

llow

par

ticip

ants

to

deve

lop

thei

r sk

ills

•C

riter

ia,

inpu

ts,

outp

uts

and

supe

rvis

ors

mus

t be

cle

arly

def

ined

•Th

is is

a k

ey s

tep

in p

rodu

cing

peo

ple

who

can

hel

p w

ith b

asic

dat

a an

alys

is

•C

ompl

etio

n of

bas

ic t

rain

ing,

or

equi

vale

nt a

s de

term

ined

by

supe

rvis

ors/

faci

litat

ors

•Ex

perie

nce

of a

pply

ing

skill

s fr

om b

asic

tra

inin

g in

ow

n w

orkp

lace

, w

ith s

uppo

rt,

incl

udin

g re

gula

r us

e of

SPS

S an

d da

tam

anag

emen

t •

Iden

tific

atio

n by

ana

lysi

s su

perv

isor

s fo

r co

ntin

ued

lear

ning

•

Nee

d, o

ppor

tuni

ty,

time,

ava

ilabi

lity

and

appr

oval

for

lear

ning

, as

for

bas

ic c

riter

ia

Can

dida

tes

who

hav

e no

t co

mpl

eted

the

bas

ic t

rain

ing

can

part

icip

ate

in t

he in

term

edia

te t

rain

ing

only

if t

hey

have

com

plet

edeq

uiva

lent

tra

inin

g an

d w

ith a

ppro

val f

rom

the

sup

ervi

sors

of

both

lear

ners

and

fac

ilita

tors

•M

anag

e da

taba

ses,

with

sup

port

•Pe

rfor

m b

asic

dat

a cl

eani

ng,

with

sup

port

•Im

prov

ed p

rodu

ctio

n an

d re

adin

g of

bas

ic d

escr

iptiv

e ou

tput

s, w

ith s

uppo

rt•

Run

t-te

sts,

ano

vas

and

chi-s

quar

es in

terp

ret

outp

uts,

with

sup

ervi

sion

fro

m a

n ex

perie

nced

dat

a an

alys

t •

Run

mul

tiple

res

pons

e an

alys

is,

with

sup

port

•Im

prov

ed e

valu

atio

n of

dat

a an

d an

alys

is o

f ou

tput

s •

Und

erst

and

sam

plin

g:Ý

Dat

a ne

eded

to

calc

ulat

e sa

mpl

e si

zes

Ý S

ampl

ing

met

hodo

logi

es a

nd t

heir

data

nee

ds

Ý Im

plic

atio

ns o

f sa

mpl

ing

desi

gn o

n an

alys

is

•Im

prov

ed d

esig

n of

que

stio

nnai

res

Acq

uire

d sk

ills

are

asse

ssed

at

a fin

al e

xam

/eva

luat

ion,

with

onl

y th

ose

who

pas

s be

ing

cons

ider

ed f

or c

ontin

ued

anal

ysis

tra

inin

g D

epen

ding

on

the

part

icip

ants

sel

ecte

d, n

ot a

ll pa

rtic

ipan

ts w

ill a

chie

ve a

ll of

the

se s

kills

INTERMEDIATE DATA ANALYSIS

31


Co

nte

nt

On

-th

e-jo

ble

arn

ing

LEA

RN

ING

PA

THD

ATA

AN

ALY

SIS

•Re

view

of

SPSS

bas

ics

•D

ata

man

agem

ent

skill

s:Ý

Man

ipul

atio

n of

tab

les

- co

mpu

ting,

rec

odin

g, a

ggre

gatin

g, r

estr

uctu

ring

Ý M

ergi

ng o

f fil

es -

add

ing

case

s an

d va

riabl

es•

Basi

cs o

f sa

mpl

ing:

Ý T

heor

y, d

efin

ition

s an

d co

ncep

tsÝ

Des

ign

of s

ampl

es -

sim

ple

rand

om,

syst

emat

ic r

ando

m,

clus

ter,

stra

tifie

d an

d co

mpl

ex

Ý C

alcu

latio

n of

sam

ple

size

Ý A

ccou

ntin

g fo

r im

plic

atio

ns o

f sa

mpl

ing

durin

g an

alys

is•

Repr

esen

tatio

n•

Con

fiden

ce in

terv

als

•W

eigh

ting

•C

ontin

uous

var

iabl

es:

Ý R

evie

w o

f de

scrip

tives

Ý T

heor

y -

dist

ribut

ion,

ske

wed

ness

, et

c.Ý

Inde

pend

ent

sam

ple

t-te

stÝ

One

-way

ano

va a

nd p

ost-

hoc

test

s•

Cat

egor

ical

var

iabl

es:

Ý R

evie

w o

f de

scrip

tives

Ý T

heor

y Ý

Mul

tiple

res

pons

e an

alys

isÝ

Chi

-squ

are

test

•Q

uest

ionn

aire

des

ign

for

data

ana

lysi

s Le

arni

ng t

ime

is s

plit

betw

een

pres

enta

tion

of t

heor

ies

and

conc

epts

, an

d ha

nds-

on p

ract

ice

to a

llow

par

ticip

ants

to

deve

lop

thei

r sk

ills

•C

riter

ia,

inpu

ts,

outp

uts

and

supe

rvis

ors

mus

t be

cle

arly

def

ined

•Th

is is

a k

ey s

tep

in p

rodu

cing

peo

ple

who

can

hel

p w

ith d

ata

anal

ysis

, w

ith le

ss s

uppo

rt t

han

at t

he b

asic

leve

l

32


Part

icip

ant

crit

eria

/p

rere

qu

isit

es

Skill

/kn

ow

led

ge

targ

ets

and

per

form

ance

exp

ecta

tio

ns

Co

nte

nt

LEA

RN

ING

PA

THR

ESPO

NSE

OPT

ION

S

•C

ompl

etio

n of

bas

ic t

rain

ing,

or

equi

vale

nt a

s de

term

ined

by

supe

rvis

ors/

faci

litat

ors

•Pa

rtic

ipat

ion

in d

esig

n, p

lann

ing

and

impl

emen

tatio

n of

a r

apid

FSA

pre

ferr

ed•

Part

icip

atio

n in

dat

a co

llect

ion

for

an F

SA

•Ba

sic

unde

rsta

ndin

g an

d ex

perie

nce

of d

esig

ning

and

impl

emen

ting

food

sec

urity

inte

rven

tions

•N

eed

to im

prov

e/de

velo

p re

spon

se a

naly

sis

abili

ties

•Ti

me

avai

labl

e fo

r le

arni

ng

•A

ppro

val o

f su

perv

isor

(s) t

o de

dica

te t

ime

to le

arni

ng e

vent

s an

d fo

llow

-up

- on

-the

-job

lear

ning

, par

ticip

atio

n in

ass

essm

ents

, rep

ortin

g, e

tc.

•Id

entif

y w

hat

info

rmat

ion

from

foo

d se

curit

y an

d liv

elih

oods

ana

lysi

s is

nee

ded

for

the

resp

onse

ana

lysi

s•

Des

crib

e lin

ks b

etw

een

resp

onse

s to

que

stio

ns a

nd f

ood

secu

rity

deci

sion

s •

Eval

uate

out

com

es o

f th

e liv

elih

ood,

foo

d se

curit

y, n

utrit

ion

and

cont

extu

al a

naly

sis

and

iden

tify

fact

ors

that

det

erm

ine

wha

t –

if an

y -

resp

onse

(s)

wou

ld b

e ap

prop

riate

:Ý

Ana

lyse

foo

d ai

d’s

role

in a

llevi

atin

g fo

od in

secu

rity

and/

or m

alnu

triti

onÝ

Cla

ssify

imm

edia

te a

nd lo

ng-t

erm

res

pons

e op

tions

, ac

cord

ing

to t

ype

of p

robl

em•

Revi

ew t

he a

dvan

tage

s, d

isad

vant

ages

, fe

asib

ility

and

app

ropr

iate

ness

of

each

res

pons

e op

tion:

Ý C

ompa

re a

dvan

tage

s of

cas

h/vo

uche

rs a

nd c

ombi

natio

ns o

f fo

od a

nd c

ash/

vouc

hers

Ý A

pply

crit

eria

to

dete

rmin

e th

e ne

ed f

or s

elec

tive

feed

ing

prog

ram

mes

Ý

Ana

lyse

link

s am

ong

resp

onse

s, in

a t

ime

sequ

ence

Ý Id

entif

y w

hen

and

wha

t fo

rm o

f su

pple

men

tary

fee

ding

pro

gram

mes

are

app

ropr

iate

- b

lank

et o

r ta

rget

ed,

take

-hom

e or

on-

site

•Re

com

men

d a

pack

age

of r

espo

nses

Ý D

emon

stra

te r

elat

ions

hips

bet

wee

n fo

od a

nd n

on-f

ood

resp

onse

s, in

clud

ing

(i) a

gric

ultu

re -

foo

d-fo

r-se

ed,

food

-for

-agr

icul

ture

inte

rven

tions

, (ii

) w

ater

and

san

itatio

n -

food

-for

-bui

ldin

g w

ater

/san

itatio

n in

fras

truc

ture

and

(iii

) se

nsiti

zatio

n on

hea

lth a

nd n

utrit

ion

prac

tices

- f

ood-

for-

trai

ning

, et

c.•

Iden

tify

rele

vant

fac

tors

for

eva

luat

ing

the

appr

opria

tene

ss a

nd f

easi

bilit

y of

tar

getin

g op

tions

•U

nder

stan

d ho

w n

eeds

ass

essm

ent

shou

ld in

clud

e pr

actic

al c

riter

ia t

o as

sist

tar

getin

g

•In

terv

entio

n ty

pes

- fo

od a

nd n

on-f

ood

resp

onse

s •

Info

rmat

ion

need

ed t

o en

sure

tha

t no

n-fo

od r

espo

nses

are

con

side

red

•C

ondi

tions

whe

re f

ood

aid

may

be

an a

ppro

pria

te a

nd f

easi

ble

resp

onse

•Fa

ctor

s fo

r de

term

inin

g w

hat

– if

any

- re

spon

ses

wou

ld b

e ap

prop

riate

Ý Q

uest

ions

to

dete

rmin

e w

heth

er a

res

pons

e is

req

uire

dÝ

App

ropr

iate

and

fea

sibl

e re

spon

ses,

link

ed t

o pr

oble

m a

naly

sis

Ý A

naly

sis

of t

he a

dvan

tage

s an

d di

sadv

anta

ges

of e

ach

resp

onse

, us

ing

exam

ples

Ý C

ompa

rison

of

adva

ntag

es o

f ca

sh/v

ouch

ers

and

com

bina

tions

of

food

and

cas

h/vo

uche

rsÝ

Con

trib

utio

n of

nut

ritio

n in

form

atio

n to

res

pons

e an

alys

is•

Link

ing

resp

onse

s to

a t

ime

sequ

ence

and

mon

itorin

g/as

sess

men

t ac

tiviti

es•

Opt

imal

com

bina

tions

of

food

and

non

-foo

d re

spon

ses

•N

eeds

ass

essm

ents

and

pla

nnin

g of

exi

t st

rate

gies

/pha

se-d

own

33


•O

rgan

ize/

supe

rvis

e an

d le

ad a

sses

smen

t m

issi

ons

- ra

pid

FSA

s, C

FSA

Ms,

JA

Ms,

Con

solid

ated

App

eals

Pro

cess

/Nee

ds A

naly

sis

Fram

ewor

k (C

AP/

NA

F)•

Org

aniz

e/su

perv

ise

and

lead

in-d

epth

FSA

s•

Con

duct

mar

ket

anal

ysis

•D

evel

op/im

prov

e/ad

apt

ENA

too

ls a

nd m

etho

ds

•A

ppra

ise

ENA

cap

aciti

es o

f na

tiona

l/int

erna

tiona

l ent

ities

, an

d co

ntrib

ute

tona

tiona

l cap

acity

bui

ldin

g •

Emer

genc

y pr

ogra

mm

ing

- fo

rmul

ate

resp

onse

opt

ions

•Im

prov

e pr

e-cr

isis

info

rmat

ion

syst

ems

and

food

-sec

urity

mon

itorin

g•

Esta

blis

h ba

ckw

ard

and

forw

ard

linka

ges

betw

een

VAM

/ear

ly w

arni

ng a

ndpr

ogra

mm

ing

•Im

prov

e pr

epar

edne

ss f

or E

NA

- a

vaila

bilit

y of

info

rmat

ion,

too

ls,

peop

le,

fund

s an

d lo

gist

ics

at t

he c

ount

ry a

nd r

egio

nal l

evel

s•

Act

as

reso

urce

per

son/

faci

litat

or f

or a

sses

smen

t tr

aini

ng,

or c

oach

for

on-

the-

job

lear

ning

•Re

sear

ch -

impa

ct o

f fo

od a

id o

n m

arke

ts,

non-

food

res

pons

es,

mul

ti-se

ctor

alas

sess

men

ts,

inte

grat

ed p

hase

cla

ssifi

catio

n, e

tc.

SKIL

LS,

KN

OW

LED

GE

AN

D A

BIL

ITIE

STY

PE(S

) O

F A

DV

AN

CED

ASS

ESSO

R

•A

ll

•Th

ose

with

VA

M b

ackg

roun

d•

Thos

e w

ith a

t le

ast

basi

c ec

onom

ic e

xper

ienc

e; r

egio

nal a

sses

smen

t of

ficer

(RA

O m

arke

ts)

•RA

O a

nd a

few

oth

ers

•A

ll

•A

ll•

Thos

e w

ith V

AM

bac

kgro

und

•A

ll

•A

ll

•A

ll

•Th

ose

with

rel

evan

t ba

ckgr

ound

Tab

le 7

.Pe

rfor

man

ce E

xpec

tatio

ns o

f A

dvan

ced

Ass

esso

rs


TRA

ININ

G

NEE

DS

AN

ALY

SIS 3

Chapter 3 p34_39 FIN 27-05-2007 21:22 Pagina 35

36

3 . T R A I N I N G N E E D S A N A LY S I S

A country office that has decided to organize a learning event should contact the regional bureau twoto three months before the start of the planned event. The country office should explain:

• why it has decided to hold the learning event; and• what it hopes to achieve from it.

Once the regional bureau and Country Office have agreed that the learning event is required, facilitatorswill be appointed to plan and implement it. The first step in planning a course or workshop is thetraining needs analysis. This involves consulting senior country office staff about the gaps they perceivein the local capacity to carry out FSAs. The expectations of country office senior managers often surpasswhat is feasible for a short course, however, so facilitators should be clear about what participants canreasonably be expected to learn.

Training needs analysis has four components: (i) country context, (ii) methods review, (iii) stakeholderanalysis, and (iv) skills gap analysis. Facilitators collect information from country office consultations,learners’ questionnaires, the documentation of assessment methods and country programme activities.

This process includes:

• participating in telephone or face-to-face interviews with WFP and partner staff; • preparing a skills inventory/stakeholder analysis questionnaire and distributing it to learning event

participants about two months before the event is due to begin; the standard emergency needsassessment (ENA) questionnaire can be used, with questions added to address issues raised bycountry and regional staff;

• reviewing secondary information; • analysing the results.

Key outputs include the identification of:

• food security issues in the country, and how they differ from those in other countries;• stakeholders in food security assessment in the country; • the approach to FSAs adopted by the country office and its partners;• medium- and long-term strategic priorities for the country programme;• the concerns and priorities of senior regional and country staff regarding the capacity for and quality

of FSAs and VAM; • learners’ technical expertise, education levels and current work responsibilities; • appropriate learning path(s);• skill gaps and training needs; • necessary modifications to the participant selection criteria.

3.1 COUNTRY CONTEXT

For the training needs analysis, the country office provides facilitators with the following information:

• Learners: their positions, roles and assessment and skills.• Workshop language: Are all participants fluent in this language? Are the facilitators?• Previous assessment training in the country: Have the workshop participants taken part in these?• Frequency of WFP assessments in the country.• Most frequent causes of assessments in the country: sudden-onset emergencies, slow-onset

emergencies, etc.• Most commonly used assessment methods in the country and the problems with these.


• Two or three recent assessment reports illustrating the issues that affect the country.• A summary of WFP programmes in the country.• Case studies from the country: WFP and other.

3.2 METHODS REVIEW

The methods review examines how the country office and partners currently approach needsassessments, including the methodology they use. This enables facilitators to identify weaknesses in theapproach and gaps in the assessment skills of country office staff.

During dialogue with country office staff, facilitators collect the following information.

3.2.1 Approach to Assessments

There are two basic approaches to FSAs:

• A one-off assessment is a major administrative and logistical exercise in response to a specific event.One-off assessments often involve specialists from outside the country office.

• Continuous assessment is an ongoing process of information collection. Country office staffundertake regular monitoring, calling on external expertise only when the technical demands arebeyond their own capacity.

Country offices that follow the one-off approach to assessment require intensive training of a fewspecialist assessors, who will be deployed occasionally, and usually in collaboration with externalexperts. Country offices that carry out continuous assessment require general training of more staffmembers, who monitor the food security situation as part of their daily work, providing frequentanalyses of changes.

The frequency with which a country office carries out assessments is a good indicator of the approachit follows.

3.2.2 Assessment Methodology

The use of a standard assessment methodology demonstrates that the country office and its partnersappreciate the importance of collecting and analysing information.

The country office should provide facilitators with several recent assessment reports and informationabout the methodologies it uses. This enables the facilitators to judge the quality of the assessmentsand identify strengths and weaknesses in the way they are carried out.

If the country office does not use a standard methodology or uses one that is inadequate, the facilitatorssuggest changes and incorporate these into the assessment training.

3.3 STAKEHOLDER ANALYSIS

Stakeholder analysis clarifies the existing skills, knowledge and abilities of WFP and partner staff. Thisinformation guides facilitators’ selection of learning objectives and material for workshop and post-workshop learning events.

37



The objective of stakeholder analysis is to identify:

• the main actors in assessment in the country, their roles in FSAs, and their relevance to the countryteam’s food security analysis activities;

• the concerns and priorities of senior regional and country staff regarding the capacity for and qualityof FSAs and comprehensive food security vulnerability analysis (CFSVA) and other activities;

• participants’ technical expertise, education levels and current work responsibilities.

3.3.1 Partner Involvement in Assessments

During consultations with the country office, the facilitators gather the following information:

• Which government agencies and non-governmental organizations (NGOs) are involved in foodsecurity?

• Which of these undertake regular FSAs? What methodologies do they use?• Do agencies and NGOs collaborate with each other on FSAs?• With which agencies and NGOs does WFP collaborate?• How are roles defined in WFP partnerships with other agencies - who does what?• What assessment methodologies do WFP and its partners use?• Do other agencies carry out assessment training? If so, how? Do they invite WFP representatives?• Is the country office satisfied with its current partnerships? If not, what would it like to change?

This information helps facilitators to:

• identify workshop participants from outside WFP; representatives of current or potential partnersshould be invited to the workshop;

• harmonize the different training approaches used by different agencies, which is particularlyimportant when agencies invite their partners’ staff to learning events;

• analyse partnership methodologies to find out how agencies’ different approaches to assessmentare combined when the agencies work together;

• identify partnership problems, which can be addressed during the workshop.

3.3.2 Country Office Priorities

The country office should specify:

• weaknesses in previous assessments, including those identified by participants; • individual learners’ requests for training;• upcoming assessments for which staff require training;• the need for skilled assessors for emergency preparedness.

3.3.3. Current Assessment Capacity

Two months before the start of the workshop, the country office should inform the facilitators of eachlearner’s:

• current job and responsibilities for assessments;• future assessment responsibilities; • previous assessment experience: number and type of assessments, and role(s) within these;• assessment skills, as precisely as possible; • assessment training history – details and dates.

The facilitators insert this information into a grid, as shown in Table 8.

38

3 . T R A I N I N G N E E D S A N A LY S I S


Table 8. Learner Data

3.4 SKILLS GAP ANALYSIS

Through skills gap analysis, facilitators can identify:

• the appropriate learning path(s); • skill gaps;• which gaps can be addressed through workshops, distance learning, supervised on-the-job learning

and/or technical backstopping;• learning objectives;• criteria for selecting learning event participants;• appropriate modules from the Facilitators’ Tool Kit;• how to adapt material to match learners’ existing skills and professional requirements; • how to ensure continued learning and monitor progress after the workshop.

As a first step in skills gap analysis, facilitators analyse the learner data (Table 8) to build the learners’skills, knowledge and ability profile. Skills gap analysis is based on a combination of this profile, theperformance expectations listed in Table 5, the methods review and stakeholder analysis.

3.4.1 Country Office Perceptions

The country office can identify some of the problems it has encountered in assessments. Country officeswhose staff lack assessment skills, however, are likely to be unaware of some of the factors that couldimprove the quality of their assessments.

To check for additional skill gaps that the country office may have overlooked, the facilitators shouldanswer the following questions:

• Is the current country office assessment methodology satisfactory?• If not, how should it be amended?• Does the country office have sufficient staff with the skills necessary to apply the amended

methodology?

Once the facilitators have identified the skill gaps, they can decide which learning path - beginners,basic or intermediate - to use. They also need to determine whether all the gaps can be filled in oneworkshop; this depends on the learners’ profile and the nature of the skill gap(s).

When skill gaps cannot be covered in one workshop, the facilitators suggest follow-up activities, suchas further workshops, on-the-job learning and supervised participation in assessment teams. Follow-upactivities should be prepared before the workshop and reviewed throughout its course.

39


ASSESSMENTTRAININGHISTORY

ASSESSMENTSKILLS

ASSESSMENTEXPERIENCE(NO., TYPE,ROLE)

FUTUREASSESSMENTRESPONSABILITIES

CURRENT JOB ANDASSESSMENTRESPONSIBILITIES

Name



SESS

ION

A

DA

PTA

TIO

N

GU

IDEL

INES 4


42

4 . S E S S I O N A D A P TAT I O N G U I D E L I N E S

Every workshop is different from the others. The requirements of the country office and the skills andcapacities of the learners vary from case to case, and workshop content and facilitation must reflect thisdiversity. The Facilitators’ Tool Kit provides standard modules with exercises and examples. These shouldbe reviewed for each workshop and adapted to suit the workshop objectives and learners’ profile.Facilitators should consider the following points when selecting and adapting material:

• Are the learners managers or fieldworkers? Do they need to know how to organize and coordinateassessments, or how to collect information in the field?

• What needs have the country office and learners identified?• Has the methods review indicated weaknesses in the country office assessment approach? The

facilitators may identify issues that the country office has overlooked. • Do the country office’s reports provide sufficient material for workshop case studies? What other

material is available? It is useful to use a mixture of material from the country and elsewhere.

If the standard material is not appropriate, the facilitators may design new material.

This chapter provides guidance on adapting material in the Facilitators’ Tool Kit to the specific needs ofeach workshop. It is broken down into four sections: content, exercises, case studies and agenda.

There are no definite rules for adaptation, but facilitators should always remember that the modules arenot rigid and can be adjusted according to the circumstances.

4.1 ADAPTING CONTENT

4.1.1 Basic Principles

The following basic principles should be kept in mind when reviewing workshop material:

• The adapted materials must have coherent workshop objectives, learning objectives, content -subjects and presentation - and KATs.

• It is important to focus on the essential points to convey during each workshop session. Messagesshould be kept simple - for advanced learners as well as beginners.

• Experienced learners will already have some knowledge; workshop sessions should build on this.Learners’ existing knowledge should be ascertained during workshop preparation, especiallystakeholder analysis.

• The appropriate balance between presentations and exercises differs according to the level of thelearners, with advanced learners benefiting from a higher proportion of presentations. Evenadvanced learners have limited capacity to absorb messages presented in a one-way format,however, so exercises should always be included.

• For all levels, presentations should last no more than 25 minutes, and preferably range from 10 to15 minutes.

4.1.2 Key Steps

Workshop material is adapted to the context in four steps.

Pre-workshop

Step 1: Choose the modules to be covered during the workshop:• The country office indicates priority areas, based on perceived weaknesses and staff

learning needs.


• Facilitators suggest modifications to these, based on methods review and skills gap analysis.• Appropriate modules and sessions are chosen from the Facilitators’ Tool Kit.

Step 2: Ascertain learners’ existing skills and experience:

• Training needs analysis and distance learning exercises clarify the skills, experience andtraining history of workshop participants.

• The learners’ profile (Table 8) is matched to the learning paths (Table 5).• Learning objectives are adjusted for the workshop and for each module (see Section 2.1)

Step 3: Adapt workshop material - first revision:

• Check that the learners have the prerequisite skills and knowledge detailed in the workshopmodule session guidelines.

• Check that the contents of the selected modules are compatible with the learners’ experience andprevious training/education. For example, beginners with little experience and only secondaryschool education will not be ready for, or need, details about sampling methodologies.

• Check that the modules are relevant to learners’ current or expected jobs. For example, anational coordinator is unlikely to need training in PRA.

• Check that the expectations of the country office and partners are covered.• Information that goes beyond the learners’ capacities and needs can be included, as this

challenges them and helps them to understand the wider context. If too much advancedmaterial is included, however, learners will gain little from the workshop.

Workshop module contents that are not appropriate to the learners can be removed. The facilitatorscan also add material if the standard modules do not cover learners’ needs. When material is added orremoved, it may be necessary to reorganize the module to ensure that it flows.

During the workshop

Step 4: Adapt workshop material - further revision:

• Continuously assess the relevance of the workshop materials to the learners’ needs, andmake any necessary adaptations.

This is a very important step. No matter how well a workshop is prepared, there will always be somecomponents that do not work well. Facilitators cannot identify the most appropriate trainingapproaches until they have met the learners and started the workshop.

The following are common reasons for adjusting material during a workshop:

• The material is too/insufficiently detailed: facilitators remove/add material.• Learners respond poorly to certain features, such as presentations: facilitators increase the

proportion of other features, such as exercises.• Some sessions take more/less time than expected.• Learners are particularly interested in specific topics: facilitators add extra sessions on those topics.

Some of these issues are more easily addressed than others - it is easier to remove material than to addit, for example. Much depends on the facilitators’ knowledge of the module(s) to be changed. Iffacilitators are not confident about changing modules, they should retain them as they are, and merelyremove sections that are too complex for the group.

Daily feedback sessions help to gauge the learners’ reactions. Each day, two or three volunteers can beappointed to consult the other learners, identify the strengths and weaknesses of that day’s sessions,and communicate these to the facilitators, who should take them into account when planning the nextday’s activities.

43



44

4 . S E S S I O N A D A P TAT I O N G U I D E L I N E S

4.2 ADAPTING EXERCISES

Exercises are a fundamental component of workshops; at least one exercise should be included in everysession. Each of the modules in the Facilitators’ Tool Kit includes exercises targeted to the audience ofthat module. Learners who have absorbed the session will benefit from doing exercises that relate tothe session’s learning objectives. When learning objectives are adjusted, exercises may also need to bechanged.

The following are only two of many reasons for adapting exercises:

• If facilitators have simplified a session to suit the target group, the exercise(s) may also have to bechanged.

• Facilitators may substitute the exercise in the module with one that is more relevant to the context- for example, by including an exercise based on earthquakes in a region where earthquakes arecommon. Some exercises are based on case studies, which can be replaced with local ones toincrease the linkages between what is being learned and learners’ own work. Facilitators shouldensure that all the points to be illustrated are included in the replacement case study.

When facilitators add exercises, they should consider the following:

• The exercise(s) must be consistent with the rest of the module, and reinforce its message.• A mixture of different types of exercise should be used - group work to solve problems, role-playing,

rapid question and answer sessions, etc. As well as reinforcing learning, exercises also providevariety to a workshop and act as “energizers”.

• The tasks involved in an exercise should be clearly defined. When wording is vague or complex,learners spend more time interpreting the question than answering it.

• An exercise should not include too many issues. Time is always short, so focus on only one or twomessages.

• Exercises should make people think!

4.3 ADAPTING CASE STUDIES

Case studies demonstrate how the workshop messages work in real-life assessments. There are twomain types of case study:

• Local case studies are set in contexts that the learners know; they therefore link the workshopdirectly to the learners’ experience.

• External case studies are from different parts of the world. They illustrate issues that are not directlyrelated to the local context, or demonstrate the wider applicability of concepts. Learners are usuallystimulated by external case studies.

A mixture of local and external case studies should be used, including examples from the facilitators’own experience. Learners should also be encouraged to contribute examples from their experience.

Time is limited, so the case studies selected must focus on the message to be illustrated and not merelycontain one or two useful points buried in much that is irrelevant. The facilitators can summarize thecase study to highlight its important points, or write a new study based on their own experience. Thisis one of the most effective approaches.

Case studies should be no more than one page long so that learners can absorb them during thesession. Longer studies can be distributed in advance of the session, but if learners are unable to readthem in time, the session will be wasted.


4.4 ADAPTING THE AGENDA

Facilitators should not overfill each workshop day. Learners cannot absorb more than six hours - e.g.,four 90-minute sessions - of learning a day. After this, the law of diminishing returns sets in. Iffacilitators perceive a need for additional sessions, they can offer optional, informal discussions duringor after dinner.

45




PRE-

WO

RKSH

OP

PREP

ARA

TIO

N 5

48

5 . P R E - W O R K S H O P P R E PA R AT I O N

5.1 FACILITATORS AND RESOURCE PEOPLE

5.1.1 Facilitators

As well as experience of emergencies and knowledge of WFP emergency operations, policies andprocedures, FSA workshop facilitators must also have:

• strong training skills, including the ability to respond to learners’ needs with appropriate changes toplanned activities;

• good communications and listening skills;• experience in training and facilitation;• knowledge of emergency preparedness tools, response mechanisms and operational systems;• familiarity with the WFP Emergency Food Security Assessment Handbook.

Useful additional qualifications are:

• field experience of emergency assessments, preferably with WFP, the United Nations or partnerNGOs;

• good understanding of the capacities and response mechanisms of other agencies, particularlywithin the United Nations, and of emergency coordination mechanisms.

Annex B provides sample TOR for workshop facilitators.

5.1.2. Resource People

Resource people must be experts in assessment, including the use of WFP FSA policies, procedures,methods and tools. It is helpful if they also have experience in training and working with groups.

Facilitators should meet resource people before the workshop, to brief them on the workshop’scontents and on their expected role(s). Most resource people are not trainers, so facilitators shouldguide their expertise: for example, by deciding whether to use a particular resource person as a back-up commentator - to fill gaps as they appear - or the lead presenter.

In several regional FSA workshops, the use of learners as resource people has worked well and beenmuch appreciated by the other participants. Before selecting workshop participants as resource people,facilitators should consult country office staff to identify learners with sufficient expertise.

5.2 ADMINISTRATIVE ARRANGEMENTS

Six weeks before the workshop, the facilitators inform the country office of what equipment andfacilities they require, such as:

• projector;• photocopying/printing facilities;• flip charts, pens, paper, etc.;• meals;• transport.

If field trips are included in the workshop, the country office should identify locations:

• that are no more than one hour from the participants’ accommodation;

• that have recently received assistance from WFP or one of its partners; communities that WFP hashelped are more receptive to training exercises than those with no previous contact with the organization;

• where the population has been consulted and is happy to take part in the exercise.

Table 9. Schedule for Pre-Workshop Preparation

49


2–3 months

2 months

6 weeks

4 weeks

3 days

1-3 days

DEADLINE (BEFORESTART OF WORKSHOP)

COMPLETEDACTIVITY

Country office informs regional bureau and headquarters thatit wants to carry out a learning event

Country office defines/identifies:Learners’ profile (stakeholder analysis, Table 8)Workshop languagePrevious assessment training in the country

Country office provides:Recent assessment reportsSummary of WFP programmes in the countryCase studies from the country - WFP and/or other

Facilitators undertake:Methods reviewStakeholder analysisSkills gap analysis Facilitators distribute ENA questionnaires to workshopparticipants

Facilitators notify country office of workshop requirements:Field location(s) for practical sessionsProjectorPhotocopying/printing facilitiesFlip charts, pens, paper, etc.Others - e.g., beans for proportional piling

Facilitators distribute pre-learning exercise(s) to learners

Learners’ supervisors ensure that exercises andquestionnaires are completed

Learners return completed pre-learning exercises and ENAquestionnaires to facilitators

Country office identifies workshop venue and fieldlocation(s), and organizes accommodation and transport

Facilitators prepare workshop material

Facilitators arrive in the country

Facilitators check:Country office expectationsLearners’ profile, including language skillsWorkshop facilities and materialsTransport, accommodation and eating arrangementsField location(s) - distance, suitability

Facilitators carry out:Rapid key informant interviews with occupant(s) of fieldlocation(s)

50

5 . P R E - W O R K S H O P P R E PA R AT I O N

5.3 VENUE CONCERNS

Four weeks before the workshop, the country office identifies the workshop venue, and notifies thefacilitators.

Workshop venues should provide approximately 5 m2 of floor space per participant, be well-lit, and notcontain columns etc. that could block participants’ view of screens and presenters. Figure 3 gives asuitable layout for a workshop of 25 to 30 learners in a space of approximately 125 to 150 m2. Thefacilitators should inform the venue manager of the workshop dates and times, and request that thevenue is set up by the afternoon before the start of the workshop. Facilitators should also ensure thatspace is available for optional evening sessions.

Figure 3. Sample layout of a workshop venue

DocumentsTable

Proposed Room Setup

Overhead Screen

Projector Table

ParticipantTables

Flip

Charts

Resource Personsand Facilitators Table

PROPOSED ROOM SETUP

OVERHRAD SCREEN

5.4 ARRIVAL IN THE COUNTRY

Facilitators should arrive in the country at least three working days before the workshop begins,especially if they are not familiar with the country, to carry out the following.

5.4.1 Identification of Objectives and Learners’ Profile

The information provided by the country office should be checked to ensure that everyone has the sameexpectations:

• Why is the country office organizing the learning event? How does the event fit into wider staffdevelopment processes?

• Are assessments carried out regularly in the country? If so, what methodology is used? Areassessment reports available?

• Who are the learners? Are they all from WFP, or are some from other organizations?• How much experience do learners have? Is there a wide range of experience within the group?

What are the main assessment training needs, from the perspectives of individuals andorganizations?

• What language skills do workshop participants have?

5.4.2 Administration

• Check that all the necessary materials are available at the workshop venue - projector, flip charts,etc.

• Check that arrangements have been made for transport, meals, etc.

Field trip:

• Visit the location(s) proposed for fieldwork. • Visit the local authorities concerned. Check that everyone is happy for WFP to hold a learning event

at the location(s). • Undertake a rapid key informant interview - for example, with the local administrative leader - to

establish the main livelihood strategies and problems for people in the location(s). This is very usefulwhen designing exercises.

51



WO

RKSH

OP

IMPL

EMEN

TATI

ON

LOG

ISTI

CS 6

54

6 . W O R K S H O P I M P L E M E N TAT I O N L O G I S T I C S

6.1 AT THE START

Facilitators should review the workshop preparation checklist and course agenda with all the peopleinvolved in running the course. The checklist includes the logistics issues that need to be arranged inadvance of, during and after the workshop.

Define the roles and responsibilities of the course organizers, including responsibility for:

• liaison with hotel staff: providing hotel managers with the workshop agenda, including break andlunch times; arranging the distribution of participants’ welcome packs; and clarifying where breaksand lunches will be held - if possible, arrange for flexible buffet lunches, to allow learners to workduring lunch breaks;

• participants’ financial and travel issues: issuing daily subsistence allowances (DSAs); confirming airtickets, etc; and ensuring that hotel staff and participants understand what is – and what is not –included in the workshop package;

• equipment in the workshop venue: ensuring it is secure after hours.

Decide who will welcome the participants at the start of the workshop.

Facilitators meet all session presenters and/or resource people in advance to ensure they know thestarting and ending times for their sessions. They should clarify the materials that presenters andresource people need, and what they will bring themselves, and offer them support from the workshoporganizing team.

Tables, chairs and equipment should be set up in the workshop venue on the afternoon prior tocommencement; facilitators should guide the venue staff responsible for this. If the venue is secure,participants’ supplies - workbooks, manuals, pens, pads, etc. - can also be left in the venue.

Facilitators must ensure that the venue’s Internet connection functions. If Internet access is not availablein the venue, they should identify inexpensive alternative Internet access for participants during thecourse.

All the equipment to be used during the course should be tested. Facilitators should also ensure thattheir own computers are compatible with the data projector, and should load all the slides they plan touse on to their computers.

Facilitators should check that breakout rooms are available when needed, and that tables and chairs areproperly arranged in these.

They should also find out where the venue’s and other keys are kept after hours.

For participants making their own dinner arrangements, facilitators should identify a few affordablerestaurants within easy reach of the accommodation.

Facilitators should also locate the restrooms nearest to the learning event venue.

6.2 DURING THE WORKSHOP

On the second or third day of the workshop, facilitators should circulate a list of participants with theiraddresses, phone and e-mail details and ask them to edit their entries. The list should be revised, anddistributed at the close of the workshop.

As well as providing participants with copies of all the learning event presentations, facilitators can alsoprepare a workshop CD to distribute at the end of the workshop. The CD could contain copies ofpresentations, handouts, photos from the workshop, the participant list, and other materials. Learnerscan then reproduce materials for further in-country learning once they return to their offices.


RUN

NIN

G

THE

WO

RKSH

OP 7

56

7 . R U N N I N G T H E W O R K S H O P

7.1 ENSURING A PARTICIPATORY APPROACH

Course facilitators should be committed to using participatory methods. An assessment learning eventshould not be series of lectures, but should also include a lot of two-way communication. Successfullearning requires the exchange of ideas, information, points of view and experiences among learnersand facilitators.

A key aim of the FSA Basic Skills course is to build teams among WFP and its potential partners in acrisis. It is therefore essential that facilitators create a participatory environment, where everyone feelsthat her/his input is valued. Course presenters should not wave the WFP flag, criticize WFP partners orflaunt their own technical prowess. Instead they should focus on building relationships to strengthenthe capacities of the FSA community, and on learning new concepts, tools and techniques with theircolleagues.

Facilitators should ensure that all participants assume ownership of the course, add input, ask questionsand help teach. This strengthens the partnerships that WFP needs to carry out its assessmentresponsibilities.

The following are guidelines for running a participatory workshop that encourages participants’ buy-in:

• Be flexible: Be constantly aware of the interest, energy, spirit and body language of the learners. Arethey getting what they need from the session? Is the message reaching them, or is a rapid changeof plans required? Decide whether an unplanned discussion would be more useful than thescheduled presentation. If not, interrupt the discussion and offer to continue it after hours. Learnersvalue a flexible approach to learning sessions.

• Be firm when appropriate: One or a few learners should not be allowed to dominate a session. Ifone learner seems to be answering every question - thereby silencing the others - call on anotherone, even if his/her hand is not up. Encourage at least a partial answer, before turning to otherlearners to fill in. Dominant learners will realize that answers and dialogue must be shared.

• Be transparent: Facilitators are not expected to be experts in everything. They should never inventinformation in response to a question they cannot answer. Instead they should ask for the learners’help. If none of the learners have the necessary knowledge to answer, the question can be postedon a flip chart “parking lot” of unanswered questions, for consideration throughout the workshop.

7.1.1 Seeking Help

It is likely that some learners will have experience of some of the concepts, tools or techniques that arepresented in the workshop. Facilitators should identify in advance any participants whose expertisecould enrich the course. They should also be ready to let a learner steer a session in a direction that isof more value than the one planned. Learners and facilitators are all assembled to learn from each other.

Experienced participants who have been identified as potential resource people should be coached onhow to get their points across in a set time. Results from the preparatory learning tasks and participantintroductory interviews help to identify these people. Sometimes United Nations or NGO staff withsubstantial FSA experience are available in the country, and willing to act as resource people for alearning session.

7.1.2 Being Available

Course facilitators are “on call” every day from the moment the first workshop session starts until lateat night. Being available for all technical and administrative questions and concerns is a key part of thefacilitator’s job.

7.2 WORKSHOP METHODS

7.2.1 Varying the Methods

It is essential to vary the methodologies used throughout an FSA workshop. Analysis of case studies,work in small groups, plenary dialogue, simulations, quizzes and mini-presentations all help to buildunderstanding and skills. Facilitators should also draw on their own experience; introducing their ownmethodologies ensures dynamic, hands-on learning. Workshop agendas are full, however, sofacilitators should focus on time management, including in the selection of methodologies. Short five-to ten-minute energizers and icebreakers can be used whenever the workshop dynamic calls for them.Appointing a learner to organize the energizers ensures that they are regionally and culturally specific.

7.2.2 Using Small Groups

In general, adults work more easily, express themselves more freely, and learn more when they are insmall groups. Most workshop exercises are therefore carried out in small groups - organized randomly,as country teams, or to ensure an even spread of emergency experience across the groups. Moreexperienced learners should be identified in advance, and a few included in each working group.

Small groups should be organized to produce the best possible results. The members of each groupshould select a group facilitator and a reporter to present the group’s findings in plenary. The duties ofthe small group facilitator are to:

• encourage participation in discussions; • ensure that the necessary tasks are carried out;• manage time so that tasks are completed on time.

Facilitators should tell learners in advance whether small groups’ reports to plenary are to be verbal orwritten - e.g., on flip charts. Learners should be given the following guidelines on workshop reporting:

• Think before you write. • Use key words instead of full sentences. • Write in large letters - both upper and lower case - that can be read from 10 m away. • Write legibly, using the broad side of the marker, not the point. • When using cards to get group input, write only one idea per card so that ideas can be clustered.

A maximum of three lines should be written on each card.

7.2.3 Using Slide Presentations

Workshops include a number of mini-presentations, many of which use PowerPoint (PPT) slides to setthe scene for group activities and exercises. Slides should facilitate the presentation withoutcontrolling it.

Facilitators who are uncomfortable about using PPT can use flip charts to deliver a message. PPT slidesshould be reviewed carefully before the day of the session.

Remember: There is no need to present all the slides in a set. Facilitators should select those that bestsupport the session, by providing content and guidance.

Slides often need to be amended to suit a particular workshop. This is easily done in PPT. Whenamending slides remember to keep them simple, with a minimal amount of text, and limit the numberused. In general, three minutes per slide should be allowed, so a presentation of 15 minutes shouldinclude a maximum of five slides.

57


When using PPT, never merely read out the content of the slides to the learners. Instead, facilitatorsshould add information and point out real-life examples that illustrate what is written on the slides. Thisencourages learners to think about the content of the slides, rather than just reading them. Slides aremeant to stimulate discussion - not act as a substitute for it.

58

7 . R U N N I N G T H E W O R K S H O P


WO

RKSH

OP

FOLL

OW

-UP 8

60

8 . W O R K S H O P F O L L O W - U P

Regional technical staff, ODA capacity building staff and facilitators of the WFP FSA Basic Skills Courseare required to carry out the following follow-up activities.

8.1 WORKSHOP TEST RESULTS

The results of workshop tests (knowledge assessment test – KAT) - as percentage scores - should becommunicated to each learner as quickly as possible. They should also be forwarded to ODA, whichholds them in confidence for use only in guiding decisions about further FSA learning events and FSAteam secondments as needed.

8.2 LEARNING PROGRAMME FINAL REPORT

Facilitators should submit their final reports to the regional bureau technical staff and ODA as quicklyas possible so that lessons learned can be fed into the FSA Regional and Global Learning Programmeto inform the design of subsequent workshops.

Reports should include:

• the overall impressions and comments about the workshop - administration, venue, learners,resource people, agenda, etc.;

• the quantitative and qualitative analysis of information gained from the overall workshop, examresults, and individual sessions from learners’ workshop evaluation forms,;

• the summary of follow-up plans to continue staff development process; and• the lessons learned and recommendations.

Annex B1 and B2 on the Facilitators’ Tool Kit CD provides the format for a workshop report format.Additional examples are included in annex C5.

The final report should be circulated to the country office, the regional bureau and ODA’s CapacityBuilding Team at WFP Headquarters.

8.3 CERTIFICATES

ODA can supply the format for certificates of completion of the FSA Learning Programme. Copies ofsigned certificates should be provided to human resources staff at the regional bureau and/or countryoffice, for inclusion in participants’ personnel files.

The regional technical staff have three certificate options;

• Certificate of Participation in the FSA Course – provided to learners who attend the full workshop;• Certificate of Successful Completion - provided to learners who attend and obtain a passing test

score at the end of the workshop; • provision of two certificates: Participation and Successful Completion.

Regional bureaus should include a scheme for allocating certificates in their regional learning strategiesand action plans (Chapter 11).

Box 1

61


All learners attending the workshop completed the final exam. The exam scores and pre-workshop self assessmentresults were analyzed. Learners received the certificates based on their exam results. The following one of three ofcertificates were provided to learners:

• “Certificate of Participation:” - The learner participated in the workshop

• “Certificate of Achievement:” - The Learner participated in the workshop, demonstrated an understanding ofcourse content and passed the final text.

• “Certificate of Excellence:” - The Learner participated in the workshop, demonstrated a strong understanding ofcourse content and passed the final text.

WORKSHOP CERTIFICATES: BEST PRACTICE EXAMPLE – ODK QUANTITATIVE DATA ANALYSIS WORKSHOP


SUPE

RVIS

EDO

N-T

HE-

JOB

LEA

RNIN

G 9

The Supervised On-the-Job Learning Programme aims to enhance technical skills, FSA teammanagement skills and confidence in undertaking the tasks required for good-quality and transparentFSAs. On-the-job learning events can be formal or informal; this chapter focuses on formal supervisedon-the-job learning.

9.1 SUPERVISED ON-THE-JOB LEARNING ACTIVITIES

The following options are available to coaches and learners who are planning on-the-job learningactivities:

• Option 1: The learner and coach participate jointly in all stages of an FSA. The learner’s knowledge,skills and performance are increased through oversight, immediate feedback and technicalimprovement.

• Option 2: The learner is assigned to the regional bureau for a period, and works with the RAO orregional VAM officer, providing technical support to assessment activities.

• Option 3: The learner strengthens specific technical areas through contributing to components ofan FSA, but not a full FSA. This option can combine aspects of options 1 and 2.

• Complementary activities: The learner and coach identify additional activities based on thelearner’s learning path and learning objectives. Such activities can include reviewing technical papersselected by the coach, reviewing FSA reports produced by other assessors, and participating inmethodological debates or discussions of technical papers with WFP and external experts.

9.2 MANAGEMENT AND IMPLEMENTATION OF SUPERVISED ON-THE-JOB LEARNING EVENTS

The RAO and/or the regional VAM officer manage and implement on-the-job learning events for theregion, including selecting coaches and learners. Implementation involves the following steps:

• Selecting potential learners, and ensuring that they are committed to taking part in supervised on-the-job learning.

• Obtaining line managers’ approval and support for learners’ participation in the learning event, andclarifying its objectives.

• With the learner, identifying the areas where technical knowledge, skills and/or experience need tobe improved.

• Identifying the learner’s current knowledge, skills and abilities by interviewing her/him and matchinghis/her updated ENA questionnaire and FSA workshop outputs with the technical performancerequirements for FSA learning paths (Table 5) and the course descriptions (Table 5).

• Identifying the learner’s current FSA learning path level - beginner, basic or intermediate - and skillgaps.

• Using the pre-on-the-job learning questionnaire to establish learning objectives and a work plan,including time frames and outputs. Learning objectives should cover three areas:performance/actions, conditions and criteria/standards (Chapter 2.1).

• Identifying opportunities and locations for supervised on-the-job learning that match the learner’s

64

9 . S U P E R V I S E D O N - T H E - J O B L E A R N I N G

learning path and skills gap analysis, and tailoring activities to meet development targets.Developing a budget that covers travel and the costs of the learner’s involvement in supervised on-the-job learning. In 2007, funding for the programme will be available through ODA.

• Discussing the learner’s current abilities, skill gaps and work plan with his/her line manager,identifying the links between learning activities and the learner’s ongoing work plan at the countryoffice, and obtaining the line manager’s approval of learning objectives and plans.

• Implementing the activities included in the plan. Supervised on-the-job learning should combinepractical tasks for FSAs, FSA components and other activities.

• On completion of the activities, reviewing achievements and challenges, updating the line manageron progress made, and recommending ways of sustaining and developing the learner’s abilities.

• Completing supervised on-the-job learning feedback form(s) for submission to the regional bureau’sODA capacity building focal point, the learner and her/his line manager.

• The coach, learner and line manager should agree milestones for reviewing progress, andmechanisms for technical backstopping when required.

(Chapter 10.2 provides information on the monitoring and evaluation [M&E] of supervised on-the-joblearning activities.)

9.3 DURATION

On-the-job learning events can last for one week to two months. Work plans can include participationin an FSA, facilitation of a workshop and/or other complementary activities that address the learningobjectives.

9.4 LEARNING OBJECTIVES

As mentioned in Chapter learning objectives should cover performance, conditions and criteria. TheBloom Taxonomy Cognitive and Psychomotor Domains are useful tools for identifying clear supervisedon the job learning objectives.

Chapter 2 focused on the Bloom Cognitive Domain including, Knowledge, comprehension, application,analysis, synthesis and evaluation. The learning objectives for on-the-job learning activities should alsoincorporate the Bloom Taxonomy Psychomotor Domain. The Psychomotor Domain “relates to the abilityto do acts relevant to the field of study extension.”13

6513. Ferris L.J. Timothy, S.M. Aziz, A Psychomotor skills extension to Blooms' Taxonomy of Education objectives for Engineering Education. Exploring Innovation in

Education and Research Tainan, Taiwan. 1 - March 2005.


Table 10. Bloom Taxonomy Psychomotor Domain14

6614. Based upon R. H. Dave, as reported in R. J. Armstrong et al., Developing and Writing Behavioural Objectives (Tucson, AZ: Educational Innovators Press, 1970)

and Ruple, A. Judith, Angela Clark Burba, National Guidelines for Educating EMS Instructors, Ferris L.J. Timothy, S.M. Aziz, A Psychomotor skills extension toBlooms' Taxonomy of Education objectives for Engineering Education. Exploring Innovation in Education and Research Tainan, Taiwan. 1 - March 2005. BenjaminBloom's Taxonomy of Learning Domains - Cognitive, Affective, Psychomotor Domains. http://www.businessballs.com/bloomstaxonomyoflearningdomains.htm

9 . S U P E R V I S E D O N - T H E - J O B L E A R N I N G

Imitation /Observation

Manipulation

Precision /Competent

Articulation /consolidation

Naturalization &Mastery

Copy action ofanother; observe and replicate

Reproduce activityfrom instruction ormemory

Execute skill, reliably,independent of help

Adapt and integrateexpertise to satisfy anon standardobjective

Automatedunconscious masteryof activity and relatedskills at strategic level

Copy, replicate, duplicate, imitate

Re-create, build, perform,execute, implement, produce

Demonstrate, complete, show,perfect, calibrate, control,perform without assistance,perform with out error

Construct, solve, combined,coordinate, integrate, adapt,develop, formulate, modify,customize, demonstrateproficiency, perform withconfidence

Design specify, manage, invent,project manage, performautomatically

Watch coach and repeataction, process or activity.Observe a skill and attempt torepeat it

Carry out tasks from writtenor verbal instructions

Independently perform theskill or produce the product,with accuracy, proportion, andexactness; without assistanceor instruction

Modify the skill or product theproduct to fit new situations;combine more than one skillin sequence with harmonyand consistency

Completion of one or moreskills with ease and makingthe skill automatic with limitedexertion. Ability to multitaskeffectively. Define aim,approach and strategy for useof activities to meet strategicneeds

KEY WORDSCATEGORY BEHAVIOURDESCRIPTION

EXAMPLES OFDEMONSTRATION AND

EVIDENCE TO BE MEASURED



M&

E FR

AM

EWO

RKFO

R FS

A L

EARN

ING

EVEN

TS 10

WFP’s FSA Learning Programme has three components: distance learning, workshops, and on-the-joblearning. The M&E framework15 and its associated data collection tools allow WFP to gauge the extentto which each component contributes to the overall goal of improving FSA quality at the country,regional and global levels. This chapter focuses on the M&E framework and associated tools for FSAworkshops and on-the-job learning.

Table 11 outlines the types of data used to assess whether the activities, outputs, outcomes and impactof each activity have been achieved. Given the time lag between implementation of a component andthe emergence of an impact on assessment quality, impacts will not be measured until 2008.

Table 11. FSA Learning Strategy M&E Framework

6815. Workshops mentioned in Table 10 can be applied to various FSA learning events: emergency FSAs, JAMs, CFSAMs, and thematic/technical assessments.

1 0 . M & E F R A M E W O R K F O R F S A L E A R N I N G E V E N T S

Activities

Outputs

Outcomes

Impact

Level 1

Level 2

Level 3

Level 4

Measureslearners’reaction

Measureswhetherlearningobjectiveshave beenmet

Measureswhetherknowledgehas beenapplied post-workshop

Measuresimpact oftraining onassessmentquality

Courseevaluations

Courseevaluations

Learner andcoach survey

KATs

KATs

Coach evaluationof learner

n/a

Annual onlinesurvey

Annual onlinesurvey

n/a

Case studies

Case studies

Distancelearning

Workshops

On-the-joblearning

Distancelearning

Workshops

On-the-joblearning

Distancelearning

Workshops

On-the-joblearning

Distancelearning

Workshops

On-the-joblearning

2008onward

2007M&E

TERMSKIRKPATRICK

MODEL LEVELDESCRIPTION

TYPES OFLEAMINGEVENTS

DATA SOURCE

10.1 M&E FRAMEWORK FOR FSA WORKSHOPS

10.1.1 Tools for Monitoring Activities and Outputs

Workshop evaluations and KATs monitor the delivery of activities and the outputs, respectively. Bothtools are designed for use during the workshop by the facilitation team, and follow a more systematicdata collection approach than that used at present. Workshop evaluations and KAT templates areflexible, so users can adjust them to suit the information needs, learning objectives and time/resourceconstraints of the workshop. Using the templates reduces the time spent developing tools for eachworkshop, and allows the aggregation of data from all workshops.

10.1.2 Tools for Evaluating Outcomes and Impacts

Annual online surveys and case studies evaluate whether intended outcomes and impacts, respectively,have been achieved. They take time and resource constraints into account and represent a “light”approach to linking workshop activities/outputs to post-workshop application of knowledge andskills/outcomes and improved assessment quality/impacts.

ODA will be responsible for analysing data from annual online surveys, and will report disaggregatedand aggregated findings to regional bureaus and country offices. Combined with tools for trackingparticipation in learning events, annual online surveys will also help to track learners’ progress, at boththe individual and regional levels. The annual online survey questionnaire is provided in the CD-ROMaccompanying this document, and the online learner tracking system and learning materials repositorywill be available from June 2007.

10.1.3 Workshop Evaluations for Monitoring of Activities

√ When: during the workshop - end of each module or week.√ Applied by: the facilitation team.√ Data analysis and reporting: onsite by the facilitation team, or automated.

The workshop evaluation solicits learners’ reactions to aspects of the FSA workshop.16 The aspectsselected for evaluation depend on the information needs and the time and resources available foranalysing and applying the information collected. To increase its flexibility, the workshop evaluation pro-forma is divided into four components, which can be used in different combinations as shown in Table12. Data collection tools for each component are provided on the CD-ROM that accompanies thisdocument.

6916. The formal workshop evaluation described in this section aims to identify lessons learned for improving future workshops. It should be distinguished from the

use of volunteers to gather informal feedback from workshop participants (described in Chapter 4.1), which aims to guide the adaptation of the followingday’s sessions.


Activities Level 1Measureslearners’reaction

Distancelearning

Workshop

On-the-joblearning

Courseevaluations

Courseevaluations

Learner andcoach survey

M&E TERMKIRKPATRICKMODEL LEVEL

DESCRIPTIONLEARNING EVENT T

DATASOURCES

Component 1: This represents a minimalist approach to workshop evaluation and includes learners’reactions to individual sessions (1a) and to the workshop as a whole (1b). 1a should be applied at theend of each module or day, and 1b at the end of the workshop. The main format used is multiple-choicequestions, with additional space for learners’ brief comments on which aspects of the workshop needimprovement. The pro-forma for 1a covers all potential sessions, but only those included in theparticular workshop will have to be evaluated. The pro-forma for 1b may be adjusted to the needs ofthe facilitation team, country office and regional bureau, but it is designed to be universally applicable.

Component 2: This seeks learners’ feedback on how the workshop could be improved, and should beapplied at the end of the workshop. The main format is open-ended questions to generate deeperinsight into improving workshop quality than that provided by component 1’s quantifiable data. Thequalitative nature of this component demands more time and resources for analysing and using theinformation gathered. The facilitation team, country office and regional bureau must therefore gaugethe added value of collecting these data against the costs of doing so.

Component 3: This provides a reflexive baseline against which to gauge the knowledge and skills gainedduring the workshop, and should be applied at the end of each module or day.17 The format isexclusively multiple-choice questions that demonstrate the new knowledge gained and the pre-existingknowledge bolstered by each session. This component should be used only when there is a clear needto attribute knowledge and skills gains to the workshop.

The facilitation team can combine these components according to the information needs and the datacollection and analysis burdens associated with each component (Table 12).

Table 12. Workshop Evaluation Components and Possible Combinations

Table 13. Selecting a Combination of Components

7017. The “reflexive baseline” is based on learner’ recall rather than pre-workshop skills assessment. Although the latter is more rigorous, its time and resource needs

are too great for the M&E of FSA workshops. In addition, repeated asking of the same questions is likely to result in respondent fatigue.18. Chapter 10.1.4 provides additional information on the purpose of KATs (M&E term - Outputs, Kirkpatrick - level 2).


1a

1b

2

3

Gauge learner reaction to individual sessionsand identify areas for improvement

Gauge learner reaction to averall learningevent and identify areas for improvement

Solicit more detailed feedback fromparticipants on how to improve learningevent

Rough approximation of knowledge/skillsgained (attribution)

COMPONENT INFORMATION NEEDS 4 POSSIBLE COMBINATIONS

Minimalist 1a – Learners’reactions to sessionor module

1b – Learners’reactions to overallworkshop (sessionsand environment)

1a - End of eachmodule or session

1b - End ofworkshop

• Limited time and resources• Only opinions and appraisals of each

session and the overall learningenvironment are required

• Insufficient time for collecting andanalysing qualitative information onimproving sessions

• KATs are required.18

POSSIBLECOMBINATIONS

COMPONENTS WHEN RATIONALE

Min

imal

ist

Expa

nded

Expa

nded

+ A

ttrib

utio

n

Att

ribut

ion

+M

inim

alis

t

With the exception of the narrative responses generated in component 2, each question in theworkshop evaluation provides a potential descriptive indicator for monitoring purposes. Theseindicators will usually be percentages of respondents providing a specific answer, or average responsesfrom among all the respondents, for example:

• percentage of respondents rating the clarity and effectiveness of a session as “good” (4 = good);• respondents’ average score for the session’s clarity and effectiveness according to whether they

rated it 5 = excellent, 4 = good, 3 = average, 2 = poor, 1 = unacceptable, or 0 = does not apply.

Composite indicators may be generated for all the sessions in a module or workshop. Although theyprovide an overview of learners’ reactions or new skills - through analysing the data from component 3 -these indicators must be interpreted with care, because they can mask important differences in reactionsfrom session to session and among learners. The following is an example of a composite indicator:

• respondents’ average score for the clarity and effectiveness of all the sessions in a module accordingto whether they rated it 5 = excellent , 4 = good, 3 = average, 2 = poor 1 = unacceptable, or 0 =does not apply.

Box 2

10.1.4 KATs for Monitoring Outputs

√ When: during the workshop - end of each module, day or workshop. (optional: Pre-workshopexam)20

√ Applied by: the facilitation team.√ Data analysis and reporting: onsite by the facilitation team, or automated.

7119. Ibid 20. A pre-workshop exam can be delivered to obtain a greater understanding of the learner’s level prior to the workshop to allow facilitators to greater

understanding of knowledge and skill gaps. Combining the analysis a pre and post workshop exam will provide an understanding of what the learner hasachieved through the event.


Expanded

Expanded andattribution

1a, 1b and 22 - Feedback onimprovement andoverall methods

1a, 1b, 2 and 33 -Approximation ofskills and knowledgegained

1a - End of eachsession1b and 2 - End ofworkshop

1a and 3 - End ofeach session1b and 2 - End ofworkshop

• Regional bureau and country office want toimprove tools and methods based onfeedback; e.g., pilot workshops

• Sufficient time and resources for reviewingnarrative feedback from learners

• Sufficient time within workshop agenda forlearners to provide narrative feedback

• KATs are required.19

• See first two points above• Pre- and post-KATs are not possible • Alternative method for measuring

knowledge gained is required

POSSIBLECOMBINATIONS

COMPONENTS WHEN RATIONALE

The data entry and analysis burden associated with workshop evaluations, KATs and annual online surveys can limitthe effective use of the information collected.

Instead of entering these data, they could be scanned into QuestionMark software, which provides automatedreporting of basic descriptives, composite averages and correlations. ODA will manage automated analysis andreporting, working with country offices, regional bureaus and facilitation teams to ensure that the use ofQuestionMark-compatible tools does not increase the time and resources needed for data collection.

Microsoft Excel, SPSS and other database/spreadsheet software also provide options for collating and analysingevaluation data, but these packages require manual entry and analysis.

AUTOMATED DATA ENTRY AND ANALYSIS USING QUESTIONMARK SOFTWARE

KATs are an integral part of the learning process, allowing learners and facilitators to gauge the extentto which the messages of each session have been absorbed. When linked to learning objectives, KATsalso gauge the extent to which the learning objectives have been met.

As depicted in the Module and Session Outline for the Facilitators’ Tool Kit, each session has severallearning objectives. Each learning objective can be assigned one of the following three values describingthe depth of learning that the session is intended to impart:21

• knowledge - memorization, or the ability to recall information;• comprehension - understanding, or the ability to state concepts in one’s own words;• application - ability to apply concepts to situations in the field.

The depth of learning for a particular learning objective is constant, but the selection of which sessionsand learning objectives to include in a workshop varies according to the audience and aims of theworkshop (Box 2).

KATs have been developed for each of the learning objectives outlined in the Facilitators’ Tool Kit; theaccompanying CD-ROM includes a sample KAT format. Each KAT aims to assess whether the depth ofknowledge associated with the learning objective has been achieved. For example:

• KATs for learning objectives aimed at imparting basic knowledge ask learners to recall messagespresented during the session.

• KATs for learning objectives aimed at increasing comprehension test whether learners haveunderstood messages, rather than merely memorizing them.

• KATs for learning objectives aimed at enabling learners to apply new concepts or skills test learners’ability to do so by presenting them with novel scenarios.

Box 3

7221. These values are based on the Bloom taxonomy (Table 2).


Outputs Level 2Measures whetherlearning objectiveshave been met

Distancelearning

Workshop

On-the-job learning

KATs

Knowledgeassessment tests

Coach evaluation oflearner


DESCRIPTIONLEARNING

EVENT TYPESDATA

SOURCES

If most learners are government ministers and project managers - i.e., the users of assessments - the workshoplearning objectives may be limited to the knowledge depth of learning, for example:

• describing the main components of the analysis plan. If most learners are likely to participate in assessments, the depth of learning may be expanded to “comprehension”,for example:

• explaining the importance of the analysis plan; • explaining the relationship between the analysis plan and other steps in assessment planning and

implementation.

If most learners will be developing analysis plans as part of their roles in FSAs, the depth of learning may beexpanded to “application”, for example:

• preparing an analysis plan.

FITTING THE LEARNING OBJECTIVES TO THE AUDIENCE

The number of KATs included in a quiz depends on the frequency of quizzes - once a session or oncea module, for example - but each quiz should include at least one KAT for each learning objectivecovered.

KATs for each of the learning objectives in each session are available from ODA’s “test question bank”.These include a variety of question types - true/false, multiple-choice, multiple-response, short answer,matching, ordering, etc. – so that facilitation teams can vary the formats used in tests. The test questionbank is not intended to be prescriptive, however, and facilitation teams can develop their own quizquestions.

Before selecting KATs, the learning objectives for each workshop session must be identified. This ischallenging when the workshop participants are heterogeneous in terms of pre-existing knowledge,skills and likely roles in using and conducting FSAs.22

Facilitation teams that develop their own KATs for issues not covered by the quiz question bank shouldsubmit the new KATs to ODA after the workshop for review and potential inclusion in the quiz questionbank.

As with workshop evaluation data, KAT data can generate a variety of monitoring indicators, includingperformance indicators for individual learners and aggregate performance indicators for all learners,which may be disaggregated by KAT, learning objective, session or module. Examples of KAT data thatcan be used in monitoring include:

• percentage of KATs that an individual learner answered correctly;• percentage of learners answering at least 70 percent of all KATs correctly;• average percentage of KATs answered correctly across all sessions and all learners.

10.2 M&E FRAMEWORK FOR ON-THE-JOB LEARNING

Learner and coach evaluations monitor on-the-job learning activities. Coaches’ pre- and post-learningevent appraisals of learners’ FSA skills, knowledge and abilities monitor outputs.

10.2.1 Monitoring On-the-Job Learning Outputs

√ When: before and after the on-the-job learning event. √ Applied by: supervisor and learner.√ Data analysis and reporting: by supervisor.

The pre-on-the-job learning questionnaire allows the coach and learner to identify clear learningobjectives, tasks and outputs. The learner and coach use the FSA learning path descriptions or thematicand technical course descriptions to evaluate the learner’s capacity and identify skill and knowledge gaps.

7322. Heterogeneity was repeatedly cited as one of the most problematic aspects of FSA workshops, suggesting a need to revisit the participant selection process.


Outputs Level 2Measures whetherlearning objectiveshave been met

Distance learning

Workshop

On-the-job learning

KATs

KATs

Coach evaluation oflearner


DESCRIPTIONLEARNING

EVENT TYPESDATA

SOURCES


The learning objectives and work plan are based on skills gap analysis. For each learning objective, thecoach and learner, through self-rating, evaluate the learner’s abilities in each task.

At the end of the learning event, the coach uses the post-on-the-job learning questionnaire to appraisethe learner’s achievements under each objective and task. This appraisal also helps the coach andlearner to identify ways of applying the skills enhanced by on-the-job learning to an FSA or relatedactivity, and to plan how to address other skill gaps. The following is an example of an indicator thatcan be used in monitoring

• Percentage change in score of coaches’ pre and post ratings.

10.2.2 Monitoring On-the-Job Learning Activity Outputs

√ When: after the on-the-job learning event. √ Applied by: learner.√ Data analysis and reporting: ODA, in collaboration with the regional bureau.

The on-the-job learning survey is designed to solicit learners’ and coaches reactions to the learningactivity, including the extent to which expectations were met, unexpected outcomes and suggestionsfor improving implementation.

10.3. EVALUATING OUTCOMES THROUGH THE ANNUAL ONLINE SURVEY

√ When: annually, for previous 12 months.√ Applied by: ODA, in collaboration with the regional bureau, via e-mail.√ Data analysis and reporting: ODA, in collaboration with the regional bureau.

Evaluation of outcomes assesses whether learners are applying their new knowledge and skills to theirdaily work and requires the collection of data long after the learning event has finished. This places asignificant burden on learners, so a minimalist approach to outcome evaluation is warranted.

74


Activities Level 1Measureslearners’reaction

Distance learning

Workshop

On-the-joblearning

Course Evaluation

Course Evaluation

Coach and learnersurvey

M&E TERMKIRKPATRICKMODEL LEVEL DESCRIPTION

LEARNING EVENT TYPES

DATA SOURCES

Outcome Level 3

Measures whetherknowledgehas been appliedpost-workshop

Distance learning

Workshops

On-the-joblearning

N/A

Annual online survey

Annual onlinesurvey



DATA SOURCES


For the annual online survey,23 respondents answer a series of multiple-choice questions designed totake between five and ten minutes to complete. Via e-mail, the country office or regional bureau sendsall FSA learning event participants from the previous 12 months a brief explanation and the link to theonline survey. A degree of non-response is expected, particularly among government counterpartswhose job mobility makes them difficult to track.

As well as providing data for outcome evaluation, the annual online survey also helps to track learners’progress after completion of learning activities. Unlike the workshop evaluation, it will therefore not beanonymous.

The annual online survey allows WFP to generate the following indicators at the country, regional andglobal levels:

a) Percentage of learners using the skills/knowledge gained during the learning event, measured usinga three-unit Likert scale: not used at all, used somewhat, used frequently. (“Use” includesapplication of knowledge and skills to both FSA and non-FSA job functions.)

b) Percentage of learners participating in an FSA since the learning event. c) Percentage of learners participating in food security monitoring, JAMs and/or CFSAMs since the

learning event.d) Percentage of learners participating in other learning events, including FSA distance learning, FSA

on-the-job learning and other WFP or non-WFP training.e) Percentage of learners performing each of the following 12 FSA functions under supervision, during

an FSA since the learning event.f) Percentage of learners performing each of the following 12 FSA functions in a leading or primary

responsibility role, during an FSA since the learning event.g) Percentage of learners finding that the learning event prepared them for their roles in FSAs,

disaggregated by their FSA function.h) Percentage of learners identifying each of the following 12 FSA functions as one of the three areas

in most need of skills development.

The 12 FSA functions used to disaggregate indicators are:

1. Team leader2. Management of assessment

3. Market analysis

4. Design of assessment (sampling, data collection instruments, analysis plan)

5. Management of interviewers and/or enumerators

6. Data collection

7. Data entry

8. Quantitative data management and analysis

9. Qualitative data management and analysis

10. Data interpretation, drawing of conclusions and reporting

11. Response options analysis

12. Monitoring and early warning

7523. The annual online survey will use online survey software similar to Monkey: www.surveymonkey.com.


10.4 EVALUATING IMPACT THROUGH CASE STUDIES

√ When: annually, beginning in 2008.√ Applied by: ODAN.√ Data analysis and reporting: by ODAN.

Gauging the extent to which FSA learning events improve FSA quality is an important means ofvalidating WFP’s approach to achieving this goal.24 However, the wide range of factors affecting FSAquality makes it extremely difficult – if not impossible – to quantify the causal effects of learnersapplying the knowledge and skills gained during an FSA learning event.

The time-lag between learning events and their impacts on assessment quality limits the effectivenessof information collected by the Strengthening Emergency Needs Assessment Implementation Plan(SENAIP) evaluation in March to April 2007. Selected case studies could provide an alternativeinformation source without creating an undue data collection and analysis burden.

A small number of case studies conducted in 2008-2009 would provide the basis for substantiating theimpact of FSA learning events on improved assessment quality. The impact of learning events onassessment quality would then need to be reassessed only when WFP makes significant changes to itslearning strategy.25

FSA learning strategies are only one element of WFP’s overall strategy for improving assessment quality,however. The case study approach would therefore be only one of the evaluation tools needed forgauging the impact of this overall strategy on improving assessment quality.26

76

24. WFP’s strategy was based on an assumed link between knowledge and skills gained during learning events and increased FSA quality. Using this sameassumption to validate the strategy’s design is therefore extremely problematic.

25. If no such impact is established, WFP should revisit its strategy for improving FSA quality.26. The quality of FSAs is currently appraised using a checklist of FSA report components. Although useful, this is widely recognized to be an inadequate measure

of the quality of the FSA process and of the internal/external constraints faced by FSA teams in the field. The use of case studies is one way of gauging thesemore difficult to measure aspects of FSAs.


Impact Level 4Measures Impactof training onassessment quality

Distance learning

Workshops

On-the-joblearning

N/A

Case Studies

Case Studies



DATA SOURCES



REG

ION

AL

FOO

DSE

CU

RITY

LEA

RNIN

GST

RATE

GY

AN

DA

CTI

ON

PLA

NS 11

Regional learning strategies focus on improving assessment performance through building theassessment skills, knowledge and abilities of staff and partners in the region. Regional technical staffshould direct appropriate learning events to countries to ensure that country staff have the skillsnecessary to lead or contribute to FSAs. Regional senior staff should endorse assessment staff’sdevelopment plans and outcomes.

Regional strategies are based on analysis of skill gaps, programme cycles, seasonal assessment plans,staff and partner roles in assessments, and contingency plans. The results of this analysis are used totarget food security workshops linked to assessments, on-the-job learning activities and technicaltraining, taking into account leadership and technical skills and abilities, assessment methods, thecountry context and staff development priorities.

The regional strategy:

• makes an inventory of current and planned FSA and analysis activities in the region, and identifiesassociated duties and tasks;27

• makes an inventory of regional assessment capacities, and identifies gaps and opportunities forensuring quality assessments within the region;

• develops learners’ profiles, based on learners’ current job responsibilities, roles in FSA and analysis,and skills, knowledge and abilities;28

• clusters countries with similar emergency contexts, types of assessment and analysis activities andcapacities;

• determines which gaps can be addressed by learning events, and recommends alternatives for thosethat cannot;

• selects the most appropriate learning event(s) to fill the skill gaps;• develops a time frame for learning events, linked to country offices’ assessment activities;• determines modalities to ensure successful implementation of learner selection criteria, based on

skills gap analysis; • identifies activities that expose learners to FSAs and related activities, through understudying; • identifies ways in which learning events can benefit staff’s career development, linked to regional

bureau and country office expectations, staff appraisals, performance and competencyenhancement, etc.;

• establishes M&E for the learning strategy and action plan that feeds into regional and global (ODA)systems for tracking learners, outcomes and impacts of learning events;

• provides country directors with a synthesis of the learning strategy and action plan, and ensures thatcountry directors understand and agree with the rationale and components and their own roles inimplementing the strategy.

11.1 RESPONSIBILITIES FOR POST-LEARNING EVENT FOLLOW-UP

ODA, regional bureaus and country offices each have joint responsibilities for learner follow-up. Aftereach learning event, learners should be given the opportunity to apply and develop their new skillsthrough participation in FSA and analysis activities. ODA and regional bureaus should maintain a recordof staff members and partners trained, and ensure that learning events are followed by opportunitiesfor enhancing skill acquisition and sustainability.

7827. Table 5 is a useful tool for identifying tasks. 28. Table 5 provides a useful breakdown for developing learner profiles.

1 1 . REGIONAL FOOD SECURITY LEARNING STRATEGY AND ACTION PLANS

Box 4

79


From 12 to 16 February 2007, the Regional Bureau for East and Central Africa, Kampala (ODK) undertook anEnglish-language version of the VAM Basic Data Analysis, attended by 18 participants from six countries: Uganda,the United Republic of Tanzania, Rwanda, Kenya, Ethiopia and Somalia. The purpose of the workshop was to buildthe skill and confidence of WFP staff in using SPSS to analyse household survey data, interpret the outputs andpresent findings as graphic or tables.

Learner selection was based on clear criteria; the materials used provided learners with real data and the opportunityto apply basic analysis techniques to actual household surveys.

At the end of the workshop, learners responded to a brief test based on the learning objectives and sessions covered.

ODK’s learner follow-up action plan included the following elements:

• Based on the results of the test and on learners’ performances during the workshop, participants were selectedfor on-the-job learning and capacity building activities for the following ten months.

• Based on the 2007 schedule of assessment activities, learners were matched to lead assessors/coaches, and anon-the-job learning plan was drawn up for each learner.

• Based on needs identified from the post-on-the-job learning evaluation, learners’ future skill development,through distance coaching and follow-up, was planned.

LEARNER FOLLOW-UP:

BEST PRACTICE EXAMPLE FROM A BASIC-LEVEL QUANTITATIVE DATA ANALYSIS WORKSHOP

80

1 1 . REGIONAL FOOD SECURITY LEARNING STRATEGY AND ACTION PLANS

Too

ls f

or

po

st-l

earn

ing

eve

nt

trac

kin

g a

nd

pro

mo

tio

n o

f p

ract

ical

ap

plic

atio

n

•Pr

ovid

e gu

idel

ines

for

cap

acity

ana

lysi

s, c

oach

ing

and

on-t

he-jo

b le

arni

ng a

ctiv

ities

.

•Pr

ovid

e re

gion

al t

echn

ical

sta

ff w

ith in

tera

ctiv

etr

acki

ng s

yste

ms

for

mon

itorin

g an

d co

nsol

idat

ing

info

rmat

ion

on le

arni

ng e

vent

s an

d le

arne

rs.

•Pr

ovid

e re

port

ing

tool

s fo

r m

onito

ring

lear

ners

’re

actio

ns t

o le

arni

ng e

vent

s an

d th

e ex

tent

to

whi

ch le

arni

ng o

bjec

tives

hav

e be

en m

et.

Sup

po

rt f

un

ctio

n

•A

ssis

t th

e re

gion

al b

urea

u w

ith t

he r

egio

nal a

naly

sis

of le

arni

ng a

ctiv

ities

, out

puts

and

out

com

es, a

ndw

ith s

kill

prof

iles.

•Pr

ovid

e te

chni

cal b

acks

topp

ing,

if r

equi

red,

to

regi

onal

bur

eau

tech

nica

l sta

ff in

upd

atin

g po

st-

wor

ksho

p co

untr

y of

fice

actio

n pl

ans,

re-

eval

uatin

gsk

ill s

ets

and

deve

lopi

ng m

odal

ities

for

tra

ckin

gle

arne

rs’

prog

ress

, us

ing

the

lear

ning

rep

osito

ry,

asse

ssor

s’ d

atab

ase

and

othe

r m

echa

nism

s.

•Pr

ovid

e te

chni

cal f

eedb

ack

on r

egio

nal b

urea

u an

dco

untr

y of

fice

actio

n pl

ans

and

impl

emen

tatio

n of

on-t

he-jo

b le

arni

ng a

ctiv

ities

.

An

nu

al s

urv

ey

•Fa

cilit

ate

an a

nnua

l on-

line

surv

ey o

f fo

od s

ecur

ityle

arne

rs,

incl

udin

g in

form

atio

n on

pos

t-le

arni

ngev

ent

asse

ssm

ent

and

anal

ysis

act

iviti

es,

furt

her

trai

ning

req

uire

men

ts,

and

feed

back

on

the

lear

ning

pro

gram

me.

Ens

ure

resu

lts a

re a

cces

sibl

eto

reg

iona

l bur

eaus

.

OD

A,

HEA

DQ

UA

RTE

RS

CO

UN

TRY

OFF

ICE

(CD

, SU

PER

VIS

OR

AN

D L

EAR

NER

)

Lear

nin

g r

epo

sito

ry a

nd

wo

rksh

op

rep

ort

•U

pdat

e th

e le

arni

ng r

epos

itory

with

dat

a on

com

plet

ed le

arni

ng e

vent

s an

d pa

rtic

ipat

ing

lear

ners

(in

wor

ksho

ps a

nd o

n-th

e-jo

b le

arni

ng).

•C

ircul

ate

wor

ksho

p m

ater

ial t

o ap

prop

riate

sta

ffm

embe

rs (

lear

ners

, co

untr

y of

fice,

reg

iona

lbu

reau

and

OD

A).

Dev

elo

p s

ho

rtlis

t o

f le

arn

ers

for

furt

her

ski

lld

evel

op

men

t

•A

fter

lear

ning

eve

nts,

rev

iew

the

reg

iona

las

sess

men

t an

d ca

paci

ty a

naly

sis.

•Re

-clu

ster

sta

ff a

ccor

ding

to

thei

r sk

ills,

know

ledg

e, a

bilit

ies

and

func

tions

in a

nas

sess

men

t.

•Ba

sed

on t

he c

lust

ers

and

the

coun

try

cont

ext,

prio

ritiz

e le

arne

rs f

or f

urth

er s

kill

deve

lopm

ent

and

follo

w-u

p. T

rack

the

pro

gres

s of

the

sele

arne

rs/g

roup

s.

•Id

entif

y ac

tiviti

es t

hat

expo

se t

hese

lear

ners

to

asse

ssm

ents

and

rel

ated

act

iviti

es.

Co

un

try

off

ice

lear

nin

g a

ctio

n p

lan

s

•Fo

r co

untr

y of

fice

wor

ksho

ps,

gain

con

sens

us o

fco

untr

y of

fice

seni

or s

taff

on

lear

ner

follo

w-u

ppl

ans

for

cons

olid

atin

g sk

ill a

cqui

sitio

n th

roug

hpr

actic

al a

pplic

atio

n.

•If

appr

opria

te,

iden

tify

mod

aliti

es f

or a

dditi

onal

lear

ning

eve

nts

that

tar

get

spec

ific

skill

set

s an

dle

arne

rs.

REG

ION

AL

BU

REA

U

Ass

esso

rs’

dat

abas

e

•Le

arne

rs u

pdat

e th

eir

own

prof

iles

on t

heas

sess

ors’

dat

abas

e (e

.g.,

thei

r pa

rtic

ipat

ion

inas

sess

men

ts a

nd le

arni

ng e

vent

s).

Perf

orm

ance

an

d C

om

pet

ency

En

han

cem

ent

Pro

gra

mm

e (P

AC

E) a

nd

pra

ctic

al a

pp

licat

ion

•Su

perv

isor

and

lear

ner

inte

grat

e po

st-le

arni

ngev

ent

appl

icat

ion

resp

onsi

bilit

ies

into

PA

CE.

•Id

entif

y as

sess

men

t an

alys

is a

ctiv

ities

for

lear

ners

to p

artic

ipat

e in

.

Co

un

try

off

ice

follo

w-u

p l

earn

ing

act

ion

pla

n

•W

ith t

he r

egio

nal b

urea

u, r

evie

w t

he u

pdat

edsk

ills,

kno

wle

dge

and

abili

ty a

naly

sis.

•W

ith r

egio

nal b

urea

u te

chni

cal s

taff

, pr

epar

e a

coun

try

offic

e po

st-le

arni

ng e

vent

act

ion

plan

.

•W

ith r

egio

nal b

urea

u te

chni

cal s

taff

, se

lect

sta

ffw

ho m

eet

the

crite

ria f

or f

urth

er t

echn

ical

ski

llde

velo

pmen

t.

Tab

le 1

4.W

FP R

espo

nsib

ilitie

s fo

r Po

st-L

earn

ing

Even

t Fo

llow

-up

81


•Pr

epar

e an

ann

ual r

evie

w o

f gl

obal

lear

ning

prog

ram

me

impl

emen

tatio

n.

OD

A,

HEA

DQ

UA

RTE

RS

CO

UN

TRY

OFF

ICE

(CD

, SU

PER

VIS

OR

AN

D L

EAR

NER

)

Post

-wo

rksh

op

pra

ctic

al a

pp

licat

ion

•Se

lect

lear

ners

for

for

mal

(su

perv

ised

) an

din

form

al (

dist

ance

coa

chin

g) o

n-th

e-jo

b le

arni

ngac

tiviti

es.

•Id

entif

y ac

tiviti

es t

hat

expo

se t

he le

arne

rs w

hoha

ve s

ucce

ssfu

lly c

ompl

eted

lear

ning

eve

nts

toas

sess

men

ts.

An

nu

al r

egio

nal

rev

iew

of

lear

nin

g

•C

ompl

ete

a re

gion

al e

valu

atio

n of

lear

ning

eve

nts

and

post

-lear

ning

app

licat

ion

of n

ew s

kills

and

know

ledg

e.

•U

pdat

e co

untr

y di

rect

ors

and

agre

e fo

llow

-up

for

the

follo

win

g ye

ar.

REG

ION

AL

BU

REA

U

Prepared by: Charisse Tillman, Jeremy Loveless, Jeff Klenk, and Greg Collins

First Edition - February, 2007© World Food Programme, Emergency Needs Assessment Branch (ODAN)

The opinions and views contained in this report reflect those of the authors, and do not necessarily reflect those ofthe World Food Programme.

United Nations World Food Programme Headquarters: Via C.G. Viola 68, Parco de’ Medici, 00148, Rome, Italy

This document has been produced with financial assistance from the European Union. The views expressed herein canin no way be taken to reflect the official opinion of the European Union.

Printed in June 2007Graphic Design: Energylink

www.wfp.orghttp://odan.wfp.org/repository