needs assessment 101 - ucanr.edu · katherine webb-martinez, associate director program planning...
TRANSCRIPT
Needs Assessment 101 Katherine Webb-Martinez, Associate Director Program Planning and Evaluation Margaret Lloyd, Small Farms Advisor
“A systematic set of procedures undertaken for the purpose of setting needs-based priorities and making decisions about program design/improvement and allocation of resources.”
(Altschuld & Kumar, 2010)
What is a needs assessment?
What it can be Different potential uses
Definitely • Performance appraisal • Developing your program: from
goal setting to evaluation Maybe • Commissioned study • Journal article • Justify requests for funding
Choice of scale and scope Informal vs. formal
approaches
What do we mean by “need?” Definition: the measurable discrepancy between “what is” or the present state for group or situation of interest and the “what should be” or desired state (Witkin & Altschuld, 1995)
needs not solutions
Why do needs assessment?
Improve services and products for clientele Strategically use limited resources For better program planning and evaluation Part of job description
Developing your program
1. Assess situation and needs 2. Set priorities 3. Develop and implement methods/activities
(services and products) 4. Evaluate: measure outcomes
Program Development Model
University of Wisconsin Extension
When? How Often?
Generally within your first year At the start of project/programs; but also… Iterative and ongoing, given dynamics of
changing situations
Things to consider before getting started
1. Find out what is already known or available
2. Determine if you need more information from stakeholders
3. Clientele readiness for assessment
4. Political aspects of the activity
How? Basic Steps: 1. Make sure you understand context/current state 2. Identify needs 3. Prioritize needs 4. Consult ANR Strategic Vision to frame the issues
Context is important Extension programs exist within a situation or environment that is often complex and changing.
The more we know about the situation, the more solid the foundation of our program.
Examine the available assets and relevant research, knowledge and experience to better understand the situation.
Involving others helps to build a better understanding of the context or situation, as well as buy-in to the program
First steps for new advisors
Get out and introduce yourself Develop relationships Define your clientele Get the lay of the land
This informs who and what to ask about needs
How Identify and Prioritize Needs
Possible data collection methods: Review the literature, build on existing needs assessments Interview, e.g. with key informants: meetings with experts, farm calls with growers, etc. Survey Focus groups Mixed-method approach, including attending meetings and tours with key informants
Document/Literature Review
Content analysis of existing information, secondary data
Little to no participant burden
Use it for: Don’t recreate
the wheel Mixed methods
approach
Who to ask?
1. Determine the sampling frame: compile a comprehensive and accurate list of your target audience
2. Choose sample type: • Convenient sample • Stratified sample (diverse audience) • Random sample
Best practices
Use existing lists & send introductory letter to gauge interest
Plan to contact more Tip: Start with the number of interviews you plan to do, and add 5-7 people for outright rejection and another 5-8 people to cover other withdrawals during process
Protecting Human Subjects
“A human subject is as a living individual about whom an investigator conducting research obtains (1) data through intervention or interaction with the individual; or (2) identifiable private information.”
Institutional Review Board = IRB
Core Principles: Respect, Beneficence, Justice
Do you need IRB approval?
Do the human subjects represent a vulnerable population (e.g., youth, institutionalized individuals, or others whose participation may be considered involuntary)?
Is it likely that participants’ identities and/or contact information can be linked to their responses?
Will evaluation results be published (in peer-reviewed journals)?
Adapted from Ellen Taylor-Powell’s Building Capacity in Evaluating Outcomes
Interviews USE WHEN…
personal contact is desirable sample group is smaller sample group is unlikely top respond to a written survey you are not sure what is most important to potential respondents
Best practices •be consistent and neutral with all interviewees •use probing questions
Elizabeth will share her approach using interviews later…
Surveys USE WHEN… need to collect standardized information from large numbers of individuals face-to-face meetings are not feasible privacy is important or independent opinions and responses are needed time is a serious issue
Best practices •be clear as to the purpose & communicate it •keep focused: only ask what you need to •pilot test •design, design, design!
Margaret will share her approach using a survey later…
Ensuring Culturally Competent Needs Assessment
Cultural Competence: “a set of congruent behaviors, attitudes, and policies that come together in a system, agency, or among professionals and enables that system, agency, or those professionals to work effectively in cross-cultural situations.”
[Cowles (2005)]
Ensuring Culturally Competent Needs Assessment
Cultural Considerations Group Discussion How might participant characteristics affect your needs assessment? • Language • Age • Abilities: mental, physical, social • Male-female interactions, communication styles, family relationships, decision-making styles • Attitudes to conflict • Concept of time • Approaches to knowing and ways of knowing
How might you make the following data collection methods more culturally sensitive?
• Written questionnaire
• Mailed survey
• Observations
• Interviews
Borrowed from Building Capacity in Evaluating Outcomes, Univ. of WI - Extension
Tips for Writing the Questions Work with clientele and/or other stakeholders to develop the questions
Ask both open-ended and close-ended questions
Don’t ask yes or no questions
Yvonne Nicholson 2012
Yvonne Nicholson 2012
Yvonne Nicholson 2012
Survey Prioritizing Questions Example
LISTED BELOW ARE PUBLIC ISSUES RECENTLY IDENTIFIED IN THE COUNTY. RANK THE MOST SERIOUS ISSUES BY PLACING THE MOST SERIOUS IN BOX 1, THE SECOND MOST SERIOUS IN BOX 2, AND THE THIRD MOST SERIOUS IN BOX 3. A. ………
B. ………
C. ………
D. ……….
E. ……….
F. ……….
G. ……….
MOST SERIOUS ISSUE
2ND MOST SERIOUS ISSUE
3RD MOST SERIOUS ISSUE
Adapted from Nancy Ellen Kiernan’s A Ranking Question for a Needs Assessment Tipsheet#26, Penn State Cooperative Extension
Using an Evaluation for a Needs Assessment Example
TOPICS XXXX
YYYY
To plan for the future, circle the extent to which you would like additional training in each topic below.
No Some In Depth
Training Needed Training Needed Training Needed
No Some In Depth
Training Needed Training Needed Training Needed
Nancy Ellen Kiernan’s Using an Evaluation for a Needs Assessment Tipsheet#87, Penn State Cooperative Extension
Group Assessment: Focus Groups & Forums
Don’t know what questions to ask Or want to know more from survey
responses Foster trust and relationship-building Piggy back off existing meetings For mixed methods approach Have time/personnel to analyze
Best Practices • 4-10 people in
person
Use when…
Final thoughts on choosing methods Consider purpose, participants, and resources available when selecting you method • The goal is to obtain trustworthy, authentic and credible evidence
Often a mix of methods is preferred or needed given diverse clientele
Will share Michelle ‘s approach using mixed methods: survey & focus group
Susie will share mixed methods approach: literature review & key informant interviews
Mixed Methods : Lynn Schmitt-McQuitty Mapping the Community Assets of Seaside project
Phase 1 - Interview community members, organizations and businesses to assess community perceptions of youth and youth opportunities in Seaside.
Phase 2 - Creative arts to educate others on how the community is viewed by youth. Youth were asked to answer the following questions in their art projects: How do you see your community? Where do you feel safe and/or valued? What are your hopes and goals?
Priority Setting Filters To help frame the needs/issues: ANR Strategic Vision Your job description Any relevant mandates Resources available Other agencies/organization Local perspectives and dynamics
Next Steps After Needs Assessment Developing Focused, Relevant Programs
UCCE Needs Assessment Resources http://ucanr.edu/sites/CEprogramevaluation/Needs_Assessment_Resources/