needs assessment 101 - ucanr.edu · katherine webb-martinez, associate director program planning...

Needs Assessment 101 Katherine Webb-Martinez, Associate Director Program Planning and Evaluation Margaret Lloyd, Small Farms Advisor

“A systematic set of procedures undertaken for the purpose of setting needs-based priorities and making decisions about program design/improvement and allocation of resources.”

(Altschuld & Kumar, 2010)

What is a needs assessment?

What it can be Different potential uses

Definitely • Performance appraisal • Developing your program: from

goal setting to evaluation Maybe • Commissioned study • Journal article • Justify requests for funding

Choice of scale and scope Informal vs. formal

approaches

What do we mean by “need?” Definition: the measurable discrepancy between “what is” or the present state for group or situation of interest and the “what should be” or desired state (Witkin & Altschuld, 1995)

needs not solutions

Why do needs assessment?

Improve services and products for clientele Strategically use limited resources For better program planning and evaluation Part of job description

Developing your program

1. Assess situation and needs 2. Set priorities 3. Develop and implement methods/activities

(services and products) 4. Evaluate: measure outcomes

Program Development Model

University of Wisconsin Extension

When? How Often?

Generally within your first year At the start of project/programs; but also… Iterative and ongoing, given dynamics of

changing situations

Things to consider before getting started

1. Find out what is already known or available

2. Determine if you need more information from stakeholders

3. Clientele readiness for assessment

4. Political aspects of the activity

How? Basic Steps: 1. Make sure you understand context/current state 2. Identify needs 3. Prioritize needs 4. Consult ANR Strategic Vision to frame the issues

Context is important Extension programs exist within a situation or environment that is often complex and changing.

The more we know about the situation, the more solid the foundation of our program.

Examine the available assets and relevant research, knowledge and experience to better understand the situation.

Involving others helps to build a better understanding of the context or situation, as well as buy-in to the program

First steps for new advisors

Get out and introduce yourself Develop relationships Define your clientele Get the lay of the land

This informs who and what to ask about needs

How Identify and Prioritize Needs

Possible data collection methods: Review the literature, build on existing needs assessments Interview, e.g. with key informants: meetings with experts, farm calls with growers, etc. Survey Focus groups Mixed-method approach, including attending meetings and tours with key informants

Document/Literature Review

Content analysis of existing information, secondary data

Little to no participant burden

Use it for: Don’t recreate

the wheel Mixed methods

approach

Who to ask?

1. Determine the sampling frame: compile a comprehensive and accurate list of your target audience

2. Choose sample type: • Convenient sample • Stratified sample (diverse audience) • Random sample

Best practices

Use existing lists & send introductory letter to gauge interest

Plan to contact more Tip: Start with the number of interviews you plan to do, and add 5-7 people for outright rejection and another 5-8 people to cover other withdrawals during process

Protecting Human Subjects

“A human subject is as a living individual about whom an investigator conducting research obtains (1) data through intervention or interaction with the individual; or (2) identifiable private information.”

Institutional Review Board = IRB

Core Principles: Respect, Beneficence, Justice

Do you need IRB approval?

Do the human subjects represent a vulnerable population (e.g., youth, institutionalized individuals, or others whose participation may be considered involuntary)?

Is it likely that participants’ identities and/or contact information can be linked to their responses?

Will evaluation results be published (in peer-reviewed journals)?

Adapted from Ellen Taylor-Powell’s Building Capacity in Evaluating Outcomes

Interviews USE WHEN…

personal contact is desirable sample group is smaller sample group is unlikely top respond to a written survey you are not sure what is most important to potential respondents

Best practices •be consistent and neutral with all interviewees •use probing questions

Elizabeth will share her approach using interviews later…

Surveys USE WHEN… need to collect standardized information from large numbers of individuals face-to-face meetings are not feasible privacy is important or independent opinions and responses are needed time is a serious issue

Best practices •be clear as to the purpose & communicate it •keep focused: only ask what you need to •pilot test •design, design, design!

Margaret will share her approach using a survey later…

Ensuring Culturally Competent Needs Assessment

Cultural Competence: “a set of congruent behaviors, attitudes, and policies that come together in a system, agency, or among professionals and enables that system, agency, or those professionals to work effectively in cross-cultural situations.”

[Cowles (2005)]

Ensuring Culturally Competent Needs Assessment

Cultural Considerations Group Discussion How might participant characteristics affect your needs assessment? • Language • Age • Abilities: mental, physical, social • Male-female interactions, communication styles, family relationships, decision-making styles • Attitudes to conflict • Concept of time • Approaches to knowing and ways of knowing

How might you make the following data collection methods more culturally sensitive?

• Written questionnaire

• Mailed survey

• Observations

• Interviews

Borrowed from Building Capacity in Evaluating Outcomes, Univ. of WI - Extension

Tips for Writing the Questions Work with clientele and/or other stakeholders to develop the questions

Ask both open-ended and close-ended questions

Don’t ask yes or no questions

Yvonne Nicholson 2012

Yvonne Nicholson 2012

Yvonne Nicholson 2012

Survey Prioritizing Questions Example

LISTED BELOW ARE PUBLIC ISSUES RECENTLY IDENTIFIED IN THE COUNTY. RANK THE MOST SERIOUS ISSUES BY PLACING THE MOST SERIOUS IN BOX 1, THE SECOND MOST SERIOUS IN BOX 2, AND THE THIRD MOST SERIOUS IN BOX 3. A. ………

B. ………

C. ………

D. ……….

E. ……….

F. ……….

G. ……….

MOST SERIOUS ISSUE

2ND MOST SERIOUS ISSUE

3RD MOST SERIOUS ISSUE

Adapted from Nancy Ellen Kiernan’s A Ranking Question for a Needs Assessment Tipsheet#26, Penn State Cooperative Extension

Using an Evaluation for a Needs Assessment Example

TOPICS XXXX

YYYY

To plan for the future, circle the extent to which you would like additional training in each topic below.

No Some In Depth

Training Needed Training Needed Training Needed

No Some In Depth

Training Needed Training Needed Training Needed

Nancy Ellen Kiernan’s Using an Evaluation for a Needs Assessment Tipsheet#87, Penn State Cooperative Extension

Group Assessment: Focus Groups & Forums

Don’t know what questions to ask Or want to know more from survey

responses Foster trust and relationship-building Piggy back off existing meetings For mixed methods approach Have time/personnel to analyze

Best Practices • 4-10 people in

person

Use when…

Final thoughts on choosing methods Consider purpose, participants, and resources available when selecting you method • The goal is to obtain trustworthy, authentic and credible evidence

Often a mix of methods is preferred or needed given diverse clientele

Will share Michelle ‘s approach using mixed methods: survey & focus group

Susie will share mixed methods approach: literature review & key informant interviews

Mixed Methods : Lynn Schmitt-McQuitty Mapping the Community Assets of Seaside project

Phase 1 - Interview community members, organizations and businesses to assess community perceptions of youth and youth opportunities in Seaside.

Phase 2 - Creative arts to educate others on how the community is viewed by youth. Youth were asked to answer the following questions in their art projects: How do you see your community? Where do you feel safe and/or valued? What are your hopes and goals?

Priority Setting Filters To help frame the needs/issues: ANR Strategic Vision Your job description Any relevant mandates Resources available Other agencies/organization Local perspectives and dynamics

Next Steps After Needs Assessment Developing Focused, Relevant Programs

UCCE Needs Assessment Resources http://ucanr.edu/sites/CEprogramevaluation/Needs_Assessment_Resources/