1 focus groups as assessment tools marie l. radford, ph.d., associate professor, rutgers university...

1

Focus Groups as Assessment Tools

Marie L. Radford, Ph.D., Associate Professor, Rutgers University

mradford@scils.rutgers.edu

LAMA/MAES Using Measurement Data for Library Planning & Assessment Committee

“Got Data, Now What? Analyzing Focus Groups and Usability Study Results”

ALA, Chicago, June 26, 2005

2

Participants

How many have conducted FG & have data to analyze?

How many are planning FG?

How many are considering FG?

How many are skeptical of value of FG?

AGENDA

Phase 1: Planning Phase 2: Conducting Focus Groups Phase 3: Analysis of Focus Group Data Phase 4: Reporting Results Using Focus Group Results in Strategic

Planning

4

What is a Focus Group?

“A focus group is an in-depth, face-to-face interview of a group of 8 to 12 people representing some target group and centered on a single topic.”

(Zweizig, Johnson, Robbins, & Besant, 1996)

5

Advantages of Using FG for Strategic Planning In-depth understanding of experiences &

meanings. Better data – synergy created:

Comments stimulate others Unexpected insights More complete info. Elicit strong & common opinions Less inhibiting, less formal.

Access needs of under-served or underrepresented groups.

6

Disadvantages Lots of planning & administration time!

Skilled moderator required Participants may be too quiet or too outspoken Analysis, summarization & interpretation of responses may be

difficult Tape transcription - time consuming &

costly

7

Limitations

Difficult to generalizeMay not be representative Analyzes perceptions, not factsSubjective analysis (reliability

requires more than 1 person)Compensate by combining methods

(surveys/observation)

8

Phase I: PlanningCrucial phase!Communicate to stakeholdersDetermine role of FG in overall

assessment What info. is needed? Identify key issues

Choose moderator & assistant (see handout)

9

More Planning Identify target group(s)

Decide # of FG, 3-4 per targeted population Identify pool

Recruit volunteers Plan 8-12 per group (over-recruit & remind) Offer reward (e.g., free copycard, food) Seek representative members Form homogeneous groups:

Academic (e.g., faculty, undergrads, grad. students) Public (e.g., adults, teenagers, non-users)

10

Yet More Planning… Design FG guide, include:

Introduction (purpose, ground rules) Ice breaker or warm-up set of questions Relevant major/probe questions (4-5) Summary or closing

Consider modified FG (see handout) Plan schedule (1.5-2.5 hours each)

When & Where Choose notes or tape recorder Choose method of data analysis

11

Phase II: Conducting Focus Groups

Bring/check supplies Flip chart, markers, masking tape, etc. If taping check equipment & have back-up

Start on time, don’t waste time Begin by creating safe climate Obtain permission to use info. & if taping Help quiet people talk, limit talkative

people Thank them!

12

Phase III: Analysis Listen/transcribe tapes Review notes Ways to analyze data:

Use data to identify categories Repeated reading/listening

Code data into pre-determined categories Use data as basis for summary statements

“capture the essence” Modified FG already identifies priorities

13

Analysis, Cont. Compare across focus

groups Compile & summarize Look for trends or

problems to study Do not overgeneralize from

results!

14

Phase IV: Reporting Results

Parts of Report Bulleted executive summary Statement of purpose

List of issues Method (keep it brief!)

How data collected How analyzed Describe participants

15

Parts of Report, Cont. Summary of findings:

What said each issue Consensus? Comparison of groups

Sample quotes (anonymous) Few, well chosen, representative

Interviewer’s impressions Recommendations:

Short term (low hanging fruit) Long term

16

Library Strategic Planning ExamplesAcademic

Rutgers University Libraries Major part of strategic planning

initiative. Focus groups of students & faculty. 3 Campuses. 3 Moderators, 3 Assistants. Recruitment difficulties. Results extremely useful.

17

Public Library Example

Brooklyn Public Visual & Performing Arts Library Inform building project. Large urban library system. FG of community groups, artists, art educators. Report used in informing decision making, fund

raising, & strategic planning.

18

Presenting to Administrators

For written or oral presentation Brevity is important Put your best foot forward! Highlight:

Positive results Strategic directions Major concerns

Begin & end with quotation Bulleted executive summary may be

all that is read (!)

19

Using Results for Strategic Planning

Allow sufficient time to review results

Use missions, goals for guidance

Look at short & long term recommendations

Prioritize recommendations

20

Using Results Cont.

Gain consensus Establish task forces or assign

responsibilities Establish objectives, target dates Establish regular (6 mo. or annual) review

process Create assessment plan

21

22

For More Information…

See list of books & articles.

Questions??

Thank you!