OverviewThe Building Blocks of Collecting Participant

Information

Constructing a Drug Court Participant Survey

Constructing a Drug Court Focus Group

Putting it all together

The Building Blocks of Collecting Participant Information

Why are we seeking feedback?

Audits

QA/CQI

Customer Feedback

Recipients of Survey Results

Management

Respondents

The Building Blocks of Collecting Participant Information

Ways to collect information

Surveys

Focus Groups

Other Methods

Nominal Group Technique

Interviews

Panels

The Building Blocks of Collecting Participant Information

Selecting your Participants

Defining your sample

Types of Sampling

Random Sample

Stratified Sample

Convenience Sample

Some Common Selection Errors

The Building Blocks of Collecting Participant Information

Some Common Selection Errors

Population Specification Error

Sample Frame Error

Selection Error

Non-Response Error

The Building Blocks of Collecting Participant Information

Frequency of Distribution

Cross-Sectional Surveys

Single time period

Longitudinal Surveys

Multiple and regular intervals

Measure changes in behavior, opinion, knowledge or attitude

The Building Blocks, cont’d

Distribution method

Who will:

Distribute survey

Collect the survey

Process the survey

Create Report

Distribute results

Data Collection Tool

Surveys

Pen and Paper

Scan forms

Survey Monkey and other cloud-based systems

Focus Groups

Handwritten notes

Audio

Video

The Building Blocks, cont’d

Survey question considerations

Length of survey

Reading level

Reliability index

The Building Blocks, cont’d

Type of questions

Open-ended

True false

Likert scale

Other

The Building Blocks, cont’d

Question Creation Pitfalls

Double Barreled Questions

Leading questions

Loaded Questions

Double Negative Questions

Absolute questions

Questions are worded for the audience

Categories are not exhaustive

Categories are not mutually exclusive

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

Target Population

Distribution method

Mode

Resources

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

Collaboration Research/Practitioner Relationship

Observing the Process

Additional Resources Within your organization

Research Articles

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

Key Question Areas

Intake/Getting Started

Overall Programming

Drug and Alcohol Tx

Cog Skills, Employment, and Education Programming

Case workers

Court Appearances

Post-release / Upon leaving

Facility

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

Tips for writing good questions

Consider Response Dimension

Question Wording

Question Structure

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

Tips for writing good questions

Consider Response Dimension

Checklists

True/False

Likert-Type Scales https://www.uc.edu/content/dam/uc/sas/docs/Assessment/likert-type%20response%20anchors.pdf

Question Wording

Question Structure

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

Tips for writing good questions

Consider Response Dimension

Question Wording

Complex words

Technical terms

Jargon

Double negatives

Shorthand

Question Structure

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

Tips for writing good questions

Consider Response Dimension

Question Wording

Question Structure

Double-Barreled Questions

Parallel structure

Time frames

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

“I feel welcomed by my caseworker and other clients in my program”

“I felt welcomed by my caseworker in my program.”

“I felt welcomed by other clients in my program.”

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

“Everyone really enjoys the Intensive outpatient program (IOP). Did the IOP program help you in your recent placement”

The Intensive Outpatient Program… Helped me stop using drugs/alcohol Helped me create a plan to keep drug/alcohol free after I

finish

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

“Do you think quitting smoking is a good or bad idea when trying to remain sober?”

Quitting smoking is important when trying to remain sober

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

What type of family did you grow up in? Mom as single parent

Dad as single parent

Both mom and dad

What type of family did you grow up in? Mom as single parent

Dad as single parent

Both mom and dad

Other _______

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

“What time of day do you most often consume alcohol?”

The time of day I struggle most with sobriety is: Morning (8am – 11:59am)

Afternoon (12:00 pm – 3:59 pm)

Evening (4:00 pm – 7:59 pm)

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

“Are you always depressed after you start drinking?”

“Alcohol/Drug use depresses my mood”

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

“Was your caseworker assiduous in delineating the program rules to you?”

“My caseworker clearly explained program rules and expectations.”

Design Survey Process

Develop Questions

Test & Train

Collect Data

Analyze Data

Lead time

Review with collaborators

Pilot with peers

Get final approval

Design Survey Process

Develop Questions

Test & Train

Analyze Data

Troubleshooting measures

Updating forms

Colored Forms

Data Storage & Entry

Collect Data

Design Survey Process

Develop Questions

Test & TrainCollect Data

Data Analysis

Means

Frequencies

Response Rates

Ongoing Monitoring

Continually monitor surveys to ensure fidelity

AnalyzeData

Setting up the focus group process

Preparing for the Focus Group

Clearly define purpose

Choose moderator and assistant

Set up time and location

Participant Selection

Optimal group size

Convenience sample

Consider demographics

Staff selection

Develop clearly defined inclusion/exclusion criteria

The Building Blocks of Collecting Participant Information

Common issues faced when picking focus group membership…

1. Offering pizza and soda and then selecting the first 8 clients that volunteer to do the focus group. Selection Error

2. Focus group on Cognitive Skills Sessions and selecting the first 8 respondents that volunteer. Population Specification Error.

3. Focus group containing all males when the program has both males and females. Sampling Error

4. Focus group concerning Drug Court orientation and only including those clients who are about to be successfully graduated. Non-Response Error

5. Importance of focus group input is based on who talks the most. Squeaky wheel error.

Question Development

Collaboration

Understanding Participants

Consider purpose

Open ended

Simple and Short

One Idea

Without Judgement or Presumption

Length

Revision

Categorizing and Ordering Questions

Demographic

General

Specific

Closing

Developing a Consent Form Purpose

Disclaimer if for a grant

Voluntary

Confidentiality

Disclaimer of confidentiality if expose harm to others or self

Incentive

Any known risks

Recording

Conducting the Focus Group

Set up

Welcome

Introductions

Purpose and Rules

Name tents

Begin recording

Read consent

Ask questions

Analyzing the Data

Transcribe

Assign Numbers

Manual vs. Excel

Create Themes

Summarize Findings

Qualitative vs. Quantitative

Scenario Your drug court has just received funding to implement a new, groundbreaking approach in treatment that was found to be effective with residential male offenders in rural Texas diagnosed with an opiate use disorder. We expect that no more than 15% of our entire client population will be eligible to participate in this group. In order to receive and maintain this funding, the grantor requires that participants in the new treatment group provide input in an effort to help evaluate the program. We need this information within the next six months. What would be the best way to collect this information?