an adventure in assessment
DESCRIPTION
An Adventure in Assessment. Stephen “Gavin” Weiser, M.Ed [email protected]. A Lil About Me. Gavin Weiser Residential Experiential Education Facilitator Princeton-Blairstown Center Recent M.Ed from U. South Carolina. Let’s Hear About You. - PowerPoint PPT PresentationTRANSCRIPT
An Adventure in Assessment Stephen “Gavin” Weiser, [email protected]
A Lil About Me
Gavin Weiser
Residential Experiential Education Facilitator Princeton-Blairstown Center
Recent M.Ed from U. South Carolina
Let’s Hear About You
Find someone from a similar type of program as folks are talking.
Outline
What is Assessment
Purposes of Assessment
Some Pitfalls
Methods
Learning Outcomes - Creation
Tool Selection & Creation
What is assessment?
A. What we do to people we don’t like
B. An old wooden ship
C. Something we all hate and loathe
D. A sustainable process of collecting data to improve upon services & programs
Purposes
To collect data in order to better understand what we are doing
What can this look like?
Pitfalls
“teaching to the test”
The politics
Fatigue – particular to certain methods
Some bad ways to use assessment.
Methods
Qualitative vs. quantities
Mixed Methods
Literature review
Learning Outcomes
Why are they important?
What are they? Measureable & meaningful
Examples: To gain leadership abilities.
Good or bad?
Tool Selection
Based upon what we are trying to measure
Different Types: Surveys Focus Groups Interviews
Structured vs. Unstructured Protocols Field Observation
Create a“Culture of
Assessment”What does this look like?
Vital to it’s and the programs success!
Example Project
Background of Outdoor Recreation at Carolina
Opened Fall 2008, Director: Katie Coley
Information from Katie, annual report, focus groups with current student leaders
Annual Report Strengths Weaknesses
5 year plan
Programs of Outdoor Recreation
Climbing Wall
Clinics
Adventure Trips
Bike Shop
Student Leadership
Outdoor Leadership Training
Focus Groups
8 Students Leaders; 2 groups of 4
Methodology
Provided information based on experiences and goals with outdoor recreation
Information received assisted in guiding what learning outcomes should be created
Previous Assessments in Outdoor Recreation
Office had “learning outcomes” but they were immeasurable
Same tool was used to assess all programs
Tool focused on satisfaction
Purpose of Assessment
New office which allowed for flexibility
To create a process that can be used for future assessments within Outdoor Recreation
Outdoor Recreation programs nationally have a need for quality assessments
Provide feedback on some of the learning outcomes related to the Rock Wall
Methodology
Students that have completed climbing the rock wall
University 101 and walk-up participants
A qualitative assessment was distributed post wall climb
2 week period in November – only one week is included in this analysis
Learning OutcomesStarted from focus group, Katie’s feedback and
goals
Took two goals: Teambuilding and Trust and developed outcomes from them
Created measurable outcomes
U101 Outcomes
Recognize individual’s strengths within a working group.
Understand how to apply strengths in an existing challenge.
Demonstrate a willingness to rely on others during a challenge.
Participants will learn technical skills related to the activity in which they participate.
Individuals’ Outcomes
Demonstrate an increase in willingness to go beyond limitations and boundaries.
Participants will learn technical skills related to the activity in which they participate.
ToolSelected a qualitative instrument based on:
feasibility, rationale, design, research questions
Technically mixed method
Local instrument – developed by us
Open-ended questionnaire
Challenge: probing but not leading and receiving the information that was necessary
Katie’s feedback to make improvements to the tool Created a second tool
Analysis
Used a coding procedure “Focused coding”
Codes were guided by outcomes and data
U101: coded as whole system
Individual: coded by question
Used SPSS to develop codebook and pie charts for representations
Results
Solid process of developing a survey based on targeted outcomes – Major Result
Most applicable to outcomes will be discussed
Others are informative but not directly tied to outcomes
Individual: Expectations and Challenges
Technical Skills
Cohesieveness
Team Support
Technical SkillsCohesievenessTeam Support
Challenges
Reasons For Visiting
40%
46.70% 41%
5.90%
52.9%
13.3%
Skills Learned
Individual U101
Tech-nical Skills
Co-hes-ieve-ness
Technical SkillsCohesievenessTeam SupportCommunica-tion
U101: How were challenges overcome?
Interesting to note, this came as a surprise
Technical SkillsCohesievenessTeam SupportCommunication
Points of Improvement
Tool did not always provide needed data
Not a large sample size
Difficult to know if technical skills were learned during activity
Time constraints
Strengths
Informed by focus group data
Process for developing outcomes and tool was well documented and could be replicated
Tool design – well-sequenced
Informed support from whole office
Code and analysis were manageable and clear
Recommendations for Future
Use process to develop a comprehensive tool for each activity
Connect outcomes to Campus Recreation outcomes
Use satisfaction data and skills data to publicize the wall, attract non-climbers
Be intentional in educating student participants about outcomes
Expand office staff
Partner, Pair, Share
Find that special someone from earlier
Develop outcomes and the beginnings of a tool.
