college of health sciences assessment workshop
DESCRIPTION
College of Health Sciences Assessment Workshop. Dr. Marsha Watson Director of Assessment Dr. Kenny Royal Assistant Director of Measurement & Analysis Dr. Julie F. Johnson Assessment Specialist. The Provost’s LEARNING Initiative Dual Track Implementation Strategy. - PowerPoint PPT PresentationTRANSCRIPT
College of Health Sciences Assessment Workshop
Dr. Marsha WatsonDirector of Assessment
Dr. Kenny RoyalAssistant Director of Measurement & Analysis
Dr. Julie F. JohnsonAssessment Specialist
The Provost’s LEARNING Initiative Dual Track Implementation Strategy
Completion DatesNot actively engaged in program level assessment
Actively engaged in program level assessment
Sept 2009Program level student learning outcomes revised and/or updated
Dec 2009 Assessment strategy in place
Jan-Mar 2010 Assessment strategy implemented
April 2010Assessment results available for faculty reflection and action
May 2010First cycle completed and improvement plans submitted
At least one cycle completed and improvement plans submitted
September 2010 First annual LEARNING Improvement awards announced
May 2011 Two cycles completed At least two cycles completed
August 2011 SACS Compliance Audit begins
September 2011 Second annual LEARNING Improvement awards announced
How are your stated student learning outcomes appropriate to your mission, programs, degrees, and students?
What evidence do you have that students achieve your stated learning outcomes?
In what ways do you analyze and use evidence of student learning?
How do you ensure shared responsibility for student learning and for assessment of student learning?
How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?
In what ways do you inform the public and other stakeholders about what and how well your students are learning?
Six Fundamental Questions
Review of Assessment Basics University Assessment
◦ Campus-wide assessment of student learning at the program level (e.g., General Education)
◦ University assessment is the primary charge of the Office of Assessment
University assessment is separate and distinct from evaluation of teaching effectiveness◦ Evaluation of teaching effectiveness is the responsibility of
departments/colleges Assessment data are analyzed and reported only
in the aggregate You can’t assess everything all the time!
◦ Plan for assessment that is practical, given current time and resource constraints
◦ Assess 1 or 2 outcomes per year
Assessment vs Evaluation◦ Assessment requires us to “take a step back”
from the interaction between student and teacher◦ Grades are evaluations, generally not used for
assessment Team approach to evaluation
◦ Essentially a juried assessment in that more than one individual is scoring/evaluating
◦ A periodic, objective validation process of some kind required to ensure validity and reliability
Review of Assessment Basics
Review of Assessment Basics Three levels of assessment
◦ Course◦ Program
Undergraduate majors/programs General education program Graduate majors/programs
◦ Institutional Course, Program, and Institutional
outcomes should be aligned, but are not identical
Review: Program Level Assessment
Focused on curricular, environmental improvement
Formative and Summative, Direct and Indirect methods
Curriculum mapping, program improvement
Review: Program Outcomes
Focus on broad skills developed over time◦ Not restricted to a single course or learning
experience Demonstrate acquisition of specific
disciplinary/professional knowledge and skills necessary after graduation◦ Ask: “What makes a graduate of the
program able to function and learn in a specific discipline/profession after the degree?”
Measurable◦ Confirmable through evidence
Measuring Learning Outcomes
Measures must be appropriate to outcomes◦ Avoid cumbersome data-gathering◦ Use both direct and indirect methods
Indirect methods measure a proxy for student learning
Direct methods measure actual student learning
◦ “Learning” = what students know (content knowledge) + what they can do with what they know
Defining Evidence Information that tells you something directly
or indirectly about the topic of interest Evidence is neutral -- neither “good” nor
“bad”◦ Requires context to be meaningful
Two types of assessment evidence◦ Direct (“authentic”) and Indirect
Best practice calls for multiple methods
Direct Evidence Students show achievement of learning goals
through performance of knowledge, skills:◦ Scores and pass rates of licensure/certificate exams◦ Capstone experiences
Individual research projects, presentations, performances Collaborative (group) projects/papers which tackle complex
problems◦ Score gains between entry and exit ◦ Ratings of skills provided by internship/clinical
supervisors◦ Substantial course assignments that require performance
of learning◦ Portfolios
Indirect Evidence Indirect methods measure proxies for
learning◦ Data from which you can make inferences about
learning but do not demonstrate actual learning, such as perception or comparison data
◦ Surveys Student opinion/engagement surveys Student ratings of knowledge and skills Employers and alumni, national and local
◦ Focus groups/Exit interviews◦ Course grades◦ Institutional performance indicators
Enrollment data Retention rates, placement data Graduate/professional school acceptance rates
Mapping Outcomes for Program-Level Assessment Create a visual map:
◦ Lay out program courses and learning outcomes (competencies) on a grid Refer to examples (Handouts)
◦ Identify the courses at which each competency is: Introduced Reinforced Emphasized
Basic Program Map Template
Outcomes Course #1; Baseline Assessment
Course #2 Course #3;Mid-Program Assessment
Course #4 Course #5;Capstone Assessment
Outcome 1 I R R E R
Outcome 2 R R E
Outcome 3 I E R E
Outcome 4 E R R
I = Outcome is introduced; baseline, formative assessmentR = Outcome is reinforced; formative assessmentE = Outcome is emphasized; summative assessment
Finding Evidence: An Evidence Inventory
Lets you discover the evidence you already have, such as:◦ Institutional Research data◦ Student Life data◦ Exit Surveys (seniors)◦ Alumni Surveys
Start with the obvious … but don’t stop there
Finding Evidence: Perils and Pitfalls Institutional history
◦ “We’ve already done that, and it didn’t tell us anything!”
Territory; Politics◦ Fighting for scant resources
Institutional policy/culture about sharing information◦ “I don’t want somebody ‘policing’ my
classrooms!”
Fundamental Question #1: Appropriate Evidence
Does the evidence address student learning issues appropriate to the institution?
Does the evidence tell you something about how well the institution is accomplishing its mission and goals?◦ The questions you have about student learning
should guide your choice of appropriate existing evidence and identify gaps where a new type of evidence might be needed
Assisting Academic Departments: Addressing Common Barriers
“This is a lot of work!”◦ Use some sort of evidence inventory to help
faculty understand how existing academic practices yield evidence
◦ Keep expectations reasonable, given limited time and resources
Remember: it is not necessary to gather all the evidence all of the time
Assisting Academic Departments: Addressing Common Barriers
“How do I know you won’t use this against me?”◦ Be consistent and firm in the message that
assessment is not faculty evaluation, that results will only be reported in the aggregate
◦ Remember: Assessment results will link to allocation of resources, ideally through the strategic planning process
Assessment is only a means to an end◦ The purpose of assessment is continuous
improvement of student learning The assessment cycle is complete when
assessment results have been used successfully for evidence-based decision making
Completing an Assessment Cycle
Articulate expectations in the form of student learning outcomes
Measure achievement of expectations Collect and analyze data Use evidence to improve learning Assess the effectiveness of improvement
The Assessment Cycle
Unit Assessment Plan Template (Handout)◦ Use this template as a foundation for your unit
assessment plan, revising and reshaping as necessary
Unit Assessment Plans