Download - 2005 CELT Conference1 Strategies from the Assessment Trenches: A Self-Help Clinic presented by AURA
2005 CELT Conference 1
Strategies from the Strategies from the Assessment Trenches: Assessment Trenches:
A Self-Help ClinicA Self-Help Clinic
presented by AURApresented by AURA
2005 CELT Conference 2
Topics & Format
Topics
• Program Mission, Goals & SLO Statements
• Course Alignment Matrix
• Developing & Implementing an Assessment Plan
Format
• We introduce the topic.
• Groups discuss progress & problems.
• Debrief & share strategies.
2005 CELT Conference 3
Your Program Mission, Goals & Student Learning Outcome Statements
Mission: a holistic view of the values and philosophy of the department/program
Goals: general statements about knowledge, skills, attitudes, and values expected of graduates
SLOs: clear, concise statements that describe how students can demonstrate their mastery of program goals
Courtesy of Dr. Mary Allen
2005 CELT Conference 4
Group Assignment
- SLOs
• Share your program’s SLOs.
• What challenges did you face in crafting your SLOs?
• How did you overcome the challenges? What lingering concerns remain?
• Prepare to report on a few key ideas, problems, or strategies.
2005 CELT Conference 5
The Outcomes/Course Alignment Matrix
Stage 1 – Ensuring a Cohesive Curriculum
In which courses are which outcomes introduced, practiced, mastered?
Stage 2 – Developing an Assessment Plan
What student artifacts can be used to measure progress?
Course SLO 1 SLO 2 SLO 3 SLO 4
151 I I
177 P I
201 P I
277 P P
340 M - oral presentation
M
477 P - term paper
2005 CELT Conference 6
Group Assignment – The Matrix
• Create a draft of your program’s Outcomes/Course Alignment Matrix.
• If your program has a (draft) matrix, share it.
• Prepare to report on a few key ideas, problems, or strategies.
2005 CELT Conference 7
Developing/Implementing an Assessment Plan
An assessment plan should be meaningful, manageable, and sustainable.
For each SLO, ask:
• How and where will the SLO be assessed?
• How will you collect, organize, and analyze the data?
• When and how often will this be done? By whom?
• Who will reflect on the results?
• What will ensure that results are used to implement program changes to improve student learning?
2005 CELT Conference 8
Elements of An Effective Assessment Plan
What are the building blocks needed ? Program Mission Goals Student Learning Outcomes (SLOs) Course Alignment Matrices Plan to Assess Individual SLOs … such as …
2005 CELT Conference 9
Assessment of SLOs
• Direct Assessment Methods:– Published Tests– Local Exams– Embedded Assignments & Activities– Portfolios
• Indirect Assessment Methods:– Surveys– Interviews– Focus Groups
2005 CELT Conference 10
Assessment of SLOs
• Direct Assessment Methods are Essential:
– Examine Student Work …
• Indirect Assessment Methods are Supportive:
– Examine Student / Alumni Perceptions …
• Effective Assessment Programs Use Multiple Measures … Triangulate
2005 CELT Conference 11
Measurement Using Rubrics
• What is a rubric?– “Explicit schemes for classifying products or
behaviors into categories that vary along a continuum” Mary Allen, WACS/AAHE workshop, Sept. 2004
– Holistic (along one dimension)– Analytic = traits or characteristics of the
product or behavior – Similar to grading criteria sheets that some
faculty use
2005 CELT Conference 14
Steps To Develop a Rubric
• Define the SLO and pick the work sample• Involve the potential evaluators, faculty who assign the
student work, external stakeholders, (if possible)• Brainstorm the traits or characteristics of the goal that you
want to measure– Specific traits lead to actionable results
• Determine the number of discrete categories that you want to have (generally 3-6)
• Brainstorm the descriptions of each category for each trait• Refine the traits/descriptions avoid “and” clauses
2005 CELT Conference 15
The Next Steps
To collect, organize, and analyze the data …
• When and how often will this be done?
• By whom?
• Who will reflect on the results?
• How will they measure success?
• What will ensure that results are used to implement program changes to improve student learning?
2005 CELT Conference 16
A Caution:
Assessment Plans in Early Stages are Good at Collecting Data … but Lack a Plan for How to:
Organize, Summarize the Data Systematically Reflect on Results Ensure that Results are used to Effect Change and Improve Student Learning• Effective Assessment Programs are Both Manageable and Sustainable
2005 CELT Conference 17
Evaluation and Use of Data for Continuous Improvement
•Assessment Data Must be Evaluated Systematically, Periodically
– Recommend Annual Collection / Assessment / Action Plan
•Modify Curriculum and Pedagogy to Improve Student Learning …
– Typical Actions:• Modify Content, Sequencing of Course(s)• Modify Pedagogical Approaches
2005 CELT Conference 18
Evaluation and Use of Data for Continuous Improvement
• The Entire Process (Goals, Outcomes, Assessment Measures) Must be Evaluated Periodically– Recommend Five Year Re-evaluation to
synchronize with FYR process:• Are we measuring reasonable work samples?
• Is our process meaningful? Are we better informed about what our students are learning or not?
• Is our process sustainable? Can we be more efficient and still have valid measures?
2005 CELT Conference 19
The Assessment Plan Design Process
• Wherever you begin developing your Assessment Program will NOT be the ideal place to start …
• You Must Begin Somewhere …
• Iteration will save you ...
– Continuous Refinement of Methods– Consensus and Team Building– Alignment of Program Resources with Program
Goals and Faculty Values …
2005 CELT Conference 20
Group Assignment – Assessment
Planning
• Share your program’s SLO assessment plan: – What SLO(s) do you plan to
assess this year? – How do you plan to assess it? – What is the biggest hurdle you
face in the assessment process?
• Prepare to report on a few key ideas, problems, or strategies.