wri:ng$an$exemplary$studentlearning$ summary$report ... · assump/ons(1....
TRANSCRIPT
Dr. Mary Herrington-‐Perry Office of Assessment & Accredita:on April 5th, 2017 Faculty Center for Teaching Excellence
Presented By
Wri:ng an Exemplary Student Learning Summary Report
Assump/ons 1. Your program already has learning outcomes, a curriculum
map, and an assessment plan. . . right? 2. You have assessed or will assess one or more of your
program’s learning outcomes in 2016-‐17. 3. You have analyzed or will analyze the results of these
assessments. 4. You have shared and discussed the results with the
program’s faculty (or will do so before the semester’s end). 5. And thus you will be ready to write the Student Learning
Summary Report– which is based on items 1-‐4– by your dean’s deadline . . . right?
Student Learning Summary Report Part One: Summary of Assessment Ac/vi/es
1. What learning outcome(s) did you assess this year? 2. What assignments or ac:vi:es did you use to
determine how well your students aXained the outcome?
3. What expecta:ons did you establish for students’ achievement of the outcome?
4. What were the actual results? 5. Who was responsible for collec:ng and analyzing the
results? How were they shared with the program’s faculty?
1. Learning Outcomes Student: What exactly do you expect from me?
Instructor: I want you to be able to demonstrate proficiency in using geological research tools, including Geographic Informa:on Systems and Global Posi:oning Systems.
Outcomes: Examples • Define the terminology of tax accoun:ng. • Ar/culate the historical assump:ons, central ideas, and dominant
cri:cisms of the key drama:c periods using illustra:ons from per:nent plays.
• Apply basic surveying techniques for construc:on layout and control.
• Iden/fy the mo:va:on behind behavioral issues and develop interven:ons to address inappropriate behaviors.
• Conduct a comprehensive and systema:c assessment of health and illness parameters in complex situa:ons.
• Communicate ideas clearly and coherently in wriXen work and oral presenta:ons.
Which is the stronger program-‐level learning outcome . . . and why?
A. Students will ar/culate and apply ethical principles to real world situa:ons.
B. Students will understand ethical principles in real world situa:ons.
Outcomes: More Examples Knowledge/Comprehension: Defines advanced prac:ce competencies for selected popula:ons. Analysis: Analyzes the significance of advanced nursing knowledge as it relates to selected popula:ons. Applica/on: Provides safe, cost-‐effec:ve, and culturally adap:ve advanced prac:ce nursing for special popula:ons. Evalua/on: Evaluates outcomes of advanced prac:ce nursing interven:ons, methods, or strategies. Crea/on: Synthesizes theore:cal frameworks used in the integra:on of knowledge from related sciences and humani:es, clinical knowledge, and nursing sciences as the founda:ons of advanced nursing prac:ce.
Exemplary Learning Outcomes . . . 1. Are specific, measurable, student-‐centered, program-‐
level outcomes that span mul:ple learning domains. 2. Directly integrate with Founda:onal Studies learning
outcomes or Graduate Learning Goals. 3. Reflect the most important results of program
comple:on. 4. Are consistent across different modes of delivery. 5. Are regularly reviewed (and revised, if necessary) by the
faculty and other stakeholders. 6. Are embedded in courses and experiences across the
curriculum so that students have sufficient opportunity to master them.
2. Assessment Measures Learning Outcome: Explain the central ideas and dominant cri:cisms of the key drama:c periods using illustra:ons from per:nent plays. Poten/al Measures: • Act as a spokesperson for one of the drama:c periods and par:cipate in a debate
about which one is superior, based on the key works wriXen in that period. • Write and illustrate a graphic novel that explores one of the drama:c periods. • Take on the role of literary cri:c and present the strengths and weaknesses of one
of the periods, based on its major works. • Write an original play that incorporates the key literary devices, topics, and style
of a selected drama:c period. • Complete a class evalua:on that asks them to rate how well they learned course
concepts.
Examples of Measures Direct Measures • Case studies • Clinical experiences • Discussion boards • Exams/Licensure Exams* • Exhibi:ons • Internships • Lab reports • Lesson plans • Performances • Porholios • Presenta:ons (solo, group) • Projects (solo, group) • Research papers
Indirect Measures • Alumni surveys • Exit interviews/surveys • Focus groups (students, alums,
employers) • Graduate school placement • Job placement rates • Papers accepted for publica:on • Research presented at
conferences
Direct or Indirect? A. Licensure examina:on B. Focus group C. Membership in a professional organiza:on D. Group presenta:on of culmina:ng project E. Lab report F. Student teaching experience G. Case study H. Senior recital I. Take-‐home exam J. Capstone course pass rate
Exemplary Assessment Measures . . . 1. Are (mostly) direct. 2. Provide a complete picture of student learning. 3. Are clear enough to show that they are appropriate measures of
the outcomes being assessed. They ALIGN with the measures. 4. Include high-‐impact prac:ces and ac:vi:es that mirror what
future employers will expect them to be able to do. 5. Allow performance to be gauged over :me, not just in a single
course. 6. Are assessed by rubrics, checklists, keys, etc. that provide specific
evidence of what students know and can do (and don’t know/can’t do).
7. Are accompanied by clear, specific, and appropriate expecta:ons for performance.
3. Performance Expecta/ons
• At least 70% of students will earn a score of 75% on the comprehensive exam (which is coded to outcomes).
• 100% of students will earn a score on the licensure exam that is equal to or beXer than the na:onal average (95%).
• 90% of respondents will rate their sa:sfac:on with the program at “4” (sa:sfied) or higher on a five-‐point scale. No respondents will rate their sa:sfac:on below a “3” (somewhat sa:sfied).
• The average GPA of majors will increase by 1% a semester for three years, beginning with spring 2017 (2.25).
• Students will aXend at least three career development workshops in their senior year.
Tying the Pieces Together, #1 • Outcome: Analyze, interpret, and cri:cally assess empirical
evidence in sociological research. • Measure: Oral presenta:on analyzing the misuses of
sta:s:cs/data in print media (SOC 3XXX). Presenta:ons are assessed through primary trait analysis.
• Performance Expecta/on: Students will earn a minimum ra:ng of 3 out of 5 (competent) in each of four categories on the rubric (content, organiza:on, presenta:on skills, technical skills). The content sec:on specifically looks at students’ ability to iden:fy and analyze misrepresenta:ons of sta:s:cs.
4. Results A. You’re assessing students’ knowledge of key concepts via a
comprehensive exam delivered in the senior capstone course, and the performance goal is that 90% of students will earn a score of at least 75%. You’ll collect:
– Each student’s overall score and his or her score on each of the outcome-‐specific exam ques:ons
B. You’re assessing students’ oral communica:on competency via a presenta:on assigned in mul:ple sec:ons and assessed by a common rubric, and your performance expecta:on is that 100% of students will earn a minimum ra:ng of 3 out of 5 (competent) in each of four categories on the rubric. You’ll collect:
– Copies of the completed rubrics, whose scores, sub-‐scores, and key comments you’ll enter in a spreadsheet
Using a Meta-‐Rubric to Track Results
Dimension Organiza/on (10 points)
Content & Development (50 points)
Grammar & Mechanics (30 points)
Audience Awareness (10 points)
Ave. Score on Assignment 10, Fall 2016 (N=48)
8 (80%)
38 (76%)
20 (67%)
5 (50%)
Ave. Score on Assignment 10, Spring 2017 (N=60)
9 (90%)
42 (84%)
23 (77%)
7 (70%)
Tying the Pieces Together #2 • Goal: Students will earn a minimum ra:ng of 3 out of 5
(competent) in each of four categories on the rubric . . . . – Results: 75% of students met or exceeded the benchmark (N=38), so the goal was not aXained. (Detailed results are available here.)
• Goal: 80% of students will pass the Indiana Principal Licensure
Exam, which is a requirement for Na:onal Recogni:on through our Accredita:on Agency. – Results: The goal was met: 48 of 54 (91%) candidates passed the exam in the period from June 1, 2015 to May 24, 2016. (Detailed results are available here.)
A Note About Results What the numbers mean is even more
important than the numbers themselves. Good results answer the ques:on, “What specifically did assessment help you discover about student learning?” (To be con:nued in Part Two.)
Ques/ons About Part 1 of the SLSR?
SLSR Part Two: Engagement & Improvement
1. What discoveries has assessment has enabled you to make about student learning, as well as about the curriculum, departmental processes, your assessment plan, etc.?
2. What changes and improvements will you make in response to your discoveries (or the assessment coordinator’s feedback on your previous SLSR)?
3. What will your assessment plan focus on in the coming year?
Ge_ng Down to the Details Organiza/on (10 points)
Content & Development (50 points)
Grammar & Mechanics (30 points)
Audience Awareness (10 points)
Average Score on Assignment 10, Fall 2016 (N=48)
8 (80%) 38 (76%) 20 (67%) 5 (50%)
Strengths Thesis and topic sentences generally clear Transi:ons typically effec:ve
Claims/ideas supported with evidence from experts
Fewer comma splices and agreement errors than at beginning of semester J
Recognizes that audience may have a different posi:on on the issue
Needed Improvements
Structure immature/predictable (5-‐paragraph essays)
Analysis lacks higher-‐order skills such as synthesis and reflec:on
Immature sentence structure Weak ability to paraphrase
Argument insufficient to change audience’s posi:on
Tying the Pieces Together #3 • Goal: Students will earn a minimum ra:ng of 3
(competent) . . . . • Results: . . . thus the goal was not aXained. . . . • Analysis and Plans for Improvement: While students did not
achieve the benchmark, results con:nue to show that they perform beXer in the spring than in the fall. The faculty believes spring students have an advantage because they already have completed SOC XXX, which requires them to make both individual and group presenta:ons, and that these students consequently are more confident speakers with beXer teamwork skills. Beginning in 2017, SOC 3XX will become a spring-‐only course. If student performance improves, we will adjust workloads permanently.
Tying the Pieces Together #4 • Goal: 70% of students will earn a score of 75% or beXer . . . . • Results: . . . Overall, the goal was met. • Analysis and Plans for Improvement: When all three
ques:ons are collec:vely considered, the 75% benchmark was not met. Only 20 students (64.52%) achieved the target score of 33.75. More problems were added to the curriculum this year, and during the next academic year, flipping the class will be aXempted to encourage students to take charge of their learning. Approximately 50% more :me will be spent working problems in class.
Tying the Pieces Together #5 • Goal: Seniors will achieve an average of 3.9/5.0 on the preceptor evalua:on. . . . • Results: Seniors achieved a mean score of 4.41/5.0 . . . All standards were met.
• Analysis and Plans for Improvement: Because results show that preceptors rate students’ knowledge and clinical skills higher than faculty do, :me will be given during preceptor development this summer to ensure that all preceptors are evalua:ng students in a similar manner. It is interes:ng that students overwhelmingly scored higher on prac:cal skills examina:ons than standardized pa:ent evalua:ons. Prac:cal skills examina:ons focus on isolated bits of knowledge and clinical skill. The standardized pa:ent evalua:ons are real-‐:me pa:ent encounters where students have to incorporate several components of knowledge and skill into pa:ent care (similar to what clinical prac:ce is really like). These findings highlight that more :me should be spent in the classroom and laboratory serngs to allow students to incorporate mul:ple components of pa:ent care in a manner more similar to clinical prac:ce (e.g., teaching standardized pa:ents, simula:ons with task trainers . . . .).
Exemplary Engagement & Improvement
1. Involves all program faculty in collec:ng and analyzing results. 2. Provides clear, specific, and complete details about data collec:on,
analysis, and interpreta:on to demonstrate the validity and usefulness of the assessment process.
3. Shows that students generally are achieving the performance standards expected of them and demonstrate con/nuous improvement on those they have yet to achieve/achieve less well.
4. Entails regular and specific reflec:on from stakeholders on students’ recent achievement of performance goals and the history and impact of previous plans, ac:ons, and results.
5. Results in the development and implementa:on of per:nent recommenda:ons for improvement in student learning (and also outcomes, assessment measures, performance goals, data collec:on processes, etc.).
6. Is visible.
Related Requirements of the Higher Learning Commission
• The ins:tu:on has clearly stated goals for student learning and effec:ve processes for assessment of student learning and achievement of learning goals.
• The ins:tu:on assesses achievement of the learning outcomes that it claims for its curricular and co-‐curricular programs.
• The ins:tu:on uses the informa:on gained from assessment to improve student learning.
• The ins:tu:on’s processes and methodologies to assess student learning reflect good prac:ce, including the substan:al par:cipa:on of and other instruc:onal staff members.
Ques/ons About Part 2?
Mary.Herrington-‐[email protected]
hXps://www.indstate.edu/assessment (forms, direc:ons, sample reports)