contemplating the end: how assessment can help with retention
DESCRIPTION
Contemplating the End: How Assessment Can Help with Retention. Robert Mundhenk AAACL Annual Conference 8 April 2009. The Age of Accountability. Once upon a time, colleges and universities were like doctors. . . . Then tuition and greater consumer awareness rose - PowerPoint PPT PresentationTRANSCRIPT
Contemplating the End: Contemplating the End: How Assessment Can How Assessment Can Help with RetentionHelp with Retention
Robert MundhenkAAACL Annual Conference8 April 2009
The Age of AccountabilityThe Age of Accountability
Once upon a time, colleges and universities were like doctors. . . .
Then tuition and greater consumer awareness rose
Then higher education become a political topic
The Age of AccountabilityThe Age of AccountabilityThe Spellings Commission
◦Early stirrings: Threatened regional accreditation Proposed standardized testing Argued that colleges were failing to serve
their public responsibility—and that they cost too much
◦Results: None of the above, but---
The Age of AccountablityThe Age of AccountablityThe charges remainedHigher Education Act required
increased reporting of outcomes, especially in teacher training
State legislatures and boards become increasingly interested in outcomes
“Voluntary” System of Accountability: Enrollment and graduation data Institutional climate data “Engagement” information Standardized test data on critical thinking
The Age of AccountabilityThe Age of Accountability
….but exactly what these “outcomes” are is still very murky
Thus “success” differs state to state and institution to institution
Context: AccreditationContext: Accreditation
Original purpose was to ensure quality
Peer Review Extreme confidentialityFocus traditionally on quantity
and processes rather than quality and results
Rarely concerned with learning
Context: Accreditation and Context: Accreditation and AssessmentAssessmentMid-80’s: Accreditors began to
require some attention to assessment of learning
Mid-90’s: Focus on structures and data collection
Mid-00’s: Focus of use of information to improve outcomes
Assessment and Assessment and Accountability: Serving Two Accountability: Serving Two MastersMastersNot always aligned values
institutionally; thus not always aligned activities
Assessment has moved toward improvement of learning outcomes, which tends to appeal to faculty while nonplussing external stakeholders
Assessment and Assessment and Accountability: Serving Two Accountability: Serving Two Masters?Masters?Accountability efforts seem
aimed at external stakeholders seeking “transparency” through broader, statistically based institutional outcomes—but these efforts seem pointless and bureaucratic to faculty
Why the Shift toward Why the Shift toward Accountability?Accountability?A need to determine degrees of
successCostAccountability data are relatively
easy to collect—and have been collected and reported for years—while learning outcome data are new and not particularly comparable
Can Assessment of Can Assessment of Learning and Learning and Accountability Efforts Be Accountability Efforts Be Compatible?Compatible?They ought to beServing two audiences but
essentially the same purpose, the success of students
Aligning Assessment and Aligning Assessment and Accountability: Data Set 1Accountability: Data Set 1Public Four-Year Totals Re-en AnnRet CumGr Success 1992 8,929 64.5 72.2 0.1 72.31993 10,296 64.5 72.4 0 72.41994 9,804 63.4 72.4 0 72.41995 9,655 64.7 73.8 0 73.81996 9,624 65.7 74.7 0.1 74.81997 9,928 65.2 74.7 0.1 74.71998 10,406 66.6 75.5 0 75.61999 10,684 66.5 75.7 0 75.82000 10,480 68.6 77.3 0 77.32001 10,643 68.6 77.2 0.1 77.32002 10,861 68.4 76.7 0.1 76.82003 11,527 67.7 75.9 0.3 76.22004 11,497 67.5 75.5 0.1 75.62005 12,042 68.5 76.6 0.2 76.82006 12,083 68.3 77.0 0.2 77.12007 12,248 67.0 75.4 0.5 75.82008 12,781
Aligning Assessment and Aligning Assessment and Accountability: Data Set 2Accountability: Data Set 2Public Two-Year Totals Re-en AnnRet CumGr Success1992 3,885 35.9 41.4 16.0 57.41993 3,785 38.8 44.7 12.3 57.01994 3,018 41.2 46.1 10.5 56.61995 3,498 44.3 49.2 6.5 55.71996 3,718 44.5 50.3 5.6 55.91997 4,133 45.5 51.4 4.0 55.41998 4,639 46.7 53.0 4.2 57.11999 5,041 48.2 53.9 3.5 57.42000 4,688 49.2 55.4 2.9 58.32001 5,336 51.4 56.7 3.2 59.82002 5,726 49.6 55.4 2.9 58.32003 6,107 48.0 53.2 5.3 58.42004 6,059 48.8 54.0 4.9 58.92005 6,179 46.9 52.3 4.8 57.12006 5,828 48.0 53.2 4.9 58.12007 6,678 49.4 54.0 5.7 59.72008 6,594
Aligning Assessment and Aligning Assessment and Accountability: Data Set 3Accountability: Data Set 3Total Public Re-en AnnRet CumGr Success1992 12,814 55.8 62.9 4.9 67.81993 14,081 57.6 65.0 3.3 68.31994 12,822 58.2 66.2 2.5 68.71995 13,153 59.3 67.2 1.8 69.01996 13,342 59.8 67.9 1.6 69.51997 14,061 59.4 67.8 1.2 69.11998 15,045 60.4 68.6 1.3 69.91999 15,725 60.6 68.8 1.1 69.92000 15,168 62.6 70.5 0.9 71.52001 15,979 62.9 70.3 1.1 71.52002 16,587 61.9 69.3 1.0 70.42003 17,634 60.9 68.0 2.0 70.02004 17,556 61.0 68.1 1.8 69.8
Using Retention DataUsing Retention Data
What conclusions can we draw from these data?
What kinds of decisions can we make on the basis of these data?
How can these data help us improve learning?
Aligning Assessment and Aligning Assessment and Accountability: Data Set 4Accountability: Data Set 4 Graduation Rates at Arkansas Public Universities (Source:
U.S. Department of Education College Navigator)A 38%B 39%C 33%D 34%E 20%F 33%G 33%H 58%I 18%J 44%
(Data do not include transfer-out students)
Drawing InferencesDrawing Inferences
In what ways do the graduation data qualify the retention data?
What conclusions can be drawn from these data?
A Qualification:A Qualification:Alexander Astin (“How good is your
institution’s retention rate?,” 1993): Graduation rate depends on the characteristics of the students who initially enroll
“…some institutions with ‘high’ graduation rates should really have rates that are even higher….other institutions with graduation rates that appear to be very modest are actually retaining their students at a significantly higher rate…”
Finding AnswersFinding Answers
Retention is a worthy end in itselfRaw data on retention or
graduation rates give no clues about ways to improve or increase retention and graduation rates
Assessment data and strategies may be a way to process raw data
Using Assessment to Using Assessment to Improve RetentionImprove RetentionWhat data sets or assessment
assumptions currently in place might be used to help in understanding retention information?
What data sets or assessment assumptions might be employed to improve retention?
Thinking about the End: Why Thinking about the End: Why We AssessWe AssessSelf-justification (focus on data)Improvement of instruction
(focus on teaching)Improvement of learning (focus
on outcomes)
An AssertionAn AssertionWe assess the achievement of
student learning outcomes because we want to know the degree to which we have succeeded in helping students achieve the learning goals we have set for them—and caring about learning and its improvement may actually be a way to improve retention
A Validation of the A Validation of the AssertionAssertionFrank Heppner, “On the bottom
line, good teaching tops good research,” Chronicle of Higher Education, 13 March 2009:◦“Can faculty members be trained to
be more effective teachers and so have an impact of retention? Absolutely.”
A Validation of the A Validation of the AssertionAssertionIf better teaching produces better
retention, one way to improve teaching is to provide faculty with meaningful information that tells them the degree to which students are achieving student learning outcomes
Thus better information about learning can produce better teaching, which may improve retention
What about Other Ways in What about Other Ways in Which Assessment Which Assessment Information May Have a Information May Have a Positive Effect on Positive Effect on Retention?Retention?
How Assessment Work Can How Assessment Work Can Help RetentionHelp RetentionFocus on outcomes can improve:
◦Student understanding of expectations
◦Institutional articulation of expectations
◦Syllabi and planning of experiences so that they emphasize outcomes
◦Grading and/or prioritization of grades
How Assessment Work Can How Assessment Work Can Help RetentionHelp RetentionFormative assessment can help
students understand expectations and grading strategies—and to make progress toward important outcomes over time(Mantz Yorke, “Formative Assessment and Its Relevance to Retention,” Higher Education Research & Development, 2 (2001), 115-126.
How Assessment Work Can How Assessment Work Can Help RetentionHelp RetentionConcentrating on outcomes
focuses the conversation internally and externally:
Learning becomes center of improvement efforts
Learning (not numbers) becomes center of conversations about effectiveness
How Assessment Work Can How Assessment Work Can Help RetentionHelp Retention“Drilling Down”:
Assessment data can provide detailed information about the raw retention rates:--breakdown of cohorts by various characteristics--tracking of student experience--correlating predicted retention with actual retention
How Assessment Work Can How Assessment Work Can Help RetentionHelp RetentionCorrelating institutional mission,
outcomes, and resources with graduation and retention data
--to what degree are outcomes achieved?
--have students achieved their desired objectives?
Assessment and RetentionAssessment and Retention
Assessment data cannot directly affect retention rates, but they can clarify their meaning
Assessment data and strategies can affect retention indirectly by improving student and staff understanding of their common learning goals
Assessment and RetentionAssessment and Retention
Assessment data and strategies can help institutions understand why their retention rates are where they are—and to determine how best to deal with them
For Further Information:For Further Information:
Robert T. MundhenkConsultant, Higher Education