The Landscape Changes: Assessment, Accreditation,

and AccountabilityRobert Mundhenk and Jennifer Fager

Assessment of Academic Achievement ConferenceUniversity of Wisconsin-Superior

18 January 2011

Assessment is a process in which those responsible for student learning on campus collect, analyze, and use information about student learning to improve it.

Three Views of AssessmentView 1:

“Outcomes-assessment practices in higher education are grotesque, unintentional parodies of both social science and ‘accountability.’ No matter how much they purport to be about ‘standards’ or student ‘needs,’ they are in fact scams run by bloodless bureaucrats who . . . do not understand the holistic nature of a good college education.” --Laurie Fendrich, “A Pedagogical

Straitjacket,” Chronicle of Higher Education, June 8, 2007, B6.

Three Views of Assessment:View 2

“The real culprit is the education establishment’s continuing reluctance to give the public a clear and accurate glimpse at how colleges spend their money. What’s missing, by and large, is ‘sunshine.’”--John A. Boehner, Speaker of the House of Representatives, though said as Chair, House Committee on Education and the Workforce

Three Views of Assessment:View 3:

Assessment to Improve Learning

Assessment as Meaningless Bureaucratic Imposition that Violates Academic Freedom

Assessment as an Element of Institutional Accountablity

Three Conflicting Forces:

[Colleges constantly tell us] “Everything is fine, we are doing a great job, send us more money, and leave us alone.”

--Rep. Buck McKeon, House Committee on Education

and the Workforce

The National Scene:

SPELLINGS COMMISSION (8/3/06):

◦ “ . . . The quality of student learning at U.S. colleges and universities is inadequate and, in some cases, declining.” (7)

◦ “Accreditation reviews are typically kept private, and those that are made public still focus more on process reviews than bottom-line results for learning or costs.” (13)

The New Accountability

Duncan Department of Education attacks the laxity of accreditors, including HLC, in assuring institutional and program quality

“Gainful employment” and “program integrity” assurance pushed by Duncan Department of Education

Student debt levels and employment rates

The New Accountability

“But can I assume that my graduate is going to be literate, know how to write, be exposed to math, be taught by senior faculty?”

--Anne D. Neal, National Advisory Council on Institutional Quality and Integrity, 12/20/07

The New Accountability

“Institutions are asked, ‘Are you measuring student learning?’ And they check yes or no. That must change. Whether students are learning is not a yes-or-no question. It’s How? How much? And to what effect?”

--Margaret Spellings Sept 06

The New Accountability

"We think -- I certainly think -- that it would be helpful when students head off to college that they have some basic information that they often don't have now: what's this going to cost, what's the [loan] repayment at that school, the expected graduation rate, the placement rate if I go into certain fields. . . .

The Newest Accountability

“One fallout from the gainful employment discussion is that a lot of the things people are pushing the for-profits for would be useful for all students to have, whether they're going to a land-grant or a private college, to a for-profit or a nonprofit. I think we'll be likely to look at getting students that kind of information.

John Kline (R-MN), Chair, House Committee on Education and Labor (IHE 12/23/10)

The Newest Accountability

“Robust culture of accountability” with a “consumer-friendly database that provides access to institutional performance and aggregate student outcomes” (21)

“Accreditation agencies should make performance outcomes, including completion rates and student learning, the core of their assessment . . . to . . . allow comparisons among institutions regarding learning outcomes and other performance measures” (23)

  “Develop interoperable outcomes-focused accountability

systems designed to be useful to students, policymakers, and the public. . . .”(22)

 

Spellings Commission Recommendations (8/9/06)

Educational Testing Service: “What is needed is a systematic, data-driven,

comprehensive approach to understanding the quality of two-year and four-year postsecondary education, with direct, valid, and reliable measures of student learning.”

Workplace readiness and general skills Domain-specific knowledge and skills Soft skills . . . . Student engagement with learning

A Testing Service Response

Standard IPEDS data◦ Enrollment, persistence, graduation, etc.

Student “experiences and perceptions”◦ satisfaction and engagement data

Student learning outcomes:◦ Critical thinking◦ Analytic reasoning◦ Written communication◦ “. . . Using one of three tests”

Higher Education’s Response:APLU: “Voluntary System of Accountability”

Ranking colleges based on graduates’ performance evaluations

“It’s really about improving the dialogue on curriculum, performance, and how we can build a stronger relationship between the colleges, universities, and us, because, ultimately, their students become our employees.” --Richard D. Stephens, Senior Vice President for Human Resources (Chronicle of Higher Education, 9/19/08)

The Boeing Approach

“Accountability in higher education begins with the establishment of public definitions of degrees and criterion-referenced statements of academic performance so that, when an institution awards a credential it can assert, with confidence, ‘This is what a degree represents, this is what the student did to earn the degree, and a warrantee has been issued on behalf of both institution and student.’

--Clifford Adelman, Learning Accountability from Bologna (IHEP, July 2008)

The Bologna Approach

Government and employers are seeking greater accountability from higher education

Government is demanding greater accountability and rigor of accrediting commissions

Higher education associations are responding with outcome-based models

Some approaches are clearly inadequate Institutions must deal with internal

blowback while dealing with external pressures

In Short:

How do we avoid being reductive (or worse) while accommodating the great triad of accountability, accreditation, and assessment?

The Real Question:

Refocus on student learning within Five Criteria: Mission and Integrity Engagement and Service Acquisition, Discovery, Application of Knowledge Student Learning and Effective Teaching Preparation for Future

The new Criteria see learning as infused throughout all Criteria, so responsibility for supporting, enhancing, and improving learning becomes an institutional responsibility

The New Accreditation: Higher Learning Commission

Shift from inputs to outcomes Shift from capacity to student learning Shift from processes to results Concern with learning and its improvement

infuses all criteria Focus on shared responsibility Focus on accountability, but not on reporting for its

own sake Concentration on continuous improvement

THE NEW PARADIGM

Assessment “should be informed by the institution’s mission and include explicit public statements regarding the knowledge, skills, and competencies students should possess as a result of completing course and program requirements. . . . ”

Higher Learning Commission Statement on Assessment of Student Learning (2003)

“The institution defines quality by how well it fulfills its declared learning mission.”

The institution documents student learning by: Setting clear learning goals Collecting evidence of attainment Applying collective judgment as to the meaning and utility of evidence Using evidence to effect improvement

It uses multiple sources and kinds of evidence It shares results with stakeholders

What Regional Accrediting Agencies Expect:The Consortium of Regional Accrediting Commissions Report (2003)

Assessment’s “effectiveness in improving student learning relies on its integration into the organization’s processes for program review, departmental and organizational planning, and unit and organizational budgeting.”

Higher Learning Commission Statement on Assessment of Student Learning (2003)

Emphasis on student learning Focus shifts from content and inputs to

outcomes: what students know and value—and can do with what they know and value

General emphasis on improvement over reporting

Consideration of stakeholders Focus on use of data and integration into

institutional processes

Some Broad Themes:

How are your stated student learning outcomes appropriate to your mission, programs, and degrees?

What evidence do you have that students achieve your stated learning outcomes?

In what ways do you analyze and use evidence of student learning?

Six Fundamental Questions

How do you ensure shared responsibility for student learning?

How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?

In what ways do you inform the public about what students learn—and how they learn it?

Six Fundamental Questions

Effective assessment becomes a matter of commitment to the improvement of learning by improving the processes that produce it

Effective assessment focuses on how data can be used to improve, not on how it can be reported

Effective assessment focuses on inquiry rather than evidence

In that sense, effective assessment is at the heart of what we do as academic institutions

Commitment, Not Compliance

Meaningful

Useful

Workable

Reasonable

Assessment Practice Should Be:

Outcomes, plans, strategies, and approaches need to be focused on student learning in the context of institutional mission and values

A wide range of data-collecting strategies are possible, so long as they serve institutional needs in improving learning

Attention must be given to assessing learning in non-classroom settings—and to shared responsibility for that learning

Implications

Assessment strategies must be appropriate and credible, not uniform across the institution—or the state

Assessment strategies must produce information that can:

--credibly be used to improve learning

--adequately inform stakeholders of the degree to which outcomes have been achieved

Implications

Institutions must become comfortable with risk—and with the probability of failure—in the service of improved quality of learning

Assessment data must be seen as the basis for institutional conversations about learning, not as a process of self-justification

The quality, credibility, and integrity of an institution will be assumed to be reflected in its effectiveness in producing student learning

Implications

Assessment is a transparent process in which those responsible for student learning on campus collect, analyze, and use information about student learning to improve it, while sharing information about the achievement and improvement of learning with interested stakeholders.

A Reconciling Definition:

A process based in inquiry: What do our students learn? Is what they learn what we say they will learn? How do we know they have learned? What do we do with our knowledge? How can we improve student learning?

The New Assessment Is :

Predicated on clear delineation of student learning outcomes:

What students can do with what they know Aligned at several levels:

Institutional mission Program/major purpose Course’s contribution to both

The New Assessment Is:

A form of action research that can include: Observations Judgments Validated data Grades that correlate to outcomes

The New Assessment Is:

A way to improve learning A way to improve the systems that produce

learning A way to demonstrate effectiveness to various

stakeholders A basis for conversations with stakeholders

The New Assessment Is:

A formalization of what many of us already do intuitively and individually

An attempt to judge achievement of essential outcomes rather than a compilation of grades or test scores

A process focused on change for improvement, not on institutional self-justification

The New Assessment Is: