The “Quality Agenda:” Implications for Assessment, Policy, and Professional Accreditation

Peter T. Ewell

National Center for Higher Education Management Systems (NCHEMS)

AACSB International Conference

New Orleans, LA

March 18, 2014

Logic of this Presentation

The “Quality Agenda:” Where Did It Come From?

The Historical Evolution of Discussions About Quality in Higher Education

An Evolving “Standards” Movement in Higher Educationo National: Degree Qualifications Frameworks with

the U.S. DQP as an Example

o Professional: Assessment in Business Programs

Into the Future: A “Changing Ecology” for Accreditation and Its Implications for Assessing Quality

A Prominent “Completion Agenda” in Higher Education Policy

Stimulated by Recognition of Link Between Economic and Social Benefits and Postsecondary Attainment in Many Countries

Widespread National Attainment Goals (e.g. The Obama Goal in the U.S.: 60% of Young Adults with a Postsecondary Credential by 2020)

But Concern that “Completion” Could Come at the Expense of Academic Quality

Hence, a Growing “Quality Agenda”

New Interest in Aligned Student Learning Outcomes on a National and International Basis

Shift of Attention from “Doing Assessment” to “Standards for Learning”

Stakeholder Concern About Graduate Quality, Particularly from the Employment Community

So the Current Challenge is to Raise Completion Rates while Not Losing Academic Quality

=

Changing Notions of “Quality” in Policy Discourse About Higher Education

Largely taken for granted until the emergence of formal systems of “quality assurance” like accreditation and national Quality Assurance Agencies

New conceptions of “quality” emerged gradually as colleges and universities diversified and became more complicated

Today’s conception of “quality” is thus a sedimentary construction with new notions of what should count “layered in” on top of old ones

First Incarnation: Reputation

Colleges for “The Quality” (who were members of a privileged elite or entering a professional class like the clergy, law, or medicine)

“College” as a reserved term: distinct from postsecondary institutions: “institutes,” “normal schools,” etc.

Alive and well on bumper stickers and big time sports teams [not to mention media rankings]

Second Incarnation: Resources

U.S. Accreditation Criteria of the 1920s (North Central Association): “The college should:

• Enroll at least 200 students

• Should comprise at least eight departments with at least one person of professorial rank

• Should maintain a live, well-distributed library of at least 8000 volumes”

Quantitative resource-based criteria like these officially a thing of the past…but stuff still matters a lot in popular views of “quality”

Third Incarnation: Selectivity

The rise of admissions selectivity in the 1950s

Reputation and exclusivity in a new guise: in the U.S., the role of the SAT was said to be to “uncover the hidden aristocracy of talent”

An implied theory of education: smart begets smart by association and osmosis

Fourth Incarnation: “Fitness for Purpose”

“Mission-based” quality review emerges as most appropriate for diverse postsecondary systems

Peer review and institutional audit become the primary “assessment instruments” under this approach.

“Purposes appropriate to an institution of higher education…”

But what happened to standards?

Fifth Incarnation: Outcomes

An “exo-skeletal” approach: outcomes assessment largely added on to the regular processes of teaching and learning.

The resulting paradigm:

• Statements of intended learning outcomes

• Various ways to gather evidence of attainment

• Use of the resulting information to improve

Embedded in U.S. regional accreditation standards by the mid-1990s and national QAA reviews by 2000

A Sixth Incarnation: Exit Proficiencies?

A common set of graduation proficiencies adopted by all providers

Assessments embedded in the regular teaching and learning process:

• Signature assignments in key classes

• Developed in common by teaching staff

• Graded or rated using standard rubrics

Re-positioned proficiency-based transcripts that show “student learning as academic currency”

National Qualifications Frameworks

Matrix of Identified Proficiencies by Degree Levels

Purpose to Align and ‘“Moderate Academic Standards at Various Degree Levels

Some Examples:

Bologna Process Common Outcomes Benchmarks

QFs in UK, Australia, Ireland, Scotland, and Many Others

“Tuning” in Many Disciplines

Background to the DQP in the U.S.

Qualifications Frameworks in Many Other Countries (as Noted)

AAC&U LEAP Outcomes Statements and Rubrics

State-Level Outcomes Frameworks in U.S. (e.g. UT, WI, CSU, ND, VA)

Some Alignment of Cross-Cutting Abilities Statements Among Institutional Accreditors

What Does the DQP Look Like?

Three Degree Levels: Associate, Bachelor’s, and Master’s

Five Learning Areas: Specialized Knowledge, Broad/Integrative Knowledge, Intellectual Skills, Applied Learning, and Civic Learning

Framed as Successively Inclusive Hierarchies of “Action Verbs” to Describe Outcomes at Each Degree Level

Intended as a “Beta” Version, for Testing, Experimentation, and Further Development

An Example: Communication Skills

Associate Level: The student presents substantially error-free prose in both argumentative and narrative forms to general and specialized audiences

Bachelor’s Level: The student constructs sustained, coherent arguments and/or narratives and/or explications of technical issues and processes, in two media, to general and specialized audiences

Master’s Level: The student creates sustained, coherent arguments or explanations and reflections on his or her work or that of collaborators (if applicable) in two or more media or languages, to both general and specialized audiences

Some Implications of the DQP

The DQP Asserts that Every Student Should Graduate with the Designated Proficiencies. This Means that:

The Typical Approach of Setting Outcomes as “Aspirations” and Conducting Assessments of “Average” Student Performance is not Enough

Assessment as an “Add-On” to the Curriculum is Not Enough

Assessment Must Be Embedded in Regular Student Assignments and Examination Questions and Certified at Multiple Levels on the Way to Degree Completion

AACSB’s “Assurance of Learning” Similar in Form and Intent to Assessment Requirements of

Other Specialized Accreditors

Outcomes Set by Program Consistent with Mission

Both Generic and Field-Specific Outcomes

Methods Selected by Program and Not Standardized

Outside Consultation (e.g. with Employers) Encouraged

Use of Results Emphasized as Much as Assessment Process

Assessment in Business: NILOA Survey

Fourth of Twelve Schools or Departments with Respect to Overall Assessment Activity

Fifth of Twelve in Citing “Specialized Accreditation” as Reason for Doing Assessment

Second of Twelve in Using Capstone Courses

Second of Twelve in Reporting High Levels of Faculty Involvement in Assessment

Top School or Department Wanting Better Instruments

A Changing Ecology for Accreditation

New Patterns of Student Participation

New Kinds of Providers

A Transformed and Contingent Faculty

New Approaches to Instructional Provision

Constrained Resources

Most of These Forcing Learning Outcomes as the Only Practicable Way to Determine Quality

What are Accreditors Doing in Response?

Look More Explicitly at:

Coherence of Student Experience Across Settings

How Faculty Roles are Constructed and Discharged

“Outsourcing” of Courses and Teaching

Response to Stakeholder Needs (e.g. Employers)

Transparency and Public Reporting

Use of Electronic Media and “Virtual Review”

Putting It All Together

New Discourse About “Quality” Centered on Standards for Learning that All Students Will Achieve

As Documented in an Identified Array of Proficiencies that Schools Adopt in Common

Reported Publicly to a Wide Array of Stakeholders

Review Techniques Transformed to Respond to New Ecology for Higher Education and New Modes of Provision