Peter EwellTerry Grimes

Kathy JohnsonGeorge Kuh

Terrel Rhodes

Assessment Institute Indianapolis IN October 29, 2012

Degree Qualifications Profile: What Is the DQP and What Does It

Mean for Assessment

Overview Origins of the DQP – Peter Ewell CIC DQP project – Terry Grimes AAC&U DQP Quality

Collaboratives – Terry Rhodes Ivy Tech-IUPUI Collaborative –

Kathy Johnson NILOA and the DQP

NILOA NILOA’s mission is to discover and disseminate effective use of assessment data to strengthen undergraduate education and support institutions in their assessment efforts.● SURVEYS ● WEB SCANS ● CASE STUDIES ● FOCUS GROUPS ● OCCASIONAL PAPERS ● WEBSITE ● RESOURCES ● NEWSLETTER ● LISTSERV ● PRESENTATIONS ● TRANSPARENCY FRAMEWORK ● FEATURED WEBSITES ● ACCREDITATION RESOURCES ● ASSESSMENT EVENT CALENDAR ● ASSESSMENT NEWS ● MEASURING QUALITY INVENTORY ● POLICY ANALYSIS ● ENVIRONMENTAL SCAN

● DEGREE QUALIFICATIONS PROFILEwww.learningoutcomesassessment.org

To increase the proportion of Americans with high-quality

degrees and credentials to 60 percent by 2025.

Background

Qualifications Frameworks in Many Other Countries

Bologna Process Common Outcomes Benchmarks (e.g. “Dublin Descriptors”)

AAC&U LEAP Outcomes Statements and Rubrics

State-Level Outcomes Frameworks in U.S. (e.g. UT, WI, CSU, ND, VA)

Some Alignment of Cross-Cutting Abilities Statements Among Institutional Accreditors

Lumina Degree Qualifications Profile

• Three Degree Levels: Associate, Bachelor’s, and Master’s

• Five Learning Areas: Specialized Knowledge, Broad/Integrative Knowledge, Intellectual Skills, Applied Learning, and Civic Learning

Degree Qualifications Profile

Associate

Bachelor

Master’s

Civic Learning

IntellectualSkills

Applied Learning

Specialized Knowledge

Broad, Integrative Knowledge

Lumina Degree Qualifications Profile

• Three Degree Levels: Associate, Bachelor’s, and Master’s

• Five Learning Areas: Specialized Knowledge, Broad/Integrative Knowledge, Intellectual Skills, Applied Learning, and Civic Learning

• Framed as Successively Inclusive Hierarchies of “Action Verbs” to Describe Outcomes at Each Degree Level

• Intended as a “Beta” Version, for Testing, Experimentation, and Further Development Beginning this Year

How the Panel Approached Its Work

• Wide Literature Review (Other National QFs and Outcomes Adopted by U.S. Colleges and Universities)

• Emphasis on Application and Integration (as Distinctively “American” Undergraduate Attributes)

• But Confined to Things that Institutions Actively Teach (Therefore Few Values or Attitudes Included)

• Emphasized Civic Learning as an Area Particularly Important for a Functioning Democracy

An Example: Communication Skills

Associate Level: The student presents substantially error-free prose in both argumentative and narrative forms to general and specialized audiences

Bachelor’s Level: The student constructs sustained, coherent arguments and/or narratives and/or explications of technical issues and processes, in two media, to general and specialized audiences

Master’s Level: The student creates sustained, coherent arguments or explanations and reflections on his or her work or that of collaborators (if applicable) in two or more media or languages, to both general and specialized audiences

An Example: Engaging Diverse Perspectives

Associate Level: Describes how different cultural perspectives would affect his or her interpretations of prominent problems in politics, society, the arts, and/or global relations

Bachelor’s Level: Constructs a cultural, political, or technological alternative vision of either the natural or human world, embodied in a written project, laboratory report, exhibit, performance, or community service design; defines the distinct patterns in this alternative vision; and explains how they differ from current realities

Master’s Level: Addresses a core issue in his/her field of study from the perspective of either a different point in time, or a different culture, political order, or technological context, and explains how the alternative perspective contributes to results that depart from current norms, dominant cultural assumptions, or technologies—all demonstrated through a project, paper, or performance

What Happens Next?

Growing Number of Lumina-Funded Follow-On Projects Designed to “Test Drive” the DQP (HLC, WASC, SACS, CIC, AASCU, AAC&U, Community Colleges) Involving More than 120 Institutions

Integral to teaching and learning, not an add-on “exo-skeleton”

Lumina-Funded Projects

Regional Accreditors• ACCJC (15)• HLC (23)• SACS (22)• WASC (28)

Organizations• AASCU (6 in 3

state systems)• AAC&U (21)• CIC (25)

States• Oregon (24)

http://www.learningoutcomesassessment.org/DQPNew.html#Funded

What Happens Next?

Growing Number of Lumina-Funded Follow-On Projects Designed to “Test Drive” the DQP (HLC, WASC, SACS, CIC, AASCU, AAC&U, Community Colleges) Involving More than 120 Institutions

Integral to teaching and learning, not an add-on “exo-skeleton”

NILOA Role to “Harvest” the Lessons Learned in these Projects [as well as others using the DQP outside the Lumina project universe]

Results of Projects (and other efforts) Will be Used to Refine the DQP Further in 2014.

CIC DQP CONSORTIUM

25 institutions from CIC with teams of 3

Purpose: enhance student learning

Initiatives already in place or planned

Areas of learning challenging to assess

SOME DISCOVERIES:• STUDENT LEARNING OUTCOMES

Not clearNot assessedNo consensus on interpretation

• CURRICULUM MAPPING No systematic plan for ratcheting

• FEAR OF DATA INUNDATION• SCOPE OF DQP A CHALLENGE

SOME METHODS USED IN PROJECTS1.Crosswalking with NSSE, CLA, etc.

2.Mapping DQP to General Education, etc.

3.Home grown surveys on student perception

4.Student and faculty focus groups on learning outcomes

5.Rubric workshops to create and test rubrics based on DQP areas of learning.

6.Workshops on creating and assessing course embedded signature assignments

7.Extensive data audits to find what already existed.

8.Create opportunities for peer feedback on assessments.

Quality Collaboratives

Terrel L. RhodesAssociation of American Colleges and

UniversitiesAssessment Institute

October 2012

Why Did AAC&U Join the DQP Effort?

Focus on Student Performance

Research ProjectsPapers

PerformancesCreative Works

Quality Collaboratives National Advisory Panel Nine states

California State University System Oregon Utah North Dakota Wisconsin Indiana Kentucky Virginia Massachusetts – university systems and state higher

education commission offices Twenty campus QC’s – 2 and 4 year transfer partners Connections – OR – community college lead project with

OUS; ND – HLC and NDSU; AASCU – 3 states – NY, GA, TX – civic engagement

State QC Partners CSU-Northridge and Pierce CC: University of Utah and Salt Lake CC U of Wisconsin – Oshkosh and UW – Fox

Valley; UW – Parkside and UW – Waukesha IUPUI and Ivy Tech CC U of Louisville and Elizabethtown

Community and Technical College James Madison University and Blue Ridge

CC; Virginia Commonwealth University and J.

Sargent Reynolds CC U Mass – Lowell and Middlesex CC; Fitchburg State U and Mount Wachusett

CC

Faculty Collaboration

The DQP Asks Us to Shift...

from My Work to OUR Work

Assessment Practices That Verify

Achievement

Valid Assessment of Learning in Undergraduate Education (VALUE)

Rubrics Inquiry and analysis Critical thinking Creative thinking Written communication Oral communication Reading Quantitative literacy Information literacy Teamwork Problem solving Civic knowledge and engagement Intercultural knowledge and competence Ethical reasoning and action Foundations and skills for lifelong learning Integrative learning

Ivy Tech Central Indiana and IUPUI

Quality Collaboratives Partnership

Recent Legislation in Indiana

• Caps of 120 credits per baccalaureate degree; 60 credits per associate degree

• Creation of a 30-hour transferable general education core based on 6 competency domains– Written communication– Scientific thinking– Speaking and listening– Quantitative reasoning– ‘Ways of Knowing’

• Arts and Humanities• Social and Behavioral Sciences

Indiana QC Focus

• How can we ensure that students transferring from Ivy Tech Central Indiana to IUPUI continue to succeed?

• Can we use the DQP to guide discussions of curricular alignment connected to the Statewide Transferable Core?

• Can we use DQP associate-level competencies to develop shared assessment rubrics in Writing and in Engineering?

Our Approach

• Gather instructors from both institutions for professional development workshops – either in-person or through videoconferencing

• Limiting focus to Writing (including Composition and Technical Communication) and Pre-Engineering/Engineering makes project more manageable

• Project also serves as vehicle for keeping faculty connected to state-wide work on general education

Dynamic Criteria Mapping (DCM)• Process used to foster ‘organic assessment’ among

faculty• Creates normative criteria by building them inductively

and collaboratively from review of student artifacts– Following review of artifacts, faculty nominate characteristics

that they value in student work• Values can then be mapped back to DQP to create

common assessment rubrics• Writing Faculty – day-long workshop in July ’12• Engineering Faculty – will repeat process (using pre-

engineering project artifacts) this spring

Challenges

• Faculty teaching a single course/courses have difficulty shifting from ‘what I do’ to ‘what we do’ – particularly part-time faculty

• Different levels of support for professional development and different teaching loads at 2-year and 4-year institutions

• While some ‘faculty champions’ have emerged, some are deeply concerned about resources that transfer portfolios will demand

Next Steps• Develop a transfer student portfolio that can

store direct evidence of student learning/mastery of associate-level competencies

• Continue faculty development efforts, particularly in support of creation of General Education Core at IUPUI

NILOA’s role with the DQP

NILOA is “harvesting” (collecting, analyzing, summarizing, synthesizing) what can be learned from all of the funded and unfunded work currently going on with the DQP

NILOA and the DQPDocument what is being done, by whom, and

distill lessons learned in current DQP projects at the campus/system level—tracking current and relevant future work;

Tell us via the DQP Institutional Activity Report: https://illinois.edu/fb/sec/704337

Share your DQP story? What are you doing? Who is involved? What are you learning? What assistance do you need?

What We’re Learning Across Projects DQP is a conversation starter: offers a common

vocabulary for talking about outcomes Curricular mapping: Where are students mastering

these competencies? Where are the gaps?Certify transfers, align and “streamline” systemsFaculty engagement and ownership are essential,

which take timeDoing assessment right is a continuing, perennial

challenge.

Faculty must drive assessment Identify which competencies described in the

DQP they address in their courses/ labs/studios Identify which competencies are major

objectives (probably only a few) Map those competencies to the existing

relevant assignmentsTweak existing or create new assignments/

student work to elicit the appropriate student behavior

Faculty must drive assessmentValidate competence through assignments:

– lab specifications– test questions– performance protocols– exhibit instructions– field work questions– paper topics with rubrics

www.learningoutcomeassessment.org

NILOA Website Includes:New to the DQP?• Orientation or starting point to learn more

DQP In Practice• Institutions working with the DQP are featured, outlining successes and

challenges

DQP Community Forum• Open to the public, the Community Forum is a message board to facilitate

discussion of the DQP, to share ideas & troubleshoot challenges

DQP Calendar• Calendar outlines conferences and other venues where DQP-related sessions

will occur

DQP Resource Kit• Information and tools - literature and responses to frequently asked questions

regarding the DQP

• DQP Resource Kit: – Applied and Integrative Learning– Assessment– Change Management– Competency-based Education– Course-embedded assignments– Curriculum Mapping– Data Audit– Faculty Engagement– Prior Learning– Rubrics– Signature Assignments– Student Affairs & Co-Curriculum– Transfer/Articulation– Tuning

Questions &

Discussion

www.learningoutcomeassessment.org