introduction to degree qualifications profile and learning ... and learn… · introduction to...

IntroductiontoDegreeQualificationsProfileandLearningFrameworks

NATASHAJANKOWSKI , D IRECTOR

NATIONAL INSTITUTEFORLEARNINGOUTCOMESASSESSMENT (NILOA)

RESEARCHASS ISTANTPROFESSOR, UIUC

NILOANILOA’smissionistodiscoveranddisseminateeffectiveuseofassessmentdatatostrengthenundergraduateeducation

andsupportinstitutionsintheirassessmentefforts.

● SURVEYS ● WEB SCANS ● CASE STUDIES ● FOCUS GROUPS● OCCASIONAL PAPERS ● WEBSITE ● RESOURCES ● NEWSLETTER ●

PRESENTATIONS ● TRANSPARENCY FRAMEWORK ● FEATURED WEBSITES ●ACCREDITATION RESOURCES ● ASSESSMENT EVENT CALENDAR ● ASSESSMENT

NEWS ● MEASURING QUALITY INVENTORY ● POLICY ANALYSIS ●ENVIRONMENTAL SCAN ● DEGREE QUALIFICATIONS PROFILE ● TUNING

www.learningoutcomesassessment.org

www.learningoutcomesassessment.org

www.degreeprofile.org

OurtimetodayWearegoingtotalkaboutLearningFrameworks!

WhatistheDQP?

Howwasitdeveloped?

Whatcanyoudowithit?

Whathastheimpactofitbeen?

Whatotherlearningframeworksareavailable?

Whatresourcesarethere?

Andhowdowegetstarted?

What is the Degree Qualifications Profile?

WHAT DOES ADEGREE

REPRESENT?seat time?

Carnegie credit hours?grade point averages?

required courses?

All of these suggest what degrees represent in terms of numbers.

What do degrees represent in terms of learning?

TheDegreeQualificationsProfile

Whatisit?

Anefforttodefineinexplicittermswhatdegreesshouldmean,irrespectiveofdiscipline

Who’sresponsible?

(1)Highereducation

(2)Fourdrafters

(3)LuminaFoundationforEducation

What is the Degree Qualifications Profile?

THEDQPIS

aframeworkclarifyingwhatdegreesshouldsignifyintermsofknowledgeandability.

amodelforexplicitstatementsoflearningoutcomes.

aplatformforeffectiveassessment.

THEDQPISN’T

Anattempttodictatestandardsorpromotestandardization.

Acomprehensivecollectionoflearningoutcomes.

Adeviceforevaluatingfacultyperformance.

10

OrganizationoftheProfile

Fiveareasoflearning• BroadandIntegrativeKnowledge• SpecializedKnowledge• IntellectualSkills• AppliedandCollaborativeLearning• CivicandGlobalLearning

Shownasinterrelated,notdiscrete

5 AREAS OF PROFICIENCY:

3 DEGREE LEVELS:

associate’s

bachelor’s

master’s

TrackingandMappingOver800institutionsusedtheDegreeQualificationsProfile– ourtaskwastoharvestthelearningandinformtherevision…

DocumentedUsesofDQP/Tuning

ReviewandreviselearningoutcomesstatementsMappingthecurriculumRevisingGeneraleducationAligningtransferbetween2and4yearDevelopingprogramsAccreditationStrategicplanningAssessingstudentlearningParticipatinginastatewideofnationalTuningprocess

6FoundationalPrinciples

Coherent,intentionalpathwaysSequenced,integratedlearningTransparencyandportabilityQualityassuranceInclusionandequityCollaboration

It’sallaboutstudents!

Whatarewelearning?

ImpactStudyofInstitutionsthatusedtheDQPfrom2011torevisionin2014:

• InstitutionalActivityReports• Usersurvey• Projectreports• DQPcasestudies• Institution-authoredexamplesofpractice• Reviewofinstitutionalwebsites

ImpactReports

18

MoreSuccessfulWhenConnected&Integrated

Percentageofreportedinvolvementwithadditionalinitiatives.

0%

10%

20%

30%

40%

50%

60%

RevisingGeneralEducation

RevisingProgramReview

UsingVALUERubrics

UsingLEAP DevelopingCBEPrograms

RevisingInstitutionalLearningOutcomes

19

DQPServingMultipleEnds

PercentageofreportedusesoftheDQP

0%

10%

20%

30%

40%

50%

60%

70%

80%

RevisionandAlignmentofLearningOutcomes

RevisionofGeneralEducation

AlignmentofLearningOutcomeswithExternal

Expectations

Transfer

20

DQPServingMultipleLevels

PercentageofreportedlevelsofuseofDQP

0%

10%

20%

30%

40%

50%

60%

70%

80%

Department/Program GeneralEducation Institution Co-curriculum

21

MultiplePositiveImplications

PercentageofreportedimplicationsfromDQPengagementbychangetype

0%

10%

20%

30%

40%

50%

60%

70%

80%

ReviseLearningOutcomes

DevelopCoherentandIntegrated

Program/Curricula

CreationofCurricularPathways

IncreasedFacultyEngagement

AlignmentofLearningOutcomes

ChangedProgramReviewProcesses

ChangedCourseReviewand

ApprovalProcesses

22

AreasofImpact

1. Quality-focusedconversations2. Clarifiedlearningandpersonal

developmentoutcomes3. Curriculumrevision4. AssessmentofStudentLearning

23

Conclusions

TheimpactofDQPisbroadenedanddeepenedwhenimplementedasalarger-scalechangeeffort.

CollaborationiskeytoeffectiveDQPimplementationefforts.

ThepositiveeffectsofusingtheDQPincreasewhentheworkisconnectedtoandbuildsonexistinginitiatives,structures,andprocesses.

SalutaryeffectsoftheDQPaccruewhenstudentsandtheirlearningarethemajorfocusofthework.

BiggestImpactAshiftingofperspective…

StartingInstitution

TransferInstitution

Courses

Employment

Co-Curriculum

CURRENTSTUDENTEXPERIENCEOFHIGHEREDUCATION

Assignments

Programs

GeneralEd

StartingInstitution

TransferInstitution

Courses

Employment

Co-Curriculum

Assignments

Programs

GeneralEd

HOWWEWORK

StartingInstitution

TransferInstitution

Courses

Employment

Co-Curriculum

INITIATIVESINHIGHEREDUCATION

Assignments

Programs

GeneralEd

StartingInstitution

TransferInstitution

Courses

Employment

Co-Curriculum

Assignments

Programs

GeneralEd

StartingInstitution

TransferInstitution

Courses

Employment

Co-Curriculum

Assignments

Programs

GeneralEd

StartingInstitution

TransferInstitution

Courses

Employment

Co-Curriculum

Assignments

Programs

GeneralEd

StartingInstitution

TransferInstitution

Courses

Employment

Co-Curriculum

Assignments

Programs

GeneralEd

HOWWENEEDTOWORK

Conversations“Anythingthatencouragesandfacilitatesdiscussionsregardingtheessentialquestionsofwhoourstudentsare,whoweare,andwhatourlearningoutcomesareforstudentsearningdegreesatdifferentlevelscanonlybeseenasadesirableoutcome!”

~Administrator,public,four-yearuniversity

Faculty“Ididn’trealizewhenwestartedthisprojectthatIwouldbesayingnow– thisworkhasshiftedourfacultyculture!Allfromconversationsaroundthemeaningofthedegree.Thoseconversationsenricheddialogwithfacultyacrossdisciplines,encouragedmoreattentiontoappliedlearning,andmadeusenthusiasticaboutdiscussingpedagogyandassignmentdesign.”

~Facultymember,four-yearpublicuniversity

FacultyandAssessment“UseoftheDQPhashelpedourcommunitygraspthebenefitsofassessment:helpingstudentsachievethecompetenciesatalevelappropriatetotheirdegree.Specifically,themainbenefithasbeentofocusandstrengthenfacultyparticipationinassessment.Weusedtotrytocoercethemintodoingit,nowtheyaretellinguswhyitmatters.”

~Administrator,four-yearpublicuniversity

StudentLearningOutcomes“ThefacultyfindtheDQPoutcomesingeneraltobebetterattunedthanmosttotheactualworkthattheirstudentsdo.Theylikethefactthattheyareexplicitandassignment-focusedratherthanvagueandgeneral.Thefivecategoriesmakealotofsensetousinstitutionallyandallowustoadapttheoutcomestoourmission.Thus,learningoutcomesassessmenthasreceivedagreatboostfromourparticipationintheproject.Also,departmentscanseethevaluethatassessmentbringstotheirprogramandtheirstudentsinaveryconcreteway.”

~Administrator,four-yearprivateinstitution

Curriculum“WethoughtwewereadoptingtheDQPtohelpwithassessment,butinsteadittransformedourthinkingaboutourcurriculumandhowwedesignit.TheDQPbecameaplatformfromwhichtolaunchaninstitution-widediscussionregardingcurriculumdesign.”

~Administrator,two-yearpublicuniversity

Whatcanyoudowithit?• MappingtheCurriculum• ArticulatingLearningOutcomes• AdvancingTransferStudentSuccess• RepresentingCo-curricularLearning

DQPProcess

RevisionofLearningOutcomes

AlignmentofLearningandCurriculumMapping

Revisionandredesignofcurriculum&learningexperiences

Revisionandalignmentofassessmentactivities

Revisionofpolicies

TheLearningSystemsParadigm

Consensus-based

Throughfaculty-ledconversations,reflections,andexplorationswithemployers,alumni,students,andothers,asharedunderstandingandconsensusisreachedonlearningoutcomes.Thissharedunderstandingservesasthefoundationforrevisingoutcomesforenhancedclarityanddesigningeducationalexperiences.

Alignment

Usingtheagreeduponlearningoutcomes,facultyandstaffaligneducationalexperiencesthroughouttheinstitutionforintentionalintegration,coherence,andfosteringofmultiplepathways.Alignmentinvolvescurriculummapping,scaffolding,assignmentdesign,mappingofcareerpathways,andco-curricularengagement.

Learner-Centered

Theeducationalsystemreorganizeseducationalexperiencesaround allstudentsandtheirlearning.Takingastudentviewincludesconsiderationofissuesofequity,learning-focusedtransfer,alternativedeliverymodels,flexibilityinofferings,integrationofprior-learningassessment,ensuringstackablecredentials,andbuildingmultiplepathways.

Communication

Communicationandcollaborationwithstudentsandotheraudiencesthroughtransparentdiscussionsaroundtheoutcomesandeducationalsystemworkstomaketheimplicitexplicit.Communicationinvolvesexplorationandintegrationwithadvising,alternativetranscripts,admissions,andemployers.

OtherframeworksLEAPEssentialLearningOutcomes:LEAPELOs

DegreeQualificationsProfile:DQP

Tuning(History,Communications,etc.)

EmployabilitySkillsFramework

CommonEmployabilitySkills(NACE)

CommonCoreStateStandards

Betacredentialsframework

Connectingcredentials

Internationalframeworks

Askyourself…Whatdoyouwanttodoandtowhatdoyou

wanttoconnect?Eachlearningframeworkaddressesdifferent

needsandends

Whatdoesaframeworkallowyoutodo?SetyouupforengagementwithVALUErubrics

Providespointsofcommongroundthroughasharedunderstandingandlanguage

Engagewithemployersinamoremeaningfulmannerthroughdemonstrationconversations

Mapwherelearningoccurs

Integratecurricularandco-curricular

Findpointsofsimilarityanddistinction

Moveintocomprehensivelearnerrecords

Twowaysin:MappingandAssignments

Facultyareworkingtocreateacurriculumthatintentionallybuildsinintegrated

learningopportunitiesovertimeforstudentstoapplyandpracticeaswellastransfertheirknowledgeandskillsthroughassignments,in

andoutofcourses.

MappingNationally

FindingsfromNILOA’sSurveyofProvosts2017

77%reportcurrentlymappingcurriculum

LearningOutcomesareIncreasinglyAligned

At50%ofcampuses:allprogramshavePLOsandalignthosePLOswithILOs

AssessmentforStudentSuccessisbasedonauthenticmeasuresof

studentlearningClassroom-

basedassessment

Rubrics

NationalStudentSurveys

CurriculumMapping:TheProcessFocusedoncurriculumandprogramlearningoutcomes

Two-dimensionalmatrixrepresentingcoursesononeaxisandoutcomesontheother

Facultyidentifywhichcoursesaddresswhichlearningoutcomes

Emailaroundanexcelsheetforfacultytoself-identify

Departmentchairsindividuallycompleteamap

Groupsoffacultycometogethertomapcoursestolearningoutcomes

MapLevel1LearningOutcomes

Course1 Course2 Course3 Course4

LearningOutcome1 X X

LearningOutcome2 X

LearningOutcome3 X X X

But…..Isitanindividualprocessoroneofconsensusbuilding?

Iftwofacultymembersindividuallymappedthecurriculumwouldtheyendupwiththesamemap?

Ifstudentsmappedthecurriculum,wouldtheyhavethesamemapasfaculty?

Whatisstandardofalignment?Isonepaperinoneclassenough?

Dowehaveasharedunderstandingofwhatwearebuildingtowards?

Intr

oduc

tory

Co

urse

Rese

arch

M

etho

ds

Adv

ance

d Co

nten

t Co

urse

A

Labo

rato

ry /

Pr

acti

cum

Co

urse

Adv

ance

d Co

nten

t Co

urse

B

Adv

ance

d Co

nten

t Co

urse

C

Adv

ance

d Co

nten

t Co

urse

D

Caps

tone

Co

urse

Content

SLO 1: Disciplinary knowledge base (models and theories)

Introduced Reinforced Reinforced Reinforced ReinforcedMastery / Assessed

SLO 2: Disciplinary methods Introduced Reinforced ReinforcedMastery / Assessed

SLO 3: Disciplinary applications Introduced Reinforced Reinforced ReinforcedMastery / Assessed

Critical ThinkingSLO 4: Analysis and use of evidence

Introduced Reinforced Reinforced ReinforcedMastery / Assessed

SLO 5: Evaluation, selection, and use of sources of information

Introduced Reinforced Reinforced ReinforcedMastery / Assessed

CommunicationSLO 6: Written communication skills

Introduced Reinforced Reinforced ReinforcedMastery / Assessed

SLO 7: Oral communication skills

Introduced Reinforced ReinforcedMastery / Assessed

Integrity / ValuesSLO 8: Disciplinary ethical standards

Introduced Reinforced ReinforcedMastery / Assessed

SLO 9: Academic integrity Introduced Reinforced Reinforced Reinforced ReinforcedMastery / Assessed

Project ManagementSLO 10: Interpersonal and team skills

Introduced Reinforced ReinforcedMastery / Assessed

SLO 11: Self-regulation and metacognitive skills

Introduced Reinforced Reinforced ReinforcedMastery / Assessed

Center for University Teaching, Learning, and Assessment Sample Curriculum Map (Level of Skill)

http://uwf.edu/cutla/

Intr

oduc

tory

Co

urse

Rese

arch

M

etho

ds

Adv

ance

d Co

nten

t Co

urse

A

Labo

rato

ry /

Pr

acti

cum

Co

urse

Adv

ance

d Co

nten

t Co

urse

B

Adv

ance

d Co

nten

t Co

urse

C

Adv

ance

d Co

nten

t Co

urse

D

Caps

tone

Co

urse

Content

SLO 1: Disciplinary knowledge base (models and theories)

Exam Questions Exam Questions Exam Questions Exam Questions Exam QuestionsCapstone Portfolio

SLO 2: Disciplinary methods Exam Questions Exam Questions Exam QuestionsCapstone Portfolio

SLO 3: Disciplinary applications

Exam Questions Exam Questions Class Project Term PaperCapstone Portfolio

Critical ThinkingSLO 4: Analysis and use of evidence

Term Paper Lab PaperClass

PresentationTerm Paper

Capstone Portfolio

SLO 5: Evaluation, selection, and use of sources of information

Annotated Bibliography

Term Paper Lab Paper Term PaperCapstone Portfolio

CommunicationSLO 6: Written communication skills

Reflection Essays

Lab Paper Term Paper Term PaperCapstone Portfolio

SLO 7: Oral communication skills

Class Presentation

Poster SessionClass

PresentationClass

PresentationIntegrity / ValuesSLO 8: Disciplinary ethical standards

Reflective Paper

IRB/ACUC Proposal

Reflective Paper

Capstone Portfolio

SLO 9: Academic integrityClass

Assignments & Exams

Exams & Term Paper

Class ExamsClass

Assignments & Exams

Class Assignments &

Exams

Exams & Term Paper

Exams & Term Paper

Capstone Portfolio

Project ManagementSLO 10: Interpersonal and team skills

Peer Review of Team Skills

Project Client Feedback

Peer Review of Team Skills

Capstone Portfolio

SLO 11: Self-regulation and metacognitive skills

Class Assignments &

Exams

Class Assignments &

Exams

Class Assignments &

Exams

Exams & Term Paper

Capstone Portfolio

Center for University Teaching, Learning, and Assessment Sample Curriculum Map (Assignments & Embedded Assessments)

http://uwf.edu/cutla/

TakeAwayMappingisalens,it’sawayofseeingsomethingsandthereforenotseeingothers.Oncewerealizeit’salens,weneedtoconsiderthelayersthatcanbemappedaswellashowourstudentsmovethroughandexperienceourintentionaldesign.

AlignmentHowdoyouensurealignmentbetweenourassignmentsandagivenlearningoutcomeforalearningexperience?Whatelementsofthelearningenvironmentdoweneedtoalign?Howmuchis“enough”tobeplacedinamap?Doesitneedtobetightlyalignedorlooselyaligned?

LearningOutcomes

GeneralEducation MajorCourses ActivitiesandExperienceThatProvideSupport

PossibleCareers

LearningOutcome1

Generaleducationcoursesthatsupportthelearningoutcomes

Coursesthataddressspecificoutcomes

Cocurricularelementsthatsupportspecificoutcomes

Possiblecareerpathsrelatedtothemap

LearningOutcome2

LearningOutcomes

PriorLearning

Courses OtherRequiredCourses,RecommendedElectives

ActivitiesandExperienceThatProvideSupport

Work-BasedLearningExperiences

CertificationsandLicensures

PossibleCareers

LearnerIdentified

LearningOutcome1

Priorlearningthatisacceptedinrelationtospecificoutcomes

Coursesthataddressspecificoutcomes

Othercoursesthatsupportandreinforcespecificoutcomes

Cocurricularelementsthatsupportspecificoutcomes

Employmentandotherexperiencesthatreinforcespecificoutcomes

Possiblecertificationsconnectedtotheoutcomes

Possiblecareerpathsrelatedtothemap

Elementsidentifiedbylearnersassupportinglearningoutcomes

LearningOutcome2

TransparencyAwarenessofLearningOutcomeStatements

Arestudentsevenaware?Studentfocusgroupsrevealthatmoststudentsarenotevenawaretheyarebeingassessed.Butuponlearningaboutassessment…wishtheyhadbeen

toldaboutitearlier.

Withwhomdowesharethemaps?Advising?Students?Orientationfornewfaculty?Adjuncts?Others?

LearningSystemsParadigmMappinginaLearningSystemsParadigmasksustotakeawiderlenstothinkdeeplyaboutthevariousplaceswherelearningishappeningthataren’tdocumentedinourtraditionalmaps.

Consensus-basedremindsusthatweeachhavearoletoplayinsupportingourlearnersachievementofoursharedlearningoutcomes.

Alignmentremindsusthatweneedtoexploretheconnectionspointsbetweenvariouslearningexperiencestosupportintegrationandtransferabilityoflearning.

Learning-centeredasksustobemindfulofourassumptionsandimplicitdesign,andworkwith ourlearners.

Communicatedstressesthatallofourworkisfornaughtifwedon’tshareitwithothersandtellthemwhywearedoingthingsthewayweare.

Withwhomdowesharethemaps?Advising?Students?Orientationfornewfaculty?Adjuncts?Others?

LearningSystemsParadigmMappinginaLearningSystemsParadigmasksustotakeawiderlenstothinkdeeplyaboutthevariousplaceswherelearningishappeningthataren’tdocumentedinourtraditionalmaps.

Consensus-basedremindsusthatweeachhavearoletoplayinsupportingourlearnersachievementofoursharedlearningoutcomes.

Alignmentremindsusthatweneedtoexploretheconnectionspointsbetweenvariouslearningexperiencestosupportintegrationandtransferabilityoflearning.

Learning-centeredasksustobemindfulofourassumptionsandimplicitdesign,andworkwith ourlearners.

Communicatedstressesthatallofourworkisfornaughtifwedon’tshareitwithothersandtellthemwhywearedoingthingsthewayweare.

AssignmentsasAssessments

Facultyareworkingtocreateacurriculumthatintentionallybuildsinintegrated

learningopportunitiesovertimeforstudentstoapplyandpracticeaswellastransfertheirknowledgeandskillsthroughassignments,in

andoutofcourses.

Assignmentsasawayin

Assignment

ScaffoldingLearning

EvaluativeCriteria

LearningOutcomes

Assignment

ScaffoldingLearning

EvaluativeCriteria

LearningOutcomes

Assignment

ScaffoldingLearning

EvaluativeCriteria

LearningOutcomes

Assignment

ScaffoldingLearning

EvaluativeCriteria

LearningOutcomes

Howassignmentsconnect

ProgramView

Assignment

ScaffoldingLearning

EvaluativeCriteria

LearningOutcomes

MajorGeneralEducation

Connections

Assignment

ScaffoldingLearning

EvaluativeCriteria

LearningOutcomes

Employers

Co-curricular

Implicationsfortranscripts,careerdevelopment,andpathways

Resources:Toolkit

http://www.learningoutcomeassessment.org/assignmenttoolkit.html

MoreResources

QuestionsEmail:niloa@education.illinois.edu

njankow2@Illinois.edu

http://www.learningoutcomesassessment.org

www.assignmentlibrary.org

www.degreeprofile.org