closing the loop session 5-7-18 - duke university

Closing the loop Strategies for using your findings

May 2018

Suskie,L.(2018).Assessingstudentlearning:Acommonsenseguide.JohnWiley&Sons.

Today’sdiscussionadaptedfrom:

Waysofguidingpedagogy,curriculum,

andadvising

Waysofunderstandingstudentlearning

SHARINGACTING

Waysofguidingpedagogy,curriculum,

andadvising

Waysofunderstandingstudentlearning

IdentifyStudentLearningOutcomes

Collectinformation

Analyzeandunderstandinformation

Sharing(Discuss,reflect)

Planandact

Waysofunderstandingstudentlearning(assessment)

Waysofguidingpedagogy,curriculum,

andadvising

SharingDiscussingwithothershow

weexaminestudentlearning

Discussingwithotherswhatwe’velearnedabout

studentlearning

Acting

Implementingevidence-guidedchangestotheway

weexaminestudentlearning

Implementingevidence-guidedchangestothelearningprocessand

environment

Waysofunderstandingstudentlearning(assessment)

Waysofguidingpedagogy,curriculum,

andadvising

SharingDiscussingwithothershow

weexaminestudentlearning

Discussingwithotherswhatwe’velearnedabout

studentlearning

Acting

Implementingevidence-guidedchangestotheway

weexaminestudentlearning

Implementingevidence-guidedchangestothelearningprocessand

environment

SHARINGDiscussingwithothers

howweexaminestudentlearning

FullengagementwiththecommunityofstakeholdersClarityofstandardsandmethodologyshouldcompelacceptanceoffindings Transparencyofprocess:

HowareStudentLearningOutcomesdeveloped?Whydoyouusethemeasuresyoudo?Whywerethetargetsselected?

SHARINGDiscussingwithothers

howweexaminestudentlearning

Towhom:

Facultycolleagues,Students,Collegeleadership,Externalpartners

Where:

Inprint:Tradepublications,memos,alumnimagazines

Online:Newsletters,blogs,emailsInmeetings:Facultymeetings,

studentsmeetings,workshops,retreats

When:

June1portfoliodeadlineJWhenyourinitialinterpretations

arecompleteWhenthesepartnersarereadyto

receiveanduseinformation

Waysofunderstandingstudentlearning(assessment)

Waysofguidingpedagogy,curriculum,

andadvising

SharingDiscussingwithothershow

weexaminestudentlearning

Discussingwithotherswhatwe’velearnedabout

studentlearning

Acting

Implementingevidence-guidedchangestotheway

weexaminestudentlearning

Implementingevidence-guidedchangestothelearningprocessand

environment

Towhom:

Facultycolleagues,Students,Collegeleadership,Externalpartners

Where:

Inprint:Tradepublications,memos,alumnimagazines

Online:Newsletters,blogs,emailsInmeetings:Facultymeetings,

studentsmeetings,workshops,retreats

When:

June1portfoliodeadlineJWhenyourinitialinterpretations

arecompleteWhenthesepartnersarereadyto

receiveanduseinformation

SHARINGDiscussingwithotherswhatwe’velearned

aboutstudentlearning

•  Shareresultswhichyouraudiencecandosomethingwith

•  Emphasizemeaningfulresults

•  Brevity!

•  Narrate:Tellthestoryofstudentlearning

•  Usegoodteachingpractice

•  Transparency:Beclearabouttheprocessbywhichtheassessmentplanwasdeveloped,andwhyyourteamsettheprioritiesitdid.

•  Illustrations:Figures,tables,visualsmaybemorecompellingandpersuasivethatwrittentext.

Goodpracticestoconsider

SHARINGDiscussingwithotherswhatwe’velearned

aboutstudentlearning

•  Sharegeneralizedfindingswiththosewhohavearesponsibilityforteachingexcellence(facultycolleagues,programofficers,Collegeleadership).

•  Discussyourgeneralinterpretationswithstudentgroups(e.g.,majorsunions).

•  Keepindividualstudentdataconfidentialandsecure.

•  Withrespecttopresentationsandpublications,seekpriorauthorizationfromtheDeanofAcademicAffairsinTrinityCollege.

•  FollowtherequirementsofyourIRBprotocol,ifapplicable.

Somecommentsondatasecurityandprivacy

SHARINGDiscussingwithotherswhatwe’velearned

aboutstudentlearning

•  Emphasizethebigpicture.

•  Consultwiththosewhohaveconcerns,especiallyintheplanningstages.Talkwiththosewhohaveastakeintheresults.

•  Balancenegativeswithpositives.Begentleandsensitive.

•  Providecorroboratinginformation(e.g.,aliteraturereview).

•  Documenttherigorandqualityofyourassessmentstrategy,butacknowledgeitslimitations.

•  Guideyourpartnersthroughpossiblesolutions

Whatifyouraudienceisuncomfortablewiththeresults?

SHARINGDiscussingwithotherswhatwe’velearned

aboutstudentlearning

Waysofunderstandingstudentlearning(assessment)

Waysofguidingpedagogy,curriculum,

andadvising

SharingDiscussingwithothershow

weexaminestudentlearning

Discussingwithotherswhatwe’velearnedabout

studentlearning

Acting

Implementingevidence-guidedchangestotheway

weexaminestudentlearning

Implementingevidence-guidedchangestothelearningprocessand

environment

ACTING

Dowehaveaprocessforreviewingtherigorofourassessmentstrategy?Canweimprovetheprocessbywhich…

ourprioritiesareestablished?ourdataarecollectedandunderstood?

Let’simplementbetterwaysofunderstandingstudentlearninginourprogram.

Implementingevidence-guidedchangestothewayweexaminestudentlearning

ConsiderlearninggoalsDoyouhavetoomany?Aretheyappropriateforundergraduateeducationinourdiscipline?Aretheyspecificandmeasureable?Shouldyouupdatethem?Wouldithelptomapouttherelationshipsbetweencourses?

ConsiderthemeasuresbywhichyoucollectedevidenceDidtheyrevealtheinformationyouneed?Didyouusemultiplesourcesofevidence?Areyoublendingdirectandindirectevidence?

ConsideryourtargetsWeretheyappropriate?Ifyoumetyourtargets,shouldyouraiseyourexpectationsnextyear?

ConsideryourinterpretationDidyouhaveenoughoftherightsortofevidencetodrawconclusionsaboutundergraduatestudentlearning?

ACTINGImplementingevidence-guidedchangestothewayweexaminestudentlearning

Waysofunderstandingstudentlearning(assessment)

Waysofguidingpedagogy,curriculum,

andadvising

SharingDiscussingwithothershow

weexaminestudentlearning

Discussingwithotherswhatwe’velearnedabout

studentlearning

Acting

Implementingevidence-guidedchangestotheway

weexaminestudentlearning

Implementingevidence-guidedchangestothelearningprocessand

environment

ConsiderthecurriculumDostudentshaveenoughopportunitiestopracticeandmastercompetenciesovertime?Docoursesbuildoneachotherinathoughtfulway?Arepre-requisitesappropriate?

ConsidercoursecontentArecourses’contentappropriate?Dosyllabiexplainwhatstudentswilllearnineachcourse?Arelearningtasksalignedwiththepurposeofthecourse?

ConsiderpedagogyDoweneedtoadaptourteachingmethods?Dothefacultyneedtimetodeveloptheirteaching?Doweneedtoexplorenewthinkinginstudentdevelopment,multipleintelligences,learningstyles,andepistemologies?

ConsideradvisingandacademicsupportArestudentsplacingintosuitablecourses?Arestudentssequencingcoursesandco-curricularopportunitieseffectively?Aresupportservicesutilizedandimpactful?

ACTING

Implementingevidence-guidedchangestothelearningprocessand

environment

Giveattentionandresourcestoissuessupportedbyassessment

evidence.

Youknowyourdisciplineandyourfacultycommunitybest.

Whatifeverythingisgoingwell?

Congratulateyourselves!Thenconsider…

•  Continuingtocollectdatatoconfirmstabilityoflearning•  Raisingyourstandardsforexpectedperformanceamongstudents•  Shiftingyourattentiontoanewlearningoutcome,withplansto

checkonthisoneagaininafewyears.

Worksheet

Waysofunderstandingstudentlearning(assessment)

Waysofguidingpedagogy,curriculum,andadvising

Sharing

Acting

Waysofunderstandingstudentlearning(assessment)

Waysofguidingpedagogy,curriculum,andadvising

Sharing

• Createflowchartofassessmentplan;introduceatfacultymeeting.Solicitfeedback.• Requestregularizedtimeonthefacultymeetingagendatoupdatecolleaguesonassessmentprocess

• Circulateasummarymemooffindingspriortoafacultyretreat.Preparediscussionquestions.• Hostabrownbagdiscussionofresultsbetweenfacultyandu-gradmajors.

Acting

• Assembleavertically-integratedteamtocraftstudentlearningoutcomes.• Conveneaworkinggrouptoreviseaproblematicrubric.• Ifstudentsaremasteringanoutcome,shifttoanewoutcome.• WorkwiththeOfficeofAssessmenttobuildacurriculummapoftheprogram.Collect&alignsyllabi.

• Appointacommitteetoproposerevisionstoaspecificcourse• Developevidence-basedplacementguidelinesfortherecommendedfirstcourseintheprogram• ConsultwithLearningInnovationstoreviewalternativeformatsforcoursesormodules.

Examples(seeworksheet)

Table17.4.TipsforengagingyouraudiencesTable17.6.TipsfortellingthestoryofyourCollege[orprogram’s]effectivenessTable18.1.Usingassessmentresultsfairly,ethically,responsiblyTable18.2.StrategiestoaddmoreintensivestudyofakeylearninggoalTable18.3.Strategiesthatpromotedeep,lastinglearning

ResourcesfromLindaSuskie’sAssessingStudentLearning,2nded.(2009)