1

MeasureWhatMatters(AndEverythingMatters):AssessingSustainabilityInitiativesRobertFranco,FranciscoAcoba,andKristaHiser,Kapi‘olaniCommunityCollege,HonoluluHI

Assessingsustainabilityinitiativesisamatteroffocus.Astheabove“target”graphicillustrates,zoominginononeaspectoftenputsothersattheperiphery.What’satthecenterforyou?Somecollegeskeepgreatdataoncommunityimpact,butdon’tassessstudentlearningaboutsustainability.Ontheotherhand,assessingstudentlearningaddressesdifferentquestionsandissuesthanassessingcommunityimpact.Astheadagegoes:“MeasurewhatMatters.”However,inthecaseofsustainability,everythingmatters!Whatresourcesandmethodologiesareavailableforyourassessment?Whatmatterstoyou,yourstudents,yourcampusleadership,yourcommunityandstate,andyouraccreditationagencies?Thischapterprovidesideasfordesigningastreamlinedmulti-prongedassessmentmodel,basedonKapi‘olaniCommunityCollege’sEngagement,LearningandAchievementframework(KELA).Thechapterlistsavailabletoolsformonitoringthemultipleimpactsofcampussustainabilityinitiatives,includingtheCommunityCollegeSurveyofStudentEngagement(CCSSE),participant-observerresearch,pre-andpost-testing,StudentAssessmentoftheirLearningGains(SALG),theSustainabilityLiteracyTest(Sulitest),areflectionessayevaluationprotocol,andtheCarnegieFoundation.STUDENTENGAGEMENT,LEARNINGANDACHIEVEMENTCommunitycollegesintheU.S.areintentlyfocusedonstudentachievement:retentionandcompletionnumbers,liberalartstransfer,careerandtechnicaleducationdegrees,jobplacementandevenstartingsalary.But,asuniquelyAmericaninstitutions,fromtheirinception

2

morethan110yearsago,communitycollegesarealsopreparingtheirstudentsfortheworkofdemocracyandinformedparticipationascitizensengagingindemocraticsystemslocallyandnationally,aswellascollaboratorsinarapidlydevelopingglobalsystem.Kapi’olaniCommunityCollegeusesamulti-facetedinstitutionalevaluationframeworkknownastheKELAmodel:Kapi‘olaniEngagement,Learning,andAchievement.Thisframeworkfocusesfacultyandstaffplanningandprogramimplementationonafewcentralvariables.Theachievementandengagementvariablesarequantitative:achievementvariablesaretrackedeachsemesterwhileengagementvariablesaretrackedeverytwo-yearsthroughtheadministrationoftheCommunityCollegeSurveyofStudentEngagement(CCSSE).Courselearningoutcomesareassessedbyindividualfacultywhilegeneraleducationoutcomesareassessedthroughrubric-basedanalysesofstudentartifacts(usuallywritingassignments)conductedbyfacultycommunitiesofpractice.

3

EngagementToassessstudentengagementthecollegereliesheavilyontheresultsofitsCCSSE,whichitadministersinspringsemestersineven-numberedyears.TheCCSSEorganizesitsreportingunderfivemajorbenchmarks:

1. Faculty-StudentInteraction2. Active-CollaborativeLearning3. AcademicChallenge4. StudentEffort5. StudentSupport

Onecouldhypothesizethatcurricularandpedagogicalpracticesconsideredbestpracticesinsustainabilitylearning;thatis,AppliedLearning(campusasalivinglaboratory),service-learning,place-basedlearning,learningcommunities,undergraduateresearch,andresearchandindustryinternships,shouldresultinhigherlevelsofstudentengagementacrossallfivebenchmarks.SustainabilitypedagogiestendtobeHighImpactPractices,especiallyforstudentshistoricallyunderrepresentedinhighereducation(Kuh).Kapi‘olaniusesAASHESTARScriteriatoidentifySustainabilityFocusedcourses,whichmustidentifyahigh-impactpedagogyaspartofthecoursedesignation.AssessinglearningintheseSustainabilityFocusedcoursesprovidesanopportunitytocomparethesestudentswhocompletethemandthosewhodonot.Weshouldnotpretendthatallthecontrolsrequiredofpureexperimentalresearchcanbedevelopedandappliedinthehighlydiversestudentpopulationsinthecommunitycolleges.Weshouldinsteadstriveforconsistentlyandcyclicallyappliedassessmentandevaluationmethodsthatmeetthecriteriaof“quasi-experimental”research(seeHilletal).Theresultsofthesestudiesshouldsparkandsustainadministrativeandfacultydialogueaboutimprovement.Thecommunitycollegeenvironmentitselfprovidesuniquechallengestoassessmentdesign.Forexample,identifyinggroupsof“statisticaltwins”forpureexperimentalresearchisonerous,especiallywithcommunitycollegestudentswithdiversedemographicandlinguisticbackgrounds.Addedtothisfoundationaldiversityaredifferencesinhighschoolpreparation,accesstofinancialsupport(familyandcollege-supported),full-timeandpart-timestatus,patternsofstoppingoutandreturning,clarityaboutcourse-takingchoicesandpathways,clarityaboutemploymentaspirations,andunderstandingoftherightsandresponsibilitiesofcitizenship.Allthisvariabilityhasadirectimpactonwhetherstudentsgainasenseofbelongingatthecampus,howlongittakesthemtogainthissenseofbelonging,andthenwhethertheygainasenseofbecomingthatenablesthemtocompletecertificatesanddegrees,andattaingainfulemploymentorsuccessfultransfertoabaccalaureatecampus.Thisishowwedefineengagement.

4

Facultymustdesigneffectiveinteractionswithstudentswhoareunsuretheybelongandhavenotidentifiedwhattheywillbecome.Thebestcommunitycollegefacultyseetheircourseasacentralpartofthisbelonging-becomingcontinuum,strivetounderstandthateachoftheirstudentsaresituateddifferentlyonthiscontinuum,andtrytobuildonthissituated-ness.Mostcommunitycollegefacultyareteachingfivecoursesectionspersemester,oftenwithmultiplecoursepreparations.Facultythemselvesareonacontinuumofcaringandnurturing,andtheirworkloadsandclasssizesarenotalwaysamenabletoteachingdifferentlybyusingresearch-basedhighimpactpracticeswhichenablemoreindividuallycustomizedlearningopportunitiesfordiversestudents.Toengagemorefacultyinhighimpactandempoweringsustainabilityteachingpracticesrequiresstrongstaffsupporttomaintain,assess,andimprovecommunitycollaborationsandpartnerships.Thisiswhywemeasurewhatmatters.Sustainabilityteachingandlearning,curriculumandpedagogy,donewell,providearichandrelevantcontextformanygeneraleducation,careerandcapstonecoursesonthebelonging-becomingcontinuum.ButwhatdoesdonewellmeanforLearning,AchievementandEngagementthroughsustainabilityinitiatives?Totacklethequestionofsustainabilityeducationdonewellwewouldalsowanttodevelopmethods,bothquantitativeandqualitative,thatinformusaboutstudentsustainabilitylearning.AchievementTheKELAmodelfocusesonsevenstudentachievementmeasures:

1. CourseSuccess(percentofstudentcompletingaCorbetter)2. Re-enrollmentratesfromfalltospringsemester3. Re-enrollmentratesfromfalltosubsequentfall4. Annualnumberofcertificatesofachievementanddegreescompleted5. Annualnumberoftransfertobaccalaureatecampuses6. Percentofstudentscompletingcertificatesofachievementanddegreesina

three-yearperiod7. Percentofstudentstransferringtoabaccalaureatecampusinthreeyears

Let’senvisionhowwemightdocumenttheimpactofSustainabilitycoursesontheseachievementmeasures.Measures4-7aboveareparticularlyimportantsinceachievingthesemeasuresoftenresultsinadditionalfundingtocampusesfromstatelegislaturesandsystems.InthecurrentcurricularenvironmentatKapi‘olani,mostSustainabilitycoursesareactuallycoursesectionswithspecificcourseregistrationnumbersandaSUSTidentifier.Notallsections

5

ofacourseareSustainabilitydesignated.Forexample,onlyafewEnglish100(first-yearcomposition)courseshavethesustainabilitydesignation;notallthePsychology100courseshavethedesignation.ManycoursesaretaughtbydifferentfacultyandnotallthesefacultypursuetheSustainabilitydesignationforwhattheyperceiveastheircourse.However,thisSUSTversusnon-SUSTcourseidentificationcanbeviewedasanassessmentopportunity.Wecould,forexample,comparemeasures1and2inSUSTandnon-SUSTcoursesinthefallsemesters,andmeasures1and3inspringsemesters.Wecouldcomparemeasures1,2,and3inSUSTandnon-SUSTcoursestaughtbythesamefacultyinthesamesemester.Overmultiplesemesterswecouldtrackstudentswhocomplete2ormoreSUSTcoursesandestablishtheiraveragetocertificateordegreecompletionand/ortransfer.WecoulddetermineabaselineforstudentswhointheirfirstsemestersuccessfullycompletedaSUSTcourseandthencompletedadegreeofcertificateand/ortransferredinsixsemesters.WecoulddeterminetheaveragenumberofSUSTcourses,onadegreeortransferpathway,thatyieldedthebestcompletionandtransferresults.Inalltheseinquiries,wecouldassesswhatpedagogicalapproachesmadethemostbeneficialimpact.ThefuturecurricularenvironmentatKapi‘olaniincludesanAcademicSubjectCertificateinSustainability,andwecantrackandassessthesecertificatepursuersandcompletersonmeasures4-7above.Ifwecanshowthatthesecompletershavebetteroutcomesonthesemeasures,weshouldthenbeabletorequestandreceiveadditionalfundingfromthecampus,state,system,andexternalfunders.Learning Pre-andPost-TestSurveyFrom2014-2016,Kapi‘olani,inpartnershipwiththeCommunityCollegeNationalCenterforCommunityEngagement,andtheMaricopaCommunityCollegeSystem(AZ),leada3-yearTeagleFoundation-fundedprojectentitled,“StudentLearningforCivicCapacity:StimulatingMoral,Ethical,andCivicEngagementforLearningthatLasts”(seeteachingtobigquestions.wordpress.com).Sixcommunitycollegesparticipatedintheproject:Kapi‘olani(HI),Mesa(AZ),Delgado(LA),RaritanValley(NJ),andKingsboroughandQueensborough(NY).Allparticipatingcampusesdevelopedcommunitiesofpracticeengaginganaverageoftenfacultyperyearwhotaughttoasinglebigquestion:HowdowebuildOURcommitmenttocivicandmoralresponsibilityfordiverse,equitable,healthyandsustainablecommunities?

6

Asacentralcomponentoftheproject,wedevelopedapre-andpost-testsurveywithitemsetstargetingdiversity,equity,healthandsustainability.Eachsetaddressedstudents’civicandmoralengagement,behavior,andattitudes.Thespecificsustainabilityitemsonthesurveyarelistedbelow.Theallowablestudentresponseswere:completelydisagree,disagree,undecided,agree,andcompletelyagree.

1. Iamalreadyorwillbecomeactiveinenvironmentalprograms.2. Iamwillingtoconsumelessandgowithoutsomecomfortsifithelpstoprotectthe

environment.3. WhenIbuyaproduct,Iassessthetypeofpackagingandchooseonethatisrecyclable.4. IfIhadtochoosebetweentheconstructionofahighwaythatwillalleviatetrafficorthe

protectionofaplantspecies,Iwouldchoosethehighway.5. Evenifpublictransportweremoreefficientthanitcurrentlyis,Iwouldstillchooseto

usemyowncar.6. Solvingcurrentproblemsismoreimportantthanworryingaboutthefuture.7. Iwanttofindnewwaystolivemoresustainably.8. Climatechangeisasituationthatdemandsourimmediateattention.

Thefirstsurveywasadministeredearlyinthefall2015semesterandthenmatchedwithasecondsurveyadministeredaslateaspossibleinthatsamesemester.Kapi‘olani’sresultswereanalyzedwithaT-testmeasuringgainsfrompre-topost-testsurveyandatwo-wayAnovacomparingservice-learningandnon-service-learningstudents.Inthefirstiterationofthesurvey,analysisshowedthatservice-learningstudents,manydoingenvironment-relatedprojects,scoredhigherthannon-service-learningstudentsononesurveyitem:Iwanttofindnewwaystolivemoresustainably.Analysisalsoshowedallstudentgainsonthefollowingthreeitems:

• Iamalreadyorwillbecomeactiveinenvironmentalprograms.• WhenIbuyaproduct,Iassessthetypeofpackagingandchooseonethatisrecyclable.• Iwanttofindnewwaystolivemoresustainably.

Butanalysisalsoshowedstudentlossesonthreeitems:

• IfIhadtochoosebetweentheconstructionofahighwaythatwillalleviatetrafficortheprotectionofaplantspecies,Iwouldchoosethehighway.

• Evenifpublictransportweremoreefficientthanitcurrentlyis,Iwouldstillchoosetousemyowncar.

7

• Climatechangeisasituationthatdemandsourimmediateattention.Twoitemsshowedlittlechangeeitherway:

• Iamwillingtoconsumelessandgowithoutsomecomfortsifithelpstoprotecttheenvironment.

• Solvingcurrentproblemsismoreimportantthanworryingaboutthefuture.Thereweremethodologicalchallengesintheproject-wideuseofapre-andpost-surveylearningassessmentmethodology.First,sincetheTeagleFoundationwantedbroadpublicationanddisseminationofprojectfindings,weknewwewouldneedinstitutionalresearchboard(IRB)approvalsfromtheMaricopaCommunityCollegesystemaswellasthesixindividualcommunitycolleges.TheseIRBapprovalswereeasilyattainedatsomecampuses,butsomecampuseshadmoredifficulty,especiallyiftheywereinmulti-campussystems.Eachcollegechosewhichcoursestoincludeinthesurveyassessmentanddeterminedwhentoadministerthepreandpostsurveys.Wehadinitiallyhopedtohavethecapacitytopre-andpost-matchindividualstudentresponses.Oneofthegreatestchallengeswasobtainingsuccessfulmatchesforthesurveyadministeredearlyinthesemesterwiththeoneadministeredlateinthesemester.Everyyear,duringannualmeetingsandthroughmanyelectroniccommunications,thecollegesworkedtogethertodevisestrategiestohelpstudentsrememberwhatemailaddresstheyhadusedonthefirstsurveysothatitwouldmatchwiththelatersurvey.Inthe2015assessmentadministration,allsixcampusesadministeredthepre-surveybutonlythreecampuseshadenoughpairedresponsesinthepost-surveystoidentifystatisticallysignificantstudentlearninggains.Anotherchallengewasinterpretationofthesurveyresults.Beginningin1996,Kapi‘olaniCommunityCollegehadoriginallyconductedasurveyeverytwoyearsanditwasonlyafterthreesetsofsurveyshadbeenanalyzedthattheresultsbegantoshowclearpatterns(Renner).WehadhopedthatthreeconsecutiveyearsoftheTeaglesurveywouldalsobegintoshowclearpatterns,butthelackofcoherentpatternswithinandacrosscollegesledustodoamajorrevisionofthesurveyinthethirdyear.Ourgoalsweretoimprovethematchingofearlyandlatesurveysandthusincreasesamplesizeforeachcampus,andalsotobetteralignourquestions–andhopefullyourfindings–withthethemeswithinourTeagleproject.Wealsoincreasedthedemographicsectionofthesurvey.Ultimately,webelievethatthissurveyapproachcanyieldveryusefulinformation,especiallyiflargeenoughmatchedsamplesareobtained.Kapi‘olani’sresultsthatweredescribedabove

8

ledtodiscussionsamongfacultyandchangesinclasses.Further,patternsbasedongender,ethnicity,experienceprovidingcommunityservice,andinvolvementinundergraduateresearchandotherhighimpactpracticescanbeidentifiedandusedtohelpeducatorsguidestudentstowardthepathwaysthatwillbestassisttheminmovingtowarddeeperandmoremeaningfulengagementwiththeircommunities.Thepre-andpost-testassessmentgivesasnapshotofstudentlearningonawidescale—notjustfromspecificcourses.Inaseparatefocusgroupinterview,weheardfromstudentsthattheytendtoconnectlearningaboutsustainabilitymorewithnews,socialmedia,anddocumentariesthanwithspecificcourses.Thisindicatesthatfacultyneedtobemoreexplicitwhenteachingsustainabilityissues.StudentAssessmentoftheirLearningGains(SALG)Aninterestingalternativeinstrument,whichoffersapreandposttypeofviewbutfromamorequalitativeperspective,istheSALG:StudentAssessmentofLearningGains.WhatismostinterestingabouttheSALGisthatitacceptsstudentself-reflectionasevidenceoflearning.Thisisinalignmentwithcurrentresearchonlearning,andperhapswithacommunitycollegepedagogicalperspective.TheSALGwasdevelopedthroughfundingfromtheNationalScienceFoundation(NSF)whichcontinuestorecommenditforevaluationofeducationresearch.SALGdataisacceptedasprimafacieevidenceofstudentlearningbysomeaccreditingbodies,includingACCJC.TheSALGhasbeenusedbyover17,000collegeanduniversityinstructorssince1997,andcurrentlysupportsover14,000individualusersand2000groupusers.UseoftheSALGisfree;anyonemaysignupatwww.salgsite.org.TheSALGissimilartoastudentevaluationofthecourse,exceptitfocusesonthestudent,nottheteacher.Typically,StudentEvaluationsofTeaching(SETs)focusonteacherbehaviorsandusestudentsatisfactionasaprimaryevaluativecriterion.Extensiveresearchshowsthatinnovativeteachers/pedagogiesearnlowerscoresonSETsthantraditionalcourses,evenwhenstudentslearnmoreinthoseinnovativecourses(Seymouretal).Similartothe“negativeresults”describedinthepre-andpost-testing,whenitcomestoassessingsustainabilitylearning,traditionalSETsmayyieldnegativeevaluationsofsustainabilitycurriculum—forexample,ifstudentsdidn’tlikegoingoutside,orworkinginagroup.TheSALGwasspecificallydesignedtoremedythesedeficienciesbyfocusingexclusivelyonstudents’learninggains.Itwasbuiltaroundthreeessentialprinciples:

9

• thatassessmentofacourseshouldberelativetotheexplicitgoalsofthatcourse—i.e.,whatstudentsaresupposedtolearn;

• thatstudentshavesomethingvaluabletotellusaboutwhathelpsthemlearn(andwhatdoesn’t);and

• students’learningshouldbeenhancedbyfillingoutthecourseevaluation(byincreasingtheirmetacognitionabouttheirlearning).

SALG’sdesignusestenquestionsthatcomprisethecentralidentityoftheinstrumentandencouragesinstructorstocreateacustomizedinstrumentforeachcoursetheyteach.Forexample,manyofthequestion-stemsincludesub-questionsthataskinstructorstofillindetailsofclassactivities,assignments,keyclassconcepts,andsoon.Finally,instructorsmayaddadditionalquestionstotheoriginallistoften.Whenwritingthesenewquestions,instructorsmaychoosefromavarietyofestablishedresponsescales,ortheymaycreatetheirownscales.Thisflexibility,too,iscentraltotheidentityoftheSALGbecauseitallowsinstructorstoassesstheeffectivenessoftheirpedagogicalchoicesinhelpingstudentsreachthelearninggoalsfortheircourse.HereareafewoftheitemsusedtosurveystudentsinKapi‘olani’sSustainabilityFocusedcourses.Thechoicesare:nogains,alittlegain,moderategain,goodgain,greatgain,andnotapplicable.

Asaresultofyourworkinthisclass,whatGAINSDIDYOUMAKEinyourUNDERSTANDINGofeachofthefollowing:1.Thefollowingconceptsthathavebeenexploredinthisclass• SustainableEconomics(triplebottomline,alternativetoGDP)• EcosystemServices• EcologicalFootprint/CarbonFootprint• SustainableMaterialsManagement(CradletoCradleDesign)• ClimateChangeMitigation,Adaptation,andResilience• LocalFirst• SocialJustice&Equity2.HowthesecoreconceptsofsustainabilitymightrelatetotheclassI'mtaking.3.HowideasfromthisclassrelatetootherclassesIhavetaken.4.Howstudyingthissubjectareahelpspeopleaddressrealworldissues

TheSALGwascreatedtopromoteandsupportthoughtful,effectiveteachingandsystematic,rationalimprovementofthatteachingbyprovidingmeaningfulanddetailedfeedbackabout

10

thevalueandefficacyofclassroomteaching.Becauseitprovidesvalidandmeaningfulevidenceofstudentlearning,SALGdatacanbeusedtosupportcasesforpromotionandtenurefarmoreeffectivelythanSETinstrumentsthatfocusonstudentsatisfactionandhavescantrelevancetostudentlearninggains.Service-LearningReflectionEssayMostsustainabilitydesignatedcoursesatKapi‘olanidoservice-learning.Thisisbasedontheconvictionthatlearningaboutsustainabilitymeanspracticingitaswell.Sustainabilityteachingcanbedisempoweringandpushstudentstothinkingnothingcanbedonetofixtheseglobalproblems,thattheirfateissealed.Service-Learningempowersstudentstodosomethingnowaboutsustainabilityissuesconfrontingtheircommunities.TheKapi‘olaniServiceandSustainabilityLearningprogramrequiresasemestercapstoneessayinwhichstudentsreflectontheirexperience.Herearereflectionprompts:

• Issue:Identifythesocietalorecologicalissueyouhavehelpedtoaddressthroughyourservice.Explainhowyouhavehelped.

• Learning:Discuss3concepts/theoriesfromyourcourseworkthathavehelpedyoudoyourservice.Describeexperiencesduringyourservicethathavehelpedyouunderstandthoseconcepts/theories.

• CivicContext:Asaninformedindividualandcitizen,discusstheissueyouexplainedaboveasapublicorcommunityproblem.Whatelementsofunfairnessorinjusticedoestheproblemhave?Doyoubelievemorepeopleshouldcareabouttheproblem?Whyorwhynot?Discussoneormoresolutionstotheproblem.

• Goals:Explainhowyourcourseworkandserviceactivitieshaveshapedyourpersonal,academicorprofessionalgoals.Fromthelistbelow,selectalltheonesyouareinterestedindoinganddiscussatleastoneindetail.Explainhowtheaction(s)youwilltakewillhelpreducetheimpactoftheproblem.1. Supportfamily,friends,and/orneighborswhoareaffectedbytheproblem2. Serve,orfundraisetosupport,acommunity-basedorganizationthatisworkingon

theproblem3. Patronizebusinessesthatareactivelyworkingtolessentheseverityoftheproblem

orarecommittedtonotmakingtheproblemworse4. Conveneadialogwithpolicy-makerswhoareworkingontheproblem5. Advocatewithpublicofficialsand/orlegislatorswhoareworkingontheproblem6. Start,leadorjoinacampusgroupthatisworkingontheproblem7. Takeanothercoursetogainanewperspectiveontheproblem8. Completeadegreethatwillprovidemewiththeknowledge,skills,andattitudesto

workontheprobleminmyprofession

11

9. Other(specify)Thepromptscandemonstratesustainabilitylearningintheseways:UnderIssue,thestudentcandefineasustainabilityissue,suchasinvasivespecies.UnderLearning,sustainabilitydesignatedcourseshavesustainabilitySLOs,suchas“Measureone’simpactonthetriplebottomline:People,Planet,Profit.”Studentscanspecificallyexplainhowtheylearnedandappliedtheseconceptsaspartoftheirservice.UnderCivicContext,studentsoftenindicatesocialjusticeissues,suchasenvironmentaljustice.Forexample:howdoesclimatechangeasymmetricallyaffectPacificIslanders?UnderGoals,studentsexplaintheirplansthatsupportsustainability.Forexample:alotofbotanystudentswanttomaketheirownnativeplantgardens.Writtenresponsestothesepromptsmoredirectlydemonstratewhatstudentshavelearned.However,wellwrittenpiecestaketimeandexperiencetowrite,whichmanyofthestudentsdonothave.About2/3ofourservice-learningstudentsaretakingascienceorlanguage(Chinese,JapaneseorKorean)classinwhichsubstantialwritingorwritinginEnglishisnotamajorcomponent.Theseclassesdonotrequirefirst-yearcompositionasapre-requisite.Further,aboutathirdofourstudentsareinternational,anduptohalfspeakEnglishasasecondlanguage.Itseemstoomuchtoaskthemajorityofthesestudentstoproduceahigh-quality(English-language)essaythattheircurrentclassorpriorclasses/experienceshavenotnecessarilypreparedthemtowrite.Inanycase,essaysarecollectedbytheKapi‘olaniService&SustainabilityLearningOfficeandarandomizedselectionofessaysareassessedtwiceayear(seeteachingtobigquestions.wordpress.com).Whileworkintensive,thisassessmenthelpsfacultybetterunderstandstudentlearningachievementwhileprovidingasenseofcollegialityandconstantimprovementtotheservice-learningprogram.COMMUNITYANDCIVICENGAGEMENTToassesstheimpactofsustainabilitycurriculumandpedagogyoncommunityandcivicengagement,wecanexpandthestudentcapstonereflectionmethodologydiscussedabove.Wealsoneedtodevelopasimilarqualitativeassessmentapproachwiththecommunityandcivicpartnersweengageinoursustainabilityprogramming.AftermorethanadecadeofparticipationinthedevelopmentandimplementationoftheCarnegieFoundations’CommunityEngagementClassification,andfollowingthenationalresearchinthisarea,itisusefultoenvisionhowthatclassificationmighthelpusassessandimprovecommunitysustainabilityandresilienceefforts.HereistheCarnegieFoundationdefinitionofcommunityengagement:

12

Communityengagementdescribesthecollaborationbetweeninstitutionsofhighereducationandtheirlargercommunities(local,regional/state,national,global)forthemutuallybeneficialexchangeofknowledgeandresourcesinacontextofpartnershipandreciprocity.Thepurposeofcommunityengagementisthepartnershipofcollegeanduniversityknowledgeandresourceswiththoseofthepublicandprivatesectorstoenrichscholarship,research,andcreativeactivity;enhancecurriculum,teachingandlearning;prepareeducated,engagedcitizens;strengthendemocraticvaluesandcivicresponsibility;addresscriticalsocietalissues;andcontributetothepublicgood.

Further,Carnegiestates,

[Engagement]requiresgoingbeyondtheexpertmodelthatoftengetsinthewayofconstructiveuniversity-communitycollaboration…callsonfacultytomovebeyond‘outreach,’…asksscholarstogobeyond‘service,’withitsovertonesofnoblesseoblige.Whatitemphasizesisgenuinecollaboration:thatthelearningandteachingbemultidirectionalandtheexpertiseshared.Itrepresentsabasicreconceptualizationof…community-basedwork.(O’MearaandRice)

Kapi‘olani’s2015applicationfortheCarnegieCommunityEngagementClassificationidentifieditstop15mostimportantcommunitypartnershipsand11ofthesewereworkingwithusonsustainability-relatedprojectsandprograms.Newpartnershavebeendevelopedsince2015,andweneedtodevelopourassessmentsofthesepartnerships.IntheCarnegieClassificationprocessandongoingresearch,therearefivemajorcomponentsofauthenticpartnerships:

1. Clearlinesofcommunication2. Clearlydefinedrolesandresponsibilities3. Identifyingandelevatingboththeneedsandassetsofthecampusandthepartners4. Identifyingmutuallybeneficial,reciprocal,long-termgoals.5. Activeandcollaborativelearningforall

Thesefivecomponentswouldshapefutureassessmentandevaluationsofthecommunityandcivicengagementofoursustainabilityprogramming.

13

KAPIOLANI’SFUTUREDIRECTIONSINASSESSMENTThecollegeispilotinguseoftheInternationalSustainabilityLiteracyTest,orSulitest(seesulitest.org).ThemissionoftheSulitestisto:

[I]mproveSustainabilityLiteracyworldwidebyprovidingcitizensandorganizationswithinternationallyrecognizedandlocallyrelevanttoolstoengagelearningandtocollectmeaningfulindicatorsontheawarenessofsustainabilitychallengesandsolutions.Sharingtheseindicatorsmayallowresearchers,educatorsandotherrelevantstakeholderstotailorpedagogicalapproachesandlearningexperimentstosupportEducationforSustainableDevelopment.(“VisionandMission”)

WeareusingtheSulitestaspartofasuiteofprogram-levelassessmentsofSustainabilityFocusedcourses,comparingresultsfromstudentswhotakethosecoursesversusthosewhodonot.(Thereareapproximately20S-Focusedcoursesatthecollege,whichcanbetakenaspartofanembeddedAcademicSubjectCertificateinSustainability.)TheSALGisusedasapre-andpost-testforstudentreflection;theSALGcanalsobecustomizedbyfacultytogainstudentinputonspecificassignments,texts,orclassactivitiesandhowthoseaffectedtheirlearning.Inaddition,agroupoffivefacultyworkedtogethertodesignandpilotarubricassessmenttoolforfinalprojectsandpapersacrosssustainabilitycourses.ThisrubricisinformedbytheworkofArnimWieketal.Theyestablishedspecificcriteriaforumbrellacompetenciesinsustainability:systemsthinking,anticipatorycompetence,normativecompetence,strategiccompetence,andinterpersonalcompetence.WheretheWiekrubricestablishesNovice,Intermediate,andAdvancedlevelsofdemonstration,wecreatedalevel“below”novicetoreflectthelevelofenteringcommunitycollegestudents.Whiletheanalysisisnotyetcompleted,simplytheprocessofcollaboratingacrossdisciplinesinthedevelopmentoftherubrichashelpedustomoreeffectivelyengagewithstudentsatdifferentlevelsofability,experience,andengagementwithsustainability.Thesedialoguesaboutassessmenthelpustounderstandwhatmatters.

14

WorksCitedFranco,Robert.“TheCivicRoleofCommunityColleges:PreparingStudentsfortheWorkofDemocracy.”TheJournalofPublicAffairs,vol.6,2002,pp.119-138.www.compact.org/wp-content/uploads/resources/downloads/CCivic_role-final.pdf.Hill,YaoZhangetal."Service-Learning'sRoleinAchievingInstitutionalOutcomes:Engagement,Learning,andAchievement."Service-LearningattheAmericanCommunityCollege,editedbyAmyTraverandandZivahKatz,PalgraveMacmillan,2014,pp.169-182.Kuh,George.High-ImpactEducationalPractices:WhatTheyAre,WhoHasAccessToThem,AndWhyTheyMatter.AssociationofAmericanCollegesandUniversities,2008.O’Meara,KerryAnn,andR.EugeneRice,editors.FacultyPrioritiesReconsidered:RewardingMultipleFormsofScholarship.Jossey-Bass,2005.Renner,Tanya.“Convergingdata:DemonstratingServiceLearningEffectivenessOnOurCampus.”ThirdInternationalConferenceonService-LearningResearch,2003,SaltLakeCity,UT.Paper.Seymour,Eliseetal.“CreatingaBetterMousetrap:On-lineStudentAssessmentoftheirLearningGains.”UsingReal-WorldQuestionstoPromoteActiveLearning,NationalMeetingsoftheAmericanChemicalSocietySymposium,27March2000,SanFrancisco.Paper.www.salgsite.org/docs/SALGPaperPresentationAtACS.pdf.Wiek,Arminetal.OperationalisingCompetenciesinHigherEducationforSustainableDevelopment.Routledge,2015.“VisionandMission.”Sulitest.org,2016.www.sulitest.org/en/vision-mission.html.