on the inside: engaging first year physics students in ... · physics teaches a student how to do...
TRANSCRIPT
PreparedbyEvaluationsServiceUnit,TEDI
/Users/mjd/Documents/Admin/Grants/UQteaching/Reports/PhysicsReportInterim2012_print.docx
1
Ontheinside:EngagingFirstYearPhysicsStudentsinClassroomDiscussionofHigh-levelConcepts:Pre-reading,ReflectionandPersonalisedFeedbackSchoolofMathematicsandPhysics
PreparedSemester2,2012
ForProfessorMichaelDrinkwater
PreparedbyKarenSheppardandDeanneGannawayEvaluationUnit
TeachingandEducationalDevelopmentInstituteTheUniversityofQueensland
PreparedbyEvaluationsServiceUnit,TEDI
/Users/mjd/Documents/Admin/Grants/UQteaching/Reports/PhysicsReportInterim2012_print.docx
2
ContentsOverview...................................................................................................................................1
Background.......................................................................................................................1
EvaluationOutline.............................................................................................................2
Researchproblem................................................................................................................2
RoleofTEDI..........................................................................................................................2
Reporting.............................................................................................................................2
DataCollectionProcess.....................................................................................................2
Phase1:Surveyingfirst-yearexperienceofPHYS1001/1002..............................................2
Phase2:Talkingtothestudents..........................................................................................3
Phase3:Observingtheclasses............................................................................................3
Phase4:Analysisofsecondarydata....................................................................................3
Meta-Analysis....................................................................................................................4
KeyFindings..............................................................................................................................5
PHYS1001/1002StudentExperienceSurveys...................................................................5
PHYS1001PreandPostsurvey............................................................................................5
PHYS1002PreandPostsurvey............................................................................................5
PHYS1001vs.PHYS1002Pre-surveyandpost-survey..........................................................8
Prevs.Postsurveyresultsforbothcoursesforstudentswhocompletedbothsurveys..10
StudentExperience:Focusgroups..................................................................................11
1. Thewholeprocess....................................................................................................11
2. Individualelements..................................................................................................12
Observations...................................................................................................................14
Standard................................................................................Error!Bookmarknotdefined.
Differences............................................................................Error!Bookmarknotdefined.
StudentInformation..........................................................................................................15
Lectureimplementation-resources..................................................................................15
Lectureimplementation-Process.....................................................................................15
Learningopportunities......................................................................................................15
Studentengagement.........................................................................................................16
Overallobservations..........................................................................................................16
Conclusion...............................................................................................................................18
Areasforfurtherinvestigation........................................................................................19
REFERENCES............................................................................................................................20
PreparedbyEvaluationsServiceUnit,TEDI
/Users/mjd/Documents/Admin/Grants/UQteaching/Reports/PhysicsReportInterim2012_print.docx
3
Appendices..............................................................................................................................21
APPENDIX1:PRE-SURVEY.............................................................................................21
APPENDIX2:Post–Survey..............................................................................................25
APPENDIX3:FOCUSGROUPDISCUSSIONGUIDE............................................................29
APPENDIX4:STUDENTFOCUSGROUPCONSENTFORM................................................32
APPENDIX5:FURTHERTESTING.....................................................................................33
APPENDIX6:PHYS1001/1002ObservationGuide..........................................................35
Acknowledgements
Theinvestigationteamwouldliketogratefullyacknowledgeandthankallstudentswhoparticipated
inthisevaluationfortheirtime,honestyandenthusiasmwhileengagingintheprocess.
EvaluationUnit,TEDI,UniversityofQueensland|Overview 1
Overview
BackgroundActivelearningisatermusedtodescribearangeofmethodsofteachingscience.Thefundamental
principlebehindactivelearningrejectstheviewthatpassivelylisteningtoa50-minutelectureon
physicsteachesastudenthowtodophysics.Instead,ittakestheviewthatthestudentneedstobe
activelydoingphysicswiththeguidanceofaninstructor.1AusefuldefinitionfromPrincedefines
activelearningas:
Activelearningisgenerallydefinedasanyinstructionalmethodthatengagesinthelearningprocess.Inshort,activelearningrequiresstudentstodomeaningfullearningactivitiesandthinkaboutwhattheydoing.Whilethisdefinitioncouldincludetraditionalactivitiessuchashomework,inpracticeactivelearningreferstoactivitiesthatareintroducedintotheclassroom.Thecoreelementsofactivelearningarestudentactivityandengagementinthelearningprocess.Activelearningisoftencontrastedtothetraditionallecturewherestudentspassivelyreceiveinformationfromtheinstructor.
(Prince,2004,p.1)
Anessentialprerequisiteforactivelearningisthatstudentscometoclasswithsomefamiliaritywith
thebasicmaterial.Intheprojectteam’sview,regurgitatinginformationprovidedinatextbookisan
inefficientuseofcontacttimewithaninstructor.Abetteruseofclasstimeistohelpstudents
understandthematerialinthebookandbegintoapplytheconcepts.
Educationalresearchhasdemonstratedthatwhensuccessfullyimplementedstudentstaughtinthis
mannermakegreatergainsintheirphysicsabilitiesasmeasuredbeforeandafterinstruction(Laws
etal.,1999).TheprinciplesofactivelearninghavebeenappliedintheteachingofphysicsatThe
UniversityofQueenslandsince2008.Inboth2009and2010itwasfoundthatthisresultedin
improvementofstudents’abilitiesequaltotheworld’sbestpractice,upto200%greatergain
comparedtotraditionallytaughtcourses.
Morerecentresearchinthisareahasresultedinotherexamplesofimprovedlearning.Asmallstudy
carriedoutattheUniversityofBritishColumbiahighlightedtheadvantagesofaprocessof
‘deliberatepracticeofthinkingscientifically’(activeproblemsolvinginlectures),theactofmoving
the‘simpletransferofknowledgeoutsideofclass’(pre-readingtasksandonlinequiz)andfeedback
fromclassmatesandlecturer(groupandclassdiscussion)(Deslauriersetal2011,p.862).
Thefollowingoutlinesastrategyusedtoevaluatetheimpactoftheimplementationoftheactive
learningprocessusedasthebasisforstimulatingstudentlearningandengagementinfirstyear
physicscoursesin2011and2012.TheactivelearningsystemimplementedinPHYS1001in2011had
thefollowingaimsandanticipatedbenefits:
• toencouragestudentstodevelopself-directedlearningskills,
• toencouragestudentstoreflectontheirreadingmaterial,
• toencouragestudentstoreflectontheirlearninginclassandprovidefeedback,and
• toallowteachingstafftoidentifythemostdifficultconceptsforstudents.
AnevaluationofstudentlearninginPHYS1001conductedin2011byexternalreviewerssuggested
thatthesystemdidachievetheaimsanddidresultintheintendedbenefits.
Asaresultofthesefindings,theprojectteamintheSchoolofPhysicsdecidedtoimplementthe
systemacrossbothPHYS1001andPHYS1002in2012.
1FromPHYS1001–anotsobriefguidefor2011.
EvaluationUnit,TEDI,UniversityofQueensland|Overview 2
EvaluationOutlineWhiletheevaluationstrategyusedin2011soughttoevaluatetheuseofprereadingandonline
quizzesinPHYS1001,theevaluationconductedin2012willfocusontheimplementationofactive
learninginPHYS1002.Itwillalsocontinuetocollectdataabouttheefficacyoftheactivelearning
processinPHY1001andisdesignedtoincreaseacrosstheboardbase-linedatathatcanbeusedin
otherdevelopmentalactivitiesproposedforthePhysicsdepartment.
Theevaluationstrategyusedisafour-phasestrategythatmakesuseoffocusgroups,pre-andpost-
surveysofPHYS1001andPHYS1002students,lectureobservationsandanalysisofsecondarydata.
ResearchproblemTheactivelearningapproachmightbeviewedasasuccessifstudentsarebetterabletolearnand
retainkeyconceptsandifteachingstaffareabletoidentifythemostdifficultconceptsforstudents
andprovidemoreintensiveremediationintheseareas.Additionalbenefitswouldbeincreased
studentengagementandretention,increasedmotivationandmoreindependentlearners.
Thisevaluationsoughttotestwhethertheapproachdoesinfactachievethisengagementacross
twocoursesofstudy,PHYS1001(alreadyusingtheactivelearningprocess)andPHYS1002(theactive
learningprocessrecentlyimplemented)giventhatthecoursesaredeliveredbydifferentteachers,
involvedifferentcurriculaandattractdifferentcohortsofstudentstotheclasses.Indoingthisit
mustberecognisedthatasPrince(2004)commented:
…facultyadoptinginstructionalpracticeswiththeexpectationofseeingresultssimilartothose
reportedintheliteratureshouldbeawareofthepracticallimitationsofeducationalstudies.
Educationalstudiestelluswhatworked,onaverage,forthepopulationsexaminedandlearning
theoriessuggestwhythismightbeso.However,claimingthatfacultywhoadoptaspecificmethod
willseesimilarresultsintheirownclassroomsissimplynotpossible.Eveniffacultymasterthenew
instructionalmethod,theycannotcontrolallothervariablesthataffectlearning.
(Prince2004,p.3)
Thesamemightpossiblybeinferredevenwithinfacultiesandschools,andacrosscourseswithin
thosefacultiesandschools.
RoleofTEDITheEvaluationUnit(EU)withintheTeachingandEducationalDevelopmentInstitute(TEDI)was
commissionedbyProfessorMichaelDrinkwater(Physics)toconducttheevaluationasexternal
evaluators.KarenSheppard(TEDI),projectofficerfromEU,preparedtheanalysisandreport.
ReportingThisreportdescribesthedatacollectionandanalysisprocesses;outlineskeyfindingsfromthedata
collectedandconcludeswithasummaryemergingfromthefindingsofthelearningactivitieswhich
mostengagePHYS1001andPHYS1002students.Italsoidentifiesareasofstrengthandweaknessof
thecurrentprogramsandtherecentimplementationoftheactivelearningprocessinPHYS1002,as
reportedbyPHYS1001andPHYS1002students.
DataCollectionProcess
Phase1:Surveyingfirst-yearexperienceofPHYS1001/1002Pre-andpost-surveyswereconductedwithfirst-yearstudentsenrolledinPHYS1001andPHYS1002
toidentifytheirfamiliaritywiththeactivelearningprocessandtodeterminehowmuchthisprocess
assiststhemtoacquirecoreskillsandcontentandmaintainengagement.Thesurveyinstrument,
whilesimilartothepreviousyear,wasmodifiedandincludeditemsaboutotherpartsoftheprocess.
Sample:ThesampleforthisphaseofdatacollectionincludedallstudentsenrolledinPHYS1001and
PHYS1002inSemester1,2012resultinginacohortsizeof196inPHYS1001andacohortsizeof132
inPHYS1002.Toencourageparticipationratesstudentswereofferedtheequivalentoftwotutorial
EvaluationUnit,TEDI,UniversityofQueensland|Overview 3
participationmarksiftheycompletedboththepreandpostsurveys.Atotalof193studentsagreed
toparticipateinthesurvey,providinga53%responserate.Ofthe193,171studentsactually
completedthesurvey,providinga47%responserate.Forthepostsurvey,allstudentswereinvited
totakepart.ByusingtwocollectorsinthesurveyplatformSurveyMonkey,studentswhohadnot
completedthesurveyforthefirsttimewerecapturedinthedataset.Atotalof147agreedtotake
partinthesurvey,however,only132studentsoutof361finishedthesurvey,providinga37%
responseratethesecondtimearound.
Collectionmethod:Theon-linesurveybasedaroundtheInstructionalMaterialMotivationalSurvey
(IMMS)(Keller,1987)wasadministeredviaSurveyMonkey.Thesurveytestedstudents’perceptions
oftheirexperienceswiththeactivelearningprocess.TheIMMSbasedsurvey(SeeAppendices1/2)
wasdesignedtoassessthemotivationalcharacteristicsofinstructionalmaterialsusingthe
Attention,Relevance,ConfidenceandSatisfaction(ARCS)modelofmotivation(Keller,2006).The
themesfocusedonhowthestudentsfeltinresponsetothelessonstheyhadexperiencedinthefirst
threeweeksofSemesterOne.Thesamesurveywasadministeredinthefinalweeksofthesemester
providinga‘post’measureforcomparisonpurposes.Thesurveyswereavailableonlinefor
approximatelytwoweeksandthestudentsreceivedaninitialinvitationplustworeminderstotake
partinthesurvey.
Analysis:EUcollatedthedatafrombothsurveys.EUcollatedthedatafromthesurveysand
conductedadescriptiveanalysis
Phase2:TalkingtothestudentsSample:ThesampleforthisphaseofdatacollectionconsistedofstudentsenrolledinPHYS1001and
PHYS1002.47studentsagreedtoparticipateinthefocusgroups.Atotalof11students,sixmales
andfivefemales,tookpartintwofocusgroupsconductedoveraperiodofaweek.Sevenofthe
studentswereenrolledinPHYS1001andfourwereenrolledinPHYS1002.
Collectionmethods:ThefocusgroupswereconductedatlunchtimeduringWeek10ofthe
semester,byanindependentinvestigator,KarenSheppard.Studentswereinvitedtothemeeting
andtheirattendance,whilerequested,wasvoluntary.Pizzaanddrinkswereprovidedandeach
participantreceiveda$30giftcard.Thefocusgroupswereconductedusingasemi-structured
questioningtechniqueandranforapproximatelyonehour(seeAppendix3forthefocusgroup
guidelines).
Analysis:Athematicanalysisoftherecordingsofthefocusgroupswasconducted.
Phase3:ObservingtheclassesSample:ThisphaseofdatacollectionwascarriedoutinPHYS1001andPHYS1002lecturesoverWeek12and13.ThePHYS1001lecturewasobservedoncebytwoinvestigatorsandthePHYS1002
lecturewasobservedtwice,inthefirstinstancebytwoinvestigatorsandinthesecondbyasingle
investigator.Atotalof233studentswereobserved,96inthePHYS1001lectureand65inthefirst
PHYS1002lectureand72inthesecondPHYS1002lecture.Atotalof233studentswereobserved,
96inthefirstlectureand65inthesecondlectureand72inthethirdlecture.
Collectionmethods:Thelectureobservationswereconductedduringthe11amtimeslotingeneral
lecturetheatres.TheobservationswerecarriedoutbytwoinvestigatorsfromtheEU.Thedatawas
collectedusingamodifiedobservationguideandranforapproximately50minutes(seeAppendix4
fortheguide).Theobserversmetpriortotheobservationstoagreeonthemethodofrecordingand
usedthesameobservationformforallobservations.
Analysis:Analysisoftheobservationaldatawascarriedoutbybothobserverstoimprove
consistencyandreliability.
Phase4:AnalysisofsecondarydataWillbeprovidedwiththefinalreport.
EvaluationUnit,TEDI,UniversityofQueensland|Overview 4
Meta-AnalysisThesurveyresultsandthethemesdevelopedfromthefocusgroupsandobservationdatawere
analysedtoexplorethelinksbetweenhowthestudentsactuallyengageintheactivelearning
processandtheirsuccessindevelopingthefollowing:
• Self-directedlearningskills
• Reflectiononreadingmaterial
• Reflectiononlearninginclassandprovidingfeedback.
Datawasdrawntogethertoidentifyotherpossibleindicatorsforthesuccessoftheactivelearning
systemandthefeaturesoftheimplementationoftheprogramthatencouragedimprovementin
physicsabilitiesinstudentsenrolledinPHYS1001andnowinPHYS1002.
Despitetheincreasednumberofinformantsfromthepreviousyear,thefindingsreportedhereneed
tobeexaminedandinterpretedwithsomecaution.Thefindingscanonlybeviewedasasnapshotof
thecurrentcohort.However,patternsmaybeevidentwhenfindingspresentedherearetakeninto
accountwithotherreports.
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 5
KeyFindings
PHYS1001/1002StudentExperienceSurveysStudentswererequiredtorespondtoanumberofitemswhichinvestigatedthestudents’responses
tothelearningactivitiesconductedinWeekTwoandThreeofSemesterOne.Studentswereasked
torateagreementrelatedtoeachofthestatementsineachitem.Responseswerecategorisedona
5-pointLikertscale,including1(nottrue),2(slightlytrue),3(moderatelytrue),4(mostlytrue)to5
(verytrue),withhigherscoresindicatinghigherlevelsofbelief.Inotherinstances,theywereasked
toratethelearningactivitiesas1(nohelp),2(alittlehelp),3(moderatehelp),4(muchhelp)or5
(greathelp).Includedbelowaretablessummarisingthedatacollectedoverthepre-andpost-
surveysbrokendownforcourseandthedifferentlearningactivitiesthatarepartofthecourses.
Nocomparisonofdatafromthe2012surveysismadewithpreviousyear’sresponsesforeither
PHYS1001orPHYS1002;insteadcomparisonismadebetweenresponsesfromPHYS1001students
andresponsesfromPHYS1002studentsfrom2012.Theevaluationprojectteamhasmadethe
assumptionfrom2011findingsthattheactivelearningprocessappearstoprovideadditional
learningopportunitiesforstudentsenrolledinPHYS1001.Thesecondphaseoftheevaluationnow
seekstogatherdataabouttheimplementationofactivelearningintoasecondcourseinthe
department.
PHYS1001PreandPostsurveyThedatasuggestedthatPHYS1001studentshadsimilarratingsorslightlyhigherratingsforallitems
inthepost-surveywhencomparedtothepre-survey.
Therewasaslightdropinagreementinthefollowingitems
• Thevarietyofreadingpassages,exercises,illustrationsetc.Intheassignedreadinghelpedkeepmyattentiononthelesson,
• IcouldrelatethecontentoftheassignedreadingtothingsIhaveseen,doneorthoughtaboutinmyownlife,
• Theonlinequizmotivatesmetocompletetheassignedreadingbeforeclass;
• Ifindtheinclasslectureactivitiesenjoyable;
• Thecontentoftheassignedreadingandthelecturewillbeusefultome;
• Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthecourse.
Therewasahigherlevelofagreement(≥4.00)inoneiteminthepostsurvey;thisbeingThecontentoftheassignedreadinghelpedmeunderstandthelecture.StudentsenrolledinPHYS1001alsorespondedinapositivewayfortheitemOverallhowyouwouldratethecourse.
PHYS1002PreandPostsurveyFewerstudentsstudyingPHYS1002respondedtothesurvey;howeveranumberofitemsreceived
lowerratings(≤3.0slightlyormoderatelytrue)onboththepre-andpost-surveythantheratingsfor
PHYS1001.Thesedifferences;whilenotalwayssignificantaswewillseeinlateranalysis;indicate
quiteadifferentresponsetotheactivelearningprocessinstudentsenrolledinPHYS1002.The
reasonsbehindthislesspositiveresponsewillbearfurtherinvestigation.Theitemsthatwerearted
lessinPHYS1002thanPHYS1001included:
• Therewassomethinginterestingintheassignedreading;
• Thequalityofthewritingintheassignedreadinghelpedtoholdmyattention;
• IcouldrelatethecontentoftheassignedreadingtothingsIhaveseen,doneorthoughtaboutinmyownlife;
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 6
• Theassignedreadingmademewanttoknowmoreaboutthistopic.
• Ireallyenjoyedtheassignedreadingactivities.
• Aftertheintroductoryinformation,IfeltconfidentthatIknewwhatIwassupposedtolearnfromthisassignedmaterial
• Icanunderstandtheimportantbitsoftheassignedreadingwithoutreadingallofit.
• Thewordingoffeedbackaftertheonlinequiz,orothercommentsinthislecturemademefeelrewardedformyeffort.
• Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture
• Afterworkingonthelessonforawhile,IwasconfidentthatIwouldbeabletopassatestonit.
Inthepostsurveyforsomeitemstherewasnoincreaseinpositiveresponse;infact,students
indicatedsomeitemswerelesstrueinthelatterpartofthePHYS1002courseincluding(seeTable1)
viz:
• Thevarietyofreadingpassages,exercises,illustrationsetc.intheassignedreadinghelpedkeepmyattentiononthelesson;
• Ifindithardtoidentifytheimportantbitsofthereadingmaterial;
• Ireallyenjoyedtheassignedreadingactivities;
• Thewordingoffeedbackaftertheonlinequiz,orothercommentsinthislecturemademefeelrewardedformyeffort;
• Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture;Ifindtheinclasslectureactivitiesenjoyable;
• Thecontentoftheassignedreadingandthelecturewillbeusefultome;andTheactivitiesinthelectureweretoodifficult.
Table1:Overallmeansandstandarddeviationsforallresponsestopre-andpost-surveysforPHYS1001andPHYS1002forallitems.
PHYS1001 PHYS1002
PreReadingandLessonfeedback Pre
Mean
(SD)
N=96
Post
Mean
(SD)
N=76
Pre
Mean
(SD)
N=75
Post
Mean
(SD)
N=53
Overallhowwouldyouratethiscourse? 3.93
(.73)
3.94
(.78)
3.44
(.86)
3.33
(.93)
Therewassomethinginterestingintheassignedreadingthatcaughtmy
attention.
3.10
(.92)
3.25
(.97)
3.00
(1.00)
3.13
(1.02)
Thequalityofthewritinghelpedtoholdmyattention 3.01
(1.04)
3.04
(1.07)
2.92
(1.06)
3.04
(1.19)
Thevarietyofreadingpassages,exercises,illustrationsetc.Intheassigned
readinghelpedkeepmyattentiononthelesson.
3.31
(1.07)
3.28
(1.05)
3.27
(.96)
3.25
(1.02)
IcouldrelatethecontentoftheassignedreadingtothingsIhaveseen,doneor
thoughtaboutinmyownlife.
3.67
(1.08)
3.64
(1.03)
2.89
(0.94)
3.23
(1.07)
Theassignedreadingmademewanttoknowmoreaboutthistopic. 3.01
(1.02)
3.21
(1.07)
2.75
(1.22)
3.11
(1.19)
Ifindithardtoidentifytheimportantbitsofthereadingmaterial. 2.54
(1.26)
2.66
(1.33)
3.25
(1.23)
3.19
(1.16)
Ireallyenjoyedtheassignedreadingactivities. 2.33
(0.98)
2.50
(1.01)
2.43
(1.09)
2.42
(1.14)
Thecontentoftheassignedreadinghelpedmeunderstandthelecture 3.90
(.90)
4.10
(.87)
3.55
(.93)
3.53
(1.07)
Theassignedreadingissoabstractthatitwashardtokeepmyattentiononit. 1.96
(1.02)
2.12
(1.08)
2.64
(1.18)
2.98
(1.23)
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 7
Icouldnotreallyunderstandquiteabitoftheassignedreadingmaterialforthis
lecture.
1.91
(.99)
2.28
(1.16)
2.59
(1.09)
2.66
(1.22)
Aftertheintroductoryinformation,IfeltconfidentthatIknewwhatIwas
supposedtolearnfromthisassignedmaterial.
3.16
(.95)
3.23
(1.11)
2.95
(.90)
2.93
(.96)
Theonlinequizmotivatesmetocompletetheassignedreadingbeforeclass. 3.93
(1.30)
3.63
(1.23)
3.32
(1.29)
3.34
(1.34)
Icanunderstandtheimportantbitsoftheassignedreadingwithoutreadingallof
it.
3.04
(1.20)
3.08
(1.07)
2.73
(1.07)
2.98
(1.08)
Igainedunderstandingbydoingtheassignedreading. 3.68
(1.07)
3.82
(.87)
3.37
(1.02)
3.44
(1.04)
IwouldnotfeelcomfortabletellingmyentireclasswhatIfoundmostdifficult
withthereading(e.g.onadiscussionboard).
2.46
(1.37)
2.46
(1.38)
2.80
(1.19)
2.62
(1.26)
Thewordingoffeedbackaftertheonlinequiz,orothercommentsinthislecture
mademefeelrewardedformyeffort.
2.92
(1.10)
3.00
(1.06)
2.85
(.99)
2.71
(1.23)
Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture. 3.44
(1.15)
3.45
(1.17)
3.12
(1.18
2.96
(1.36)
Ifindtheinclasslectureactivitiesenjoyable. 3.99
(0.98)
3.80
(.94)
3.25
(1.14)
3.06
(1.21)
ItiscleartomehowthecontentofthelecturesisrelatedtothingsIalready
know.
3.73
(.94)
3.73
(.97)
3.15
(.89)
3.21
(.99)
Therewerestories,picturesorexamplesinthelecturesthatshowedmehow
thismaterialcouldbeimportanttopeople.
3.49
(.93)
3.72
(.93)
3.32
(.92)
3.29
(1.09)
Thecontentoftheassignedreadingandthelecturewillbeusefultome. 3.70
(.80)
3.68
(.99)
3.35
(.91)
3.12
(1.08)
Theactivitiesinthelectureweretoodifficult. 1.69
(.87)
1.92
(.96)
2.28
(1.04)
2.23
(1.16)
Afterworkingonthelessonforawhile,IwasconfidentthatIwouldbeableto
passatestonit.
3.47
(1.08)
3.47
(1.04)
2.90
(1.08)
3.11
(1.08)
Theamountofrepetitioninthelessoncausedmetogetboredsometimes. 1.81
(.97)
2.07
(1.13)
1.99
(1.07)
2.40
(1.05)
Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthe
course.
3.76
(1.02)
3.72
(.97)
3.16
(1.03)
3.23
(1.13)
PHYS1001 PHYS1002
LearningActivityfeedback Pre
Mean
(SD)
N=96
Post
Mean
(SD)
N=76
Pre
Mean
(SD)
N=75
Post
Mean
(SD)
N=53
Completingonlineproblems(MasteringPhysics) 3.95
(1.09)
3.99
(1.25)
3.27
(1.15)
3.17
(1.13)
Participatingintutorials/conceptlabs 2.97
(1.45)
3.23
(1.26)
3.50
(1.17)
3.34
(1.28)
Completingpracticallabs 2.21
(1.72)
3.26
(1.24)
3.32
(1.23)
3.30
(1.22)
Theuseof“clickers”duringthelecture 3.73
(1.33)
3.62
(1.36)
3.66
(1.07)
3.33
(1.21)
Discussingquestionswiththepersonnexttomeduringthelecture 3.74
(1.27)
3.65
(1.40)
3.27
(1.24)
3.13
(1.33)
Discussingquestionsasawholeclassduringthelecture. 3.79
(1.12)
3.72
(1.29)
3.41
(1.20)
3.25
(1.28)
Doingtheassignedreadingtasksandonlinequiz. 3.60
(1.09)
3.56
(1.01)
3.36
(1.24)
3.46
(1.21)
AttendingPASSclasses. 2.46
(1.74)
2.08
(1.63)
2.23
(1.53)
2.44
(1.67)
PHYS1001 PHYS1002
GeneralPhysicsfeedback Pre
Mean
(SD)
N=96
Post
Mean
(SD)
N=76
Pre
Mean
(SD)
N=75
Post
Mean
(SD)
N=53
Iamenthusiasticaboutphysics 4.04
(.96)
3.88
(1.01)
3.35
(1.09)
3.33
(1.18)
Iaminterestedintakingextraphysicsclasses 3.82
(1.18)
3.64
(1.27)
2.84
(1.08)
2.70
(1.28)
IamconfidentIunderstandthecoursematerial 3.69
(.92)
3.47
(.95)
3.14
(1.02)
3.11
(.92)
Ineedtomemoriseformulaetogetagoodmarkinthisclass 2.56
(1.05)
2.77
(1.17)
3.42
(1.12)
3.26
(1.14)
Ineedtounderstandconceptsinphysicstogetagoodmarkinthiscourse 4.66
(.71)
4.56
(.64)
4.35
(.82)
4.54
(.66)
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 8
Ihavecompletedalltheassignedreadinginthiscourse 3.65
(1.38)
3.29
(1.34)
3.16
(1.32)
3.38
(1.19)
1=Nottrue,2=slightlytrue,3=moderatelytrue,4=Mostlytrue,5=Verytrue.Note:Figuresinparenthesesrepresentthestandarddeviations.
PHYS1001vs.PHYS1002Pre-surveyandpost-surveyWhencomparingPHYS1001,studentresponseswithPHYS1002students’responsestothepre-and
post-survey,itisevidentthatoverallPHYS1001studentsdemonstratedhigherratingsthan
PHYS1002acrossalloftheitems.Whenthesurveyitemswerebrokendownforlearningactivity
andgeneralphysicsfeedbackthegeneraltrendremainedthesame.Again,itshouldberemembered
thatthepostsurveysamplesizeissmallerthanthepre;however,aseriesofindependentsamplet-
testsindicatedsomestatisticallysignificantdifferencesbetweenthetwocourses.Itemstestedthat
werestatisticallysignificantwhichareofparticularinteresttothestudyincludedthefollowinginthe
pre-survey:
• Theonlinequizmotivatesmetocompletetheassignedreadingbeforeclass,
• Thecontentoftheassignedreadinghelpedmeunderstandthelecture,
• Theassignedreadingissoabstractthatitwashardtokeepmyattentiononit,
• Ifindthein-classlectureactivitiesenjoyable,
• ItiscleartomehowthecontentofthelecturesisrelatedtothingsIalreadyknow,
• Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthecourse.
ThesedifferencesareillustratedinTable2below
Table2:MeansandstandarddeviationsofPHYS1001studentspre-survey(n=96)andPHYS1002studentspre-survey(n=75)wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.Pissignificantat≤0.05
PHYS1001 PHYS1002
PreReadingandLessonfeedback PreMean
(SD)
N=96
PreMean
(SD)
N=75
Significance
Overallhowwouldyouratethiscourse? 3.93
(.73)
3.44
(.86)
<.001
Therewassomethinginterestingintheassignedreadingthatcaughtmy
attention.
3.10
(.92)
3.00
(1.00)
Thequalityofthewritinghelpedtoholdmyattention 3.01
(1.04)
2.92
(1.06)
Thevarietyofreadingpassages,exercises,illustrationsetc.Intheassigned
readinghelpedkeepmyattentiononthelesson.
3.31
(1.07)
3.27
(.96)
IcouldrelatethecontentoftheassignedreadingtothingsIhaveseen,doneor
thoughtaboutinmyownlife.
3.67
(1.08)
2.89
(0.94)
<.001
Theassignedreadingmademewanttoknowmoreaboutthistopic. 3.01
(1.02)
2.75
(1.22)
Ifindithardtoidentifytheimportantbitsofthereadingmaterial. 2.54
(1.26)
3.25
(1.23)
<.001
Ireallyenjoyedtheassignedreadingactivities. 2.33
(0.98)
2.43
(1.09)
Thecontentoftheassignedreadinghelpedmeunderstandthelecture 3.90
(.90)
3.55
(.93)
.014
Theassignedreadingissoabstractthatitwashardtokeepmyattentiononit. 1.96
(1.02)
2.64
(1.18)
<.001
Icouldnotreallyunderstandquiteabitoftheassignedreadingmaterialforthis
lecture.
1.91
(.99)
2.59
(1.09)
<.001
Aftertheintroductoryinformation,IfeltconfidentthatIknewwhatIwas
supposedtolearnfromthisassignedmaterial.
3.16
(.95)
2.95
(.90)
Theonlinequizmotivatesmetocompletetheassignedreadingbeforeclass. 3.93
(1.30)
3.32
(1.29)
.004
Icanunderstandtheimportantbitsoftheassignedreadingwithoutreadingallof
it.
3.04
(1.20)
2.73
(1.07)
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 9
Igainedunderstandingbydoingtheassignedreading. 3.68
(1.07)
3.37
(1.02)
IwouldnotfeelcomfortabletellingmyentireclasswhatIfoundmostdifficult
withthereading(e.g.onadiscussionboard).
2.46
(1.37)
2.80
(1.19)
Thewordingoffeedbackaftertheonlinequiz,orothercommentsinthislecture
mademefeelrewardedformyeffort.
2.92
(1.10)
2.85
(.99)
Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture. 3.44
(1.15)
3.12
(1.18
Ifindtheinclasslectureactivitiesenjoyable. 3.99
(0.98)
3.25
(1.14)
<.001
ItiscleartomehowthecontentofthelecturesisrelatedtothingsIalready
know.
3.73
(.94)
3.15
(.89)
<.001
Therewerestories,picturesorexamplesinthelecturesthatshowedmehow
thismaterialcouldbeimportanttopeople.
3.49
(.93)
3.32
(.92)
Thecontentoftheassignedreadingandthelecturewillbeusefultome. 3.70
(.80)
3.35
(.91)
.010
Theactivitiesinthelectureweretoodifficult. 1.69
(.87)
2.28
(1.04)
<.001
Afterworkingonthelessonforawhile,IwasconfidentthatIwouldbeableto
passatestonit.
3.47
(1.08)
2.90
(1.08)
.001
Theamountofrepetitioninthelessoncausedmetogetboredsometimes. 1.81
(.97)
1.99
(1.07)
Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthe
course.
3.76
(1.02)
3.16
(1.03)
<.001
Pissignificantat≤0.05
Similarly,itemsofparticularinteresttothestudythattestedasstatisticallysignificantincludedthe
followinginthecomparisonbetweenPHYS1001andPHYS1002post-surveys:
• Thecontentoftheassignedreadinghelpedmeunderstandthelecture,
• Igainedunderstandingbydoingtheassignedreading,
• Ifindtheinclasslectureactivitiesenjoyable,
• Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture,
• Thecontentoftheassignedreadingandthelecturewillbeusefultome,
• Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthecourse.
ThesedifferencesareillustratedinTable3below:
Table3:MeansandstandarddeviationsofPHYS1001studentspost-survey(n=76)andPHYS1002studentspost-survey(n=53)wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.
PHYS1001 PHYS1002
PreReadingandLessonfeedback PostMean
(SD)
N=76
PostMean
(SD)
N=53
Significance
Overallhowwouldyouratethiscourse? 3.94
(.78)
3.33
(.93)
<.001
Therewassomethinginterestingintheassignedreadingthatcaughtmyattention. 3.25
(.97)
3.13
(1.02)
Thequalityofthewritinghelpedtoholdmyattention 3.04
(1.07)
3.04
(1.19)
Thevarietyofreadingpassages,exercises,illustrationsetc.Intheassignedreadinghelped
keepmyattentiononthelesson.
3.28
(1.05)
3.25
(1.02)
IcouldrelatethecontentoftheassignedreadingtothingsIhaveseen,doneorthought
aboutinmyownlife.
3.64
(1.03)
3.23
(1.07)
.027
Theassignedreadingmademewanttoknowmoreaboutthistopic. 3.21
(1.07)
3.11
(1.19)
Ifindithardtoidentifytheimportantbitsofthereadingmaterial. 2.66
(1.33)
3.19
(1.16)
.018
Ireallyenjoyedtheassignedreadingactivities. 2.50
(1.01)
2.42
(1.14)
Thecontentoftheassignedreadinghelpedmeunderstandthelecture 4.10
(.87)
3.53
(1.07)
.002
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 10
Theassignedreadingissoabstractthatitwashardtokeepmyattentiononit. 2.12
(1.08)
2.98
(1.23)
<.001
Icouldnotreallyunderstandquiteabitoftheassignedreadingmaterialforthislecture. 2.28
(1.16)
2.66
(1.22)
Aftertheintroductoryinformation,IfeltconfidentthatIknewwhatIwassupposedtolearn
fromthisassignedmaterial.
3.23
(1.11)
2.93
(.96)
Theonlinequizmotivatesmetocompletetheassignedreadingbeforeclass. 3.63
(1.23)
3.34
(1.34)
Icanunderstandtheimportantbitsoftheassignedreadingwithoutreadingallofit. 3.08
(1.07)
2.98
(1.08)
Igainedunderstandingbydoingtheassignedreading. 3.82
(.87)
3.44
(1.04)
.030
IwouldnotfeelcomfortabletellingmyentireclasswhatIfoundmostdifficultwiththe
reading(e.g.onadiscussionboard).
2.46
(1.38)
2.62
(1.26)
Thewordingoffeedbackaftertheonlinequiz,orothercommentsinthislecturemademe
feelrewardedformyeffort.
3.00
(1.06)
2.71
(1.23)
Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture. 3.45
(1.17)
2.96
(1.36)
.034
Ifindtheinclasslectureactivitiesenjoyable. 3.80
(.94)
3.06
(1.21)
<.001
ItiscleartomehowthecontentofthelecturesisrelatedtothingsIalreadyknow. 3.73
(.97)
3.21
(.99)
.004
Therewerestories,picturesorexamplesinthelecturesthatshowedmehowthismaterial
couldbeimportanttopeople.
3.72
(.93)
3.29
(1.09)
.019
Thecontentoftheassignedreadingandthelecturewillbeusefultome. 3.68
(.99)
3.12
(1.08)
.003
Theactivitiesinthelectureweretoodifficult. 1.92
(.96)
2.23
(1.16)
Afterworkingonthelessonforawhile,IwasconfidentthatIwouldbeabletopassatest
onit.
3.47
(1.04)
3.11
(1.08)
Theamountofrepetitioninthelessoncausedmetogetboredsometimes. 2.07
(1.13)
2.40
(1.05)
Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthecourse. 3.72
(.97)
3.23
(1.13)
.010
* Pissignificantat≤0.05
Prevs.PostsurveyresultsforbothcoursesforstudentswhocompletedbothsurveysResponsesfromstudentswhocompletedboththepreandpostsurveysforeitherofthecourses
werecomparedbymean.Whencomparingthesamestudents’responsesthesecondtimearound
therewasfewsignificantdifferences.PairedT-testswereappliedtoaselectionofitemsfromthe
survey.Itemsthatweresignificantincludedapositiveincreasefortheitems
• Thecontentoftheassignedreadinghelpedmeunderstandthelecture• Completingpracticallabs
AmorenegativeresponsewasfoundtotheitemIfindtheinclasslectureactivitiesenjoyableforPHYS1001inthepostsurvey.Therewerenosignificantdifferenceswhencomparingtheitems
selectedfromthepreandpostPHYS1002surveys.Thesedifferencesareillustratedintable4below.
Table4:StudentswhocompletedbothpreandpostsurveysforPHYS1001andPHYS1002testedforitemsofsignificance(* Pissignificantat≤0.05).
PrePHYS1001
n=62
PostPHYS1001
n=62
Sig. 1002n=43
1002n=43
Sig.
PreReadingandLessonfeedback
Overallhowwouldyouratethiscourse? 3.97
(.70)
3.93
(.77)
.742 3.28
(.83)
3.35
(.95)
.570
Therewassomethinginterestingintheassignedreadingthatcaughtmy
attention.
3.06
(.92)
3.29
(.91)
.066 2.84
(.99)
3.09
(99)
.086
Thecontentoftheassignedreadinghelpedmeunderstandthelecture 3.83
(.87)
4.10
(.82)
.020
*3.42
(.88)
3.53
(1.08)
.499
Theonlinequizmotivatesmetocompletetheassignedreadingbefore
class.
4.00
(1.25)
3.69
(1.24)
.055 3.49
(1.18)
3.28
(1.32)
.391
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 11
Igainedunderstandingbydoingtheassignedreading. 3.75
(1.25)
3.87
(.85)
.382 3.24
(.93)
3.36
.96)
.482
Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture. 3.42
(1.15)
3.40
(1.15)
.899 2.97
(1.30)
2.90
(1.34)
.674
Ifindtheinclasslectureactivitiesenjoyable. 4.05
(.91)
3.75
(.97)
.008
*3.05
(1.13)
3.05
(1.17)
1.0
Thecontentoftheassignedreadingandthelecturewillbeusefultome. 3.74
(.79)
3.66
(1.01)
.505 3.22
(.94)
3.05
(1.05)
.227
LearningActivityfeedback
Completingonlineproblems(MasteringPhysics) 3.97
(1.10)
4.14
(1.11)
.213 3.12
(1.06)
3.14
(1.03)
.888
Completingpracticallabs 2.44
(1.73)
3.16
(1.23)
.002
*3.17
(1.25)
3.29
(1.15)
.548
Theuseof“clickers”duringthelecture 3.82
(1.26)
3.63
(1.39)
.242 3.57
(1.08)
3.52
(1.17)
.793
Discussingquestionswiththepersonnexttomeduringthelecture 3.70
(1.20)
3.70
(1.35)
1.00
0
3.14
(1.18)
3.14
(1.28)
1.000
Discussingquestionsasawholeclassduringthelecture. 3.84
(1.28)
3.76
(1.28)
.653 3.34
(1.04)
3.30
(1.27)
.824
Doingtheassignedreadingtasksandonlinequiz. 3.60
(1.12)
3.56
(1.05)
.829 3.36
(1.25)
3.55
(.99)
.345
Thedatacollectedfromthepreviousyearalsoindicateddifferencesinstudentresponsesbetween
PHYS1001andPHYS1002andtheassumptionmadewasthatstudentsinPHYS1001wereresponding
positivelytotheactivelearningprocess.Coursecoordinatorsdecidedtotestthisassumptionby
implementingtheactivelearningprocessinPHYS1002in2012.However,surveyresponsesstill
indicateagapinratingsbetweenPHYS1001andPHYS1002formostitems.
Insummary,itappearsfromthesurveydata;thatdespitetheimplementationoftheactivelearning
processinPHYS1002;thatsomesignificantdifferencesstillexistinthestudents’perceptionsand
responsestothecoursewhencomparedtothoseexperiencedbythestudentsinPHYS1001.
StudentExperience:FocusgroupsThefollowingfindingsemergedasaresultofathematicanalysisofthequalitativeresponsesand
contentdiscussedinthefocusgroups.
1. ThewholeprocessStudentsinthefocusgroupsappearabletorecognisethebenefitsoftheprocess.Somestudentsare
notnecessarilyabletoarticulatewhytheyfeltitimprovedtheirlearning,whileothersweremore
reflectiveintheirunderstanding.Thereflectivestudentsweremoreabletoconnecttheprocesswith
improvedlearningoutcomes,asapparentinthefollowingextractsfromthediscussions.
IwrotelecturesandactivitiesfirstbecauseitislikeactivelearningIguessaskingyougetspeoplethinking,lecturersaskingquestionsandinsteadofjustsittingthereyouareinterested,inotherlecturesyoujustsitthereandthereisnointeraction…Yeh…andsomeonethrowsinformationatyou.Andthenthereisnotimetoactuallythinkaboutit.Andgrouppartneractivitiestheycanbeexplainingitandiftheyknowitwillhelptoexplainitinadifferentway.Andthenthepersonlearningwillhaveanotherperson’sviewonitandthatalwayshelps.SameforthelecturesandIdidprereadingbecausewithouttheprereadingyou’renormallyabitlostandthenthathelpsinthelectures.
Ithinkithelpsuslearnfasterbecausewedon’tspendthelecturetimelookingthroughthebookwehavealreadydonethat.Andwerefineourunderstandinginthelectureandinsteadofjusttryingtounderstandittostartwith,itsavestimeandwegetmoreworkdoneinashorteramountoftime.Thereadingquizzeshelpustofocusthelearningfromtheprereadingbecausesometimethetextisabitwordy.
Iagreewithyouthatitiscalledactivelearningbecauseweareallparticipatinginthecourse,wearenotpassivelylearning,sittingthereinlecturesjustdaydreamingweareactuallysittingthereinthelectureanddoingclickingquestionsandtalkingtopeopleandIthinkthatisgoodbecauseyoucanironoutanymisconceptions.
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 12
Ithinkitisbetterthanbeingtraditionallytaughtbecauseinthesetypesofcoursesweareactivelylearningwearenotpassivelytaughtwedotheprereadingwedotheproblemsheetswegothelecturesweusetheclickersandwearenotsittingbackandobserving,weareinsidenotoutside.
2. IndividualelementsInthepreviousstudyconductedin2011,emphasiswasplacedoninvestigatingtheprocessofpre-
readingandonlinequizzesasamethodofengagingstudentsinthelearningprocess.Further
investigationwasconsideredwarrantedinordertodeterminetheeffectofotheraspectsofthe
processonstudents.Inthefocusgroupsconductedin2012,allelementsofthelearningprocess
wereinvestigated,asillustratedinthefocusgroupguide(Appendix4).Students’commentsforeach
oftheelementsareexploredbelow.
Pre-readingandonlinequizTheparticipantsofthefocusgroupscontinuedtorecognisethevalueoftheprereadingtasksand
on-linequiz.Havingapreviousencounterwiththeconceptsbeforedealingwiththeminthelecture
orexaminationallowedthestudentstoscaffoldtheirlearning.Theprereadingandquizassiststhe
studentstodevelopconceptualframeworkswhichinturnmakelearningandproblemsolvinginthe
lectureeasierandmoreaccessible.Thefeedbackfromtheonlinequizalsoprovidesalevelof
responsethatwasmoreimmediateratherthanwaitingfortheresultsfromanassignmentor
laboratoryreport.
Theexplanationandthereadingithelpedmealot…ImeanIstartedoffwithskimming….becausetheworkloadwassohugebeforethesemesterbreakIjustskimmeditandthenjustreadingthesummariesandthendoingthequestionsandIfailedthemidsemesterandthenIstartedlikereadingitmoreindepthandthenwritingnotesasIwentalong.IfoundthatIactuallyneededtodothatbecauseitdoesn’tstickotherwisehoweveritisalwaysnicetohavesomethingtogobacktoandyoucanputitinyourownwords.
Soinothersubjectswherethereisnopre-readingyoufindyourselftryingtoreadhalfatextbookforexamprepbecauseyouhaven’tlookedatitbefore.
Constantinteractionwiththetextbookmeansyouarenotconfrontedwithitwhenyougotodotheexam.
Havingthetextbookintheexamisonlyusefulifyouhavealreadyengagedwithitduringthesemester.Youcan’tfakeit.
Sotherearetwotypesoffeedbackonewhereyouaredoingquizzesandyoucanmarkstraightandtheotherwhereyourelyonthelecturertomarkandthenyougetfeedbackinthelectures.
Sothefeedbackallowsthelecturerstocorrectanymisconceptionsyoumighthaveacrossthecohort.Itkindofhelps…
Theygooverthereadinganditusuallytakesaboutfiveminutestogothroughandthentheygoontothelecture.Theyusuallyidentifythemainconcernsinthelecture.Michaelusuallyaddressestheseandthenhewillputupthebestanswersorananswerthatshouldbeansweredinanotherway.
Theparticipantsagainidentifiedtheissueoftheamountoftimerequiredtocompletethepre-
readingandonlinequizactivitiesinbothfocusgroups.Whiletheyrecognisedthebenefitsofthe
practice,theyalsoindicatedthattheworkrequiredtocompletetheactivitieswasmorethanin
othercourses.
Sometimesithardtogetorganisedaroundtheworkload.TheprereadingisusefulwhenI’mlike…whenyoudoitproperlyhoweveritisnotsoeasywhenthereareotherformsofassessmentaswell.Sointhebeginningitwasfinewhenwedidnothavemuchworkfromtheothercoursesaswell.Itwaseasytofindthetimetositdownandgothroughthereadingandtakenotesandtoworkinthephysicsroomandtaketimetodoit…notsomuchnow.
ItisprettyintenseinPhysics.
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 13
Prereadingexercisesaregreatbutthereisfartoomuch.Itisreallygreathowtheygiveustheclickerquestionsandalltheexampleshowevertheydon’tgiveusmuchdetailandtheydon’tactuallyteachthestuff(inthelecture).AndIgetthattheywantustogetthatfromthereadinghoweverIfinditsodifficulttogetthroughthereading.
ClickerquestionsandlectureactivitiesStudentscontinuetorecognisethevalueoftheinclusionofclickerquestionsandinteractive
activitiesinthelecture.Studentsalsofeltmoreconfidentinlectures,havingalreadyengagedwith
thelearningmaterial,whenrespondingtoquestionsbothwithinasmallgrouporwhenaskeddirect
questionsbythelecturer.Theuseofinclassactivitiesandclickerquestionspreparedthestudents
for“expertexplanations”andthedevelopmentofproblemsolvingskills.
TheselectureactivitieswithclickeractivitiesaregreatwhenIturnupbecauseitforcesyoutothinkaboutitandsometimesIcantuneoutduringotherlecturesbutinPhysicsIstayontrack.
Lectureactivitieswiththeclickerstheyarereallygoodbecausethereismoreexamplesandthenyougetachancetotryitthenyougetachancetoseeifyourattemptwascorrectorincorrectandthenthereisanexplanationbutasforlecturesthereisnotmuchcontentcoveredbecauseliketheybarelyteachyouitsoitisallfromthereadingactually.Participationinlectureslikewhenthereisanevenmixonthevotesandyougetpeopletryingtoexplainit.GrouppartneractivitiesareOKbutwhatifyougetstuckwithsomebodywhoisreallydumbordoesn’tevencarethenitisjustfrustrating.
Ithinktheclickerquestionsareprettyexcellentandtheycanhelpyouworkoutwhatisgoingon.Likeifyoumisunderstandsomethingthenyoutalktopeoplearoundyouandtheyhaveadifferentunderstandingofwhatishappeningandthenbecausepeopledothingsdifferentlyandtheymighthaveaclearerwaytodosomething.Thenyoutalktothemanditmakesitbetter.I’mnotsureifyouwouldcallitteamworkhoweveritisbetterthandoingitalone.
Soifyouhaveaquestionandthenyoufindoutsomeoneelsehasthesamequestionyoufeellikeyouarenotalone.
Ithinktheclickerquestionsareprettyexcellentandtheycanhelpyouworkoutwhatisgoingon.Likeifyoumisunderstandsomethingthenyoutalktopeoplearoundyouandtheyhaveadifferentunderstandingofwhatishappeningandthenbecausepeopledothingsdifferentlyandtheymighthaveaclearerwaytodosomething.Thenyoutalktothemanditmakesitbetter.I’mnotsureifyouwouldcallitteamworkhoweveritisbetterthandoingitalone.
Itshouldberecognisedthatnotallstudentsrespondedinapositivewaytotheseteachingmethods.
Somestudentspreferrednottoengagewithanyoneelseintheclassandfelt“pressured”whenthe
lecturerrequestedthattheypairupwithsomeoneelse.Otherssuggestedthatchoosingtheright
partnerwhowasalsoengagedintheprocesswasimportanttoensureappropriatelearning
outcomes.Therewerealsoindicationsfromsomestudentsthatclickerswerenotbeingfullyutilised
bysomeoftheirpeersandthatbecauseparticipationwastheonlyrequirementsomestudentswere
justclickingarandomresponse.Onestudentsuggestedthatmarksshouldbegivenforcorrect
answersnotjustanyresponsetotheproblem.
SoinlecturestherearepeoplewhoaretalkingandontheInternetanddoingotherthingsandthentheyrealisethereisaclickerquestionandtheyclickinsomethingandthentheygobacktodoingsomethingotherthanPhysics.Ithinkthereshouldbeanextrabonusforgettingthequestionright.
IthinkgroupandpartneractivitiesarethebestthingsIfindforlearningandreinforcingknowledgebutnotthatyouareforcedintobeingingroupsbecauseyoumightgetapoorgroupanditisnotbeneficialanditcanturnyouawayfromthecourse.Butifyoudogetagroupthatworksreallyitisgreat.SoIhaveabunchoffriendsandwegoandhavelunchafterthelectureandwebasicallytalkaboutthephysicsproblemswejusthadoraboutthepraccomingupanditjustreinforceseverythinganditjuststicksbetter…Peoplesometimesanswerbeforethequestionisevenupmeaningtheyarenotpayinganyattention.
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 14
Anotheridentifiedissuewasiftheactivelearningprocesswasnotadheredtoinitsentiretythen
thereweretimeswhenshortcutsweremade.Forexamplewhenstudentsindicatedthattheydid
notknowtheanswerviatheclickersthenitbecameeasierforthelecturertojusttellthemrather
thanletthemproblemsolveindependentlyorwithinagroup.Thefocusgroupparticipants
suggestedthistobeanissuewithclasssizeandpossiblytheabilitytomanagelargegroups.Further
probingwasnotabletoelicitamoredetailedanswer.
Ithinkthatsomehowwhathappensinthelecturetheatre…whathappensisthatthelecturer…ifpeopledon’tgetittheysayifwellyoudon’tknowtheanswerthenIwilljusttellyouwhatitis.Anditisprobablyjustharderwiththatmanypeopleinthelecture.
MasteringPhysicsiswhereyoucaneitherdoitornot.Thepre-readingisgreatbutuntilIgetintothelectureIdon’tstartunderstandingituntiltheyexplainitandthenIunderstandwhatwearefocussingon,thepartnerandgroupparticipationactivitiesaregood.
I’menjoyingthemall,howeverPhysicsIthinkI’mlearningmuchfasterbecauseyouhavetodotheprereadingbecauseitcountsasmarksandyouhavetoyouareencouragedtodothatandthewholegoingtolecturesandyoumakesureyougettherebecauseyoumissoutonmarkswhereastheothersyoumissonehereandthere.TheprereadingitdoesforceyoutodoitisandtheteachingstyleisdefinitivelyalotbetterthanthebiologyandchemistrybecauseeveryonehasdonetheprereadingandyoudohavethewholeinteractivityofthelecturebutthenagaintheclassismuchsmallerthanBioandChem.HowmuchsmallerIthinkmaybe1500kidsinChemistryandI’mnotsureabouttheothers.
Feedbackforteacherandstudent-theclosingoftheloopOneofthestrengthsofthelearningprocessistheconstantfeedbackintothelearningloopbyall
stakeholders.Throughthereadingandonlinequizthestudentsareabletoindicatewhatareasthey
arehavingdifficultywithinthecourseprogram.Thelecturersareabletorespondtotheseissuesvia
emailorinthelecturesituation.Astandardpartoftheteachingprocessisasegmentatthe
beginningofeachlecturewherethelecturerhasidentifiedaproblemthatmanyoftheclassare
experiencing.Theimmediacyoftheresponseisthekeyhere;students,whomayhaveexperienced
somedifficultystudyingindependently,knowthattheirquestionwillbeansweredinthebeginning
ofthelecture.Thecontinuousfeedbackfromtheonlinequizandthroughthelectureactivitiesis
alsoaverypowerfultoolforthelecturer.Itallowsthelecturertotailorthelearningprogramto
meettheneedsofthestudents.
IhavelectureactivitiesandclickersonthetopofthelistIjustfinditgoodinclassitisgoodtogetfeedbackanditisimmediatefeedbackforexamplelikeifIgetitwrongthenwellnotsomuchifIgetinwrongbutiftheclassgetsitwrongthentheyexplainitorwehavetotalktothepeoplearoundussoeitherexplainingittosomeoneorhavingthemexplainittomehelpsitquiteabitanditincreasesmyunderstandingofwhatisgoingon.Whatwasreallygoodformeheactuallyemailedmebackandgavetheresponsestothequestionsitisreallygoodtosee…thatpersonalfeedback.ThelecturerhasemailedandtoldmemyresponsesarenotgoodenoughandthatIwouldneedtowriteabitmoredetailinthefuturetoreceiveamark.Youneedtobemorespecificabouttheformulashesaid.Thelecturergoesthroughthethingsthatpeoplearehavingproblemswithatthebeginningofthelecture.Sometimesitseemslikeit’snotthatdifficultuntilafterwardsafterthelectureIwouldliketoasksomequestions.
ObservationsThefollowingsectionisinformedbythestructureoftheObservationGuide(seeAppendix6)and
outlinestheactivelearningprocessesasobservedinthelecturetheatresofPHYS1001and
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 15
PHYS1002inSemesterOne(seePhaseThree-DataCollectionProcess)intableform.Thetable
identifiesprocessesthatseemedtobefollowingamoretraditionallecturestyleinthefirstinstance
andprocessesthatseemtoexemplifyactivelearninginthesecondinstance:
Moretraditionallecturestyle Activelearninglecturestyle
StudentInformation
PowerPoint, learning outcomes for the
day,feedbackfrompreviouslecture
Structured PowerPoint including details
about learning outcomes, feedback from
thequiz,recognitionofissuesexperienced
bystudents,studentquote,connectionto
outside learning, references to pre
readingandacartoon.
Lectureimplementation-resources
Radiomicrophone,PowerPoint
presentation,visualiser,clickers
Reallifepropsandexamples,multimedia
examples
Lectureimplementation-Process
Onscreenenvironmentvisualiser
Seriesofproblemswithsomeexplanation
inbetweentheproblemsolving.
Somedefinitionsandthentoaconcept
problemthatthestudentsweredirected
todiscusswitheachotheriftheydidnot
understanditandthentovote.
Everyonehadpolledtheanswerthen
workedthroughtheproblemonthe
visualiserprovidingalongexplanationand
thenthestudentsre-polled.Some
discussionfollowed.
Introductionofideausingapractical
examplebothonthescreenandusing
props.
Conceptquestionposed.
Studentworkingontheproblem
individuallyandvoting.
Resultswerepolledbythelecturer
howevernotmadepublic.
Discussedtheresultsbriefly;students
weretheninstructedtotalkwithpeers.
Thelecturetheatrelightswentupwhen
studentsweresupposedtotalkand
dimmedslightlywhenthelecturerhadthe
floor.Thelecturermadeuseofthis
momentoffreedomandmovedaround
thetheatretoengagewithsinglestudents
andsmallgroups.
Afteraprescribedamountoftimethe
studentsre-polledandtheresultswere
discussedasawholegroup.Atdifferent
pointsofthelecturethelecturerasked
questionsofthewholegroupand
receivedresponsesfromavarietyof
students.
Throughoutthelecturetherewere
referencestothereadingthatthe
studentshadcarriedoutpriortothe
lecture.
Learningopportunities
• listening and taking notes while
lecturerexplainsproblem
• answering questions posed by
thelecturertothewholeclass
• answering questions using the
clickersafterpeerdiscussion
• workingwithapartnerorasmall
group.
• answering questions posed by the
lecturertothewholeclass
• answering questions using the
clickers,individually
• opportunity to work independently
first
• abletoworkwithapartnerorasmall
groupand
• some opportunities to interact with
thelectureronetoone.
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 16
Studentengagement
Whenaskedtotalkabouttheproblem
studentsrespondedquietlyinmostcases.
Somestudentswhowereactively
engaged,conversingwitheachotherand
pointinganddrawingdiagramsonpaper.
Studentsoftenappearedtobewaitingfor
thefirstpolltorespond,ratherthan
makinguptheirownmindsorworking
throughtheproblem.
Levelofengagementacrosstheclass
highermostofthetime.
Studentsbecamemoreinvolvedwhenthe
lecturerdrewthemintotheproblem
solvingstage.
Studentsworkedontheproblemsas
individuals.
Studentsengagedwiththestudents
aroundthemduringthepeertopeer
discussion
Allthestudentswhohadclickers
respondedtothequestions.
Studentsengagedwiththelecturerasa
wholeclass.
Table5:Similaritiesanddifferencesbetweenactivelearningandotherlecturingprocesses
OverallobservationsThissectionsummariseswhatwasparticularlyeffectiveaboutthelecturesincoveringkeyconcepts,
thequalityofinterpersonalinteractionbetweenthestudentsandlecturersandgeneralobservation.
Theactivelearningnatureofthelectureswasparticularlyeffectiveinensuringthelearning
outcomesofthelecturewereachieved.Thelecturesweresuccessfulbecause:
• Thebeginningofthelecturewassituatedinthecontextoftheprereading,theonlinequiz
andthelearningfortheday.
• Theprereadingandonlinequizwerehighlightedasimportantaspectsoflessonpreparation
andthelecturerrespondedtoquestionsandtheresultsoftheonlinequiz.Theimmediacyof
thisresponsetothereadingandonlinequizisapowerfultoolforstudentengagement.
• Thepreviouslecturewasreferredtoandthelinkswithpreviouslearningmade.
• Thelearningoutcomesoftheday’slecturewerehighlightedandreferencewasmadeto
howtheconceptsmightworkintheworld.
• Theslideswereclear,informativeandengaging.
• Thestudentsunderstoodtheirroleinthelectureprocessandrespondedtothevisualand
verbalcuesasprovidedbythelecturer.Theydiscussedproblemswhenrequired,responded
toquestionswhenaskedandgenerallylistenedtothelecturerwhenappropriate.Itis
importanttonotethattherewastimeallowedfortheindividualtoworkonaquestionprior
topollinganddiscussionwithpeers.
• Theuseofthetablettocontrolthescreenandaddtoslidesallowedthelecturertointeract
withthestudentsmorereadily;theconstant“eyeball”contactisastrengthofthismodeof
communication.
• Theradiomikeallowedthelecturerthefreedomtomovearoundthelecturetheatregiving
himcontactwithindividualsandsmallgroups.
• Theuseofsimpleprops(abikepump)groundedthelectureintherealworldallowingthe
studentstoscaffoldtheirlearningandtobuildaframeworkinwhichtodevelopthe
necessaryconcepts.
• Intheclosingstagesofthelecture,referencewasmadetofuturelecturesandtheir
relevancetotheforthcomingexaminations.
• Therewasaveryclearendtothelecture.
Otherbehavioursobservedduringthelectureswerenotasclearlypartoftheactivelearningprocess
astheobserversunderstoodit.Somedifferencesnotedbytheobserversincluded:
• Whilethelecturerreferredtoapreviouslectureandcompletedaproblemfromthatsection
ofwork,therewaslessconnectionmadewiththelearningthathadtakenplacebefore.
EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 17
• Therewaslessemphasisbeingplacedonthepre-readingandonlinequiz.Studentsare
requiredtocompletetheprereadingandonlinequizonthebasisthatthelecturewill
developtheideaandprovidethemwithmoreinsight.Withouttheemphasis,thepre
readingmaybecomedevaluedinthestudents’eyes.Mentionwasmadeofithowevernot
tothesameextentandnotvisuallyonaslide.
• Ratherthantheclickerquestionbeingposedafterthedevelopmentofaconceptitwas
oftenusedasthecatalyst.Studentsweregiventhequestion,askedtopolland/or“discussit
withtheirpeersiftheywantedto”andthenthelecturerwouldgothroughtheproblemon
thevisualiser.Therewasnottimeallowedforindividualcontemplationoraspecified
“discusswithyourneighbour”time.Asaconsequencetherewaslessanimationand
engagementinthelectureaudienceduringthistime.
• Theuseofthevisualiserastheprimarymodeofcommunicationapartthetraditionallecture
styleapproachreducedtheamountoffacetofacecontactthatthestudentshadwiththe
lecturer.Withoutthefocusofthelecturer,studentsbecameeasilydistractedbypeersand
communicationtechnology.
EvaluationUnit,TEDI,UniversityofQueensland|Conclusion 18
Conclusion
Thisinterimreportsoughttoevaluatetheimpactoftheimplementationofanactivelearning
processthatisbeingusedasthebasisforstimulatingstudentlearningandengagementinPHYS1001
in2011andnowinbothPHYS1001andPHYS1002in2012.
Theexpectedaimsandbenefitsofthissysteminclude:
• toencouragestudentstodevelopself-directedlearningskills,
• toencouragestudentstoreflectontheirreadingmaterial,
• toencouragestudentstoreflectontheirlearninginclassandprovidefeedback,and
• toallowteachingstafftoidentifythemostdifficultconceptsforstudents,
• Toincreasestudentengagementintheirlearning
Thereisevidenceinthedatacollectedthatthecontinuedimplementationandrefinementofthe
activelearningprocessdoesimprovelearningoutcomesforstudents.
1. Thenatureoftheactivelearningprocessallowsforstudentstointeractwitheachotherinthelecturetheatre.Thestudentswhoattendedthefocusgroupsessionsweregenerallyverypositiveaboutthe
activelearningprocess.Havingbeenexposedtoanumberofteachingmethodsintheir
shorttimeattheuniversity,theywereclearaboutwhatwasworkingforthem.
2. Thedatacollectedfromthesurveysprovidesalessclearpictureofthebenefitofutilisingactivelearningprocessesinfirstyearphysicscourses.Comparisonsbetweenpreandpostsurveyforbothcohorts’datadoesnotprovideany
clearerunderstandinginchangesinattitudeacrossthesemester.Somechangesare
statisticallysignificant;however,theyarenotconsistentacrossallthedata.Overall,
PHYS1001studentsaremorepositiveacrosstheentiresetofitemsthanPHYS1002students.
Thedifferencesintheresultsbetweenthetwocoursescouldbeforanumberofreasons.
ThemorerecentutilisationoftheactivelearningprocessesinPHYS1002maymean
variationsinhowtheapproachisbeingimplemented.Anecdotally,therearesuggestions
thatstudentsperceivePHYS1002courseasmorecomplexanddifficultandthusrespond
accordingly.Moreinvestigationintothesedifferencesiswarranted;adeeperunderstanding
ofthetwocoursesandhowtheyarepresentedmayprovideanswers.
3. Onecannotidentifyasingleaspectoftheapproachthatworksaboveallothersinthelecture;itistheintegrationofallthepracticesintoacoherentprocessthatmakesitsuchapowerfulteachingandlearningintervention.Moststudentsareengagedfromthefirstmomentbyaprocessthatutilises;theirpreviously
acquiredknowledgeandknowledgeapplication(theprereadingandquizstage),the
challengeofproblemsolving(clickerquestions),theoptionofthegroupthink(sharingwith
fellowstudents)andthegeneralinteractionsbetweenstudentandlecturer.
4. Observationsoflecturessuggestthatmoreexperiencedpractitionersofactivelearningteachingarebetterabletoengagethestudents.Lessexperiencedpractitionersarestillworkingtocapturetheessenceoftheactivelearning
processandtherebyfallsomewhereinthemiddle,notquiteabletoprovidethefullbenefit
ofa“traditionallecture”andnotyetabletoachievewhatbestpracticeintheactivelearning
processdelivers.
Inconclusion,thedatagatheredsuggeststhattheactivelearningteachingframeworkisavaluable
oneandthat,whentheintegratedinstructionalpracticesareadheredto,theactivelearningprocess
EvaluationUnit,TEDI,UniversityofQueensland|Conclusion 19
hasthepotentialtoreducetheatmosphereinafulllecturetheatretoonethatresemblesthatofa
smalltutorialwithallitsinherentintimacy,thepromiseofstudentengagementandimproved
learningoutcomes.
AreasforfurtherinvestigationThefindingsfromthesurveys,observationsandfocusgroupidentifiedthefollowingareasthatmay
needsomefurtherinvestigation.
1. Oneoftheissuesraisedbystudentsinthepreviousphaseoftheevaluationwastheamountoftimerequiredtocompletetheprereadingandquizsectionofthework.Thisviewwasidentifiedin2011andcontinuedtobeathemein2012.Whilethestudents
agreedthatthepre-readingwasanintegralpartoftheprocess,theyalsofounditdifficultto
finishsometimes,particularlythestudentswhowereenrolledinsomeofthemoreintensive
programs.Furtherinvestigationcouldprovidedataastowhethertheissuewasoneoftime
managementonthestudents’behalforifindeedthereadingdemandswerebeyondthe
timeprescribedbyuniversitypolicy.
2. Therewasasuggestioninthefocusgroupsthatthisaspectofbeing“stronglyencouraged”or“madeto”tositwithotherstudentswasnotalwayswelcomeandthattheforcednatureofthegroupdiscussionengenderednegativefeelsforsomerespondents.However,somestudentswereverypositiveaboutthisaspectofthelecture,tothepoint
wheretheycarriedthediscussionsoutofthelecturehallandintowhatappearedtobean
informal‘communityofpractice’(LaveandWenger,1991).Theseextremesinresponse
couldbeinterestingtoinvestigategiventhepotentiallypowerfuleffectspeerinteractioncan
haveonstudentlearning(Webb,1989).
3. Thereisaneedtosupportthedisseminationandadoptionoftheteachingpracticesinothercourses.Aswiththeintroductionofanynewapproach,regularmonitoringandreflectionallowsfor
therefiningofthedifferentaspectsoftheapproach.Iftheactivelearningapproachistobe
embeddedinfirstyearphysicscourses,thensomestandardisationoftheprocessacross
lecturersandcoursescouldbeuseful.Theproductionofabrief“howto”guideandsome
peerobservationofwhatisdeemedbestpracticemightbeadvantageous.
4. Aninstructionalpracticethatresultsinimprovedlearningoutcomesinonecoursemaynotnecessarilyprovidethesameoutcomesinasecondsituationformanyreasons.Furtherinvestigationintothecurriculumdesign,studentpopulationandpedagogyof
PHYS1001andPHYS1002mayprovideinsightintohowactivelearningprinciplesmaybest
supportthedeliveryofthesetwocourses.
Finally,thismustbeconsideredaninterimreport,bothprimaryandsecondarydataisstill
beinganalysedandwillbeincludedinasummaryreportatalaterdate.
EvaluationUnit,TEDI,UniversityofQueensland|REFERENCES 20
REFERENCES
Deslauriers,L,Schelew,E.,Wieman,C.(2011)‘ImprovedLearninginaLarge-EnrolmentPhysics
Class’,Science332,862
Keller,J.M.(1987).IMMS:Instructionalmaterialmotivationsurvey,FloridaStateUniversity
Keller,J.M.(1987).‘Strategiesforstimulatingthemotivationtolearn’PerformanceandInstruction,26,No.8,pp362-632
Lave,J.,&Wenger,E.(1991).Situatedlearning:Legitimateperipheralparticipation.Cambridge
UniversityPress.
Laws,P.,Sokoloff,D.andThornton,R.(1999).‘PromotingActiveLearningUsingtheResultsofPhysic
EducationResearch’,UniServeScienceNews,Vol13
Prince,M.(2004).‘DoesActiveLearningWork?AReviewoftheResearch’JournalofEngineeringEducation93(3),pp223-231
Webb,N.M.(1989).Peerinteractionandlearninginsmallgroups.InternationalJournalofEducationalResearch,13(1),21-39.
EvaluationUnit,TEDI,UniversityofQueensland|Appendices 29
APPENDIX3:FOCUSGROUPDISCUSSIONGUIDE
Engaging first year students in classroom discussion of high-level concepts: pre-reading, reflection and personalised feedback.
1. Introduction 1a. Welcomeandintroductionofmoderator1b. ObjectiveTheobjectiveofthisresearchistoinvestigatetheeffectivenessofanalternativemethodofteaching
introducedinMechanics&ThermalPhysicsI(PHYS1001)andElectromagnetismandModernPhysics
(PHYS1002).Thisinvolvesusinglecturetimeforin-classdiscussionbasedonpre-reading.
1c. ProcessWewillbeconductinga1.5hourfocusgroupwith5–7studentswhoareenrolledon
PHYS1001/1002insemester1,2012.
Discussionwillfocuson
• Youropinionofthetextbookandpre-readingactivities
• Yourconfidencelevelinunderstandingphysics
• YouropinionsonthelearningactivitiessuchasLabs,tutorialsandweeklyproblemsheetsin
PHYS1001/1002
• Yourexperiencesoflearningthroughdoing
Pizzaandsoftdrinksareavailable.Giftcardstothevalueof$30willbeprovidedinappreciationof
yourinvolvement.
Yourpermissionhasbeensoughttoaudiotapetheproceedingstoenabletheresearchertorevisit
theinformationtoensureaccurateanalysis.
Anyinformationthatisobtainedinconnectionwiththisstudyandthatcouldbeidentifiedas
relatingtoyouwillremainconfidentialandisdisclosedonlywithyourpermission.
Remember,youarefreetowithdrawyourconsentandtodiscontinueparticipationatanytime
withoutprejudice.Informationprovidedisfedbacktotheteachingteaminreportthatwillinclude
anoverviewofwhathasbeensaid,inade-identifiedformat.Yourlecturerswillneverbeableto
attributeyourresponsetoyouandyouarewelcometoaskforacopyofthereportaswell.
1d. Rules–honestopinions,oneatatime,lotsofgroundtocover,OKtohavedifferentopinions
1e. Introductionofparticipants-name,course,careeraspiration
Questions
1. OVERALLIMPRESSIONS
‘I’minterestedfirstofftofindoutabitaboutyourperceptionaboutPHYS1001/1002…’
PROMPTS:
• Howwouldyoudescribethiscourse?
• Whatwerethe“learningobjectives”ofthiscourse,meaningwhatwereyoumeanttolearn,
inthebigpicturesense,fromthiscourse?
TASK
EvaluationUnit,TEDI,UniversityofQueensland|Appendices 30
• Whatgradedoyouthinkthatyouwillgetforthiscourseattheendofthesemester?Write
downwhatyouthinkonthepostitinfrontofyou–don’tletanyonesee,folditupandhand
themintome.
• We’llhavealookattheminafewminutes
2. THEPRE-READINGTASKS
WhatI’dliketoturntonowisthetaskofthepre-reading.I’mnotsurewhatthatactuallyentails.
Cansomeonepleasedescribetomewhatthepre-readingandquizisactuallyabout?
• Doyoureallydoitbeforehand?Howlongbeforethelecture?Whathappensifyoudon’tdo
it?
• Howmuchtimedoesthepre-readingtaskandquiz?
• Isthattimewellspent?
• Sowhatisitthatyouactuallydo?
Let’stalkabouthowyougoaboutthetaskofdoingthepre-reading.TASK
I’vegotacopyofoneofthepre-readingtasksandquizzeshere.Canyoudescribetomehowyoudo
thisthing?
Prompts:
• Doyounotedownthepointsthatyouhaveproblemswith?
• Doyouwritedownthethingsthatyouhavetroublewith?Whogoesthoroughthemwith
you?Doesithelp?
• Whatdoyouthinkaboutthematerial–doesitreallyhelpyoumakesenseofcomplex
things?
3. COURSEDESIGN
I’dliketolookatthevariousactivitiesthatyoudointhecourse
TASK:
Firstly,I’dlikeyoutoarrangethevariouselementsofyourcoursethatwarewrittendown
onthesecardsinorderofthecontributionthattheymaketohelpingyouunderstand
physics.Puttheonethathelpsthemostupthetopandranktherestdowntowhathelps
theleast
Let’stalkabouthowthepre-readingfitsinwiththerestofthethingsthatyoudo
PROMPT:
• Howdoesthereadingassistyouin:
o Lectureactivities
o Groupactivities
o Tutorialworksheets
o Weeklyproblemsheets
o Labs
o Pre-readingexercises
• Doyougotothelectures?Why/whynot?
• Whathelpsyou‘do”science?
4. ASSESSMENTANDGRADING
EvaluationUnit,TEDI,UniversityofQueensland|Appendices 31
‘I’dliketohearaboutyourassessmentitemsnowinPHYS1001/1002…’
• WhatformsofassessmentareworkingwellforyouinPHYS1001/1002?Why?
• Doyouthinkit’sfairthatthepre-readingquizisgraded?
• Let’slookatthegradesyouwrotedownonthepost-its.Iseethatyourscoresare....
o Doyouthinkthatyouwouldbeachievingthiswithoutthepre-readingtasks?
o Whatcontributiontothegradedoyouthinkthere-readingtaskmakes?
Conclusion
Weareprettymuchdone;Ireallyappreciateyourcandidfeedback.Beforewegettothegiftvouchers,isthereanythingyouwouldliketosayaboutPHYS1001/1002?Anythingwedidnotcoverinthissessionthatyoufeelwouldbenefitustohearabout?
• Thankyouforyourparticipation
• Pleasesignforandcollectyourdebitcard
EvaluationUnit,TEDI,UniversityofQueensland|Appendices 32
APPENDIX4:STUDENTFOCUSGROUPCONSENTFORM
Project title: Engaging first year students in classroom discussion of high-level concepts: pre-reading, reflection and personalised feedback.
Youareinvitedtoparticipateinresearchinvestigatingtheeffectivenessofanalternativemethodof
teachingintroducedinMechanics&ThermalPhysicsI(PHYS1001)andElectromagnetismand
ModernPhysics(1002).Thisinvolvesusinglecturetimeforin-classdiscussionbasedonpre-reading.
TheresearchisbeingundertakenbyKarenSheppardandDeanneGannawayfromTEDIonbehalfof
M.Drinkwater,M.Davis,W.Bowen,J.Corney,T.McIntyre&M.WegenerfromtheSchoolof
MathematicsandPhysicsatTheUniversityofQueensland
Ifyoudecidetoparticipate,youwilltakepartinoneofaseriesofone-and-a-halfhourfocusgroups
withfivetosevenPHYS1001/1002UniversityofQueenslandstudents.
Studentfocusgroupswillexplorethefollowing:
• Youropinionofthetextbookandpre-readingactivities
• Yourconfidencelevelinunderstandingphysics
• YouropinionsonthelearningactivitiessuchasLabs,tutorialsandweeklyproblemsheetsin
PHYS1001/1002
• Yourexperiencesoflearningthroughdoing
Pizzawillbeprovidedduringthefocusgroup.Yourpermissionwillbesoughttoaudiotapethe
proceedingstoenabletheresearchertorevisittheinformationtoensureaccurateanalysis.A$30
DebitGiftCardwillbeprovidedtoparticipantstoacknowledgeyourparticipation.
Werespectyourrighttoprivacy.OurPrivacyPolicyanditsprocessesensurethatanyinformation
thatisobtainedinconnectionwiththisstudyandthatcouldbeidentifiedasrelatingtoyouwill
remainconfidentialandwillbedisclosedonlywithyourpermission.Yourindividualresponseswill
neverberevealedtoyourlecturers.
Ifyoudecidetoparticipate,youarefreetowithdrawyourconsentandtodiscontinueparticipation
atanytimewithoutprejudice.
Ifyouhaveanyquestions,pleasefeelfreetoaskus.Ifyouhaveanyadditionalquestionslater,
KarenSheppard,ResearchOfficer,EvaluationServicesUnit,TEDI(email:[email protected])
willbehappytoanswerthem.
I have read the information above, and agree to participate in this study. Name of Participant: ……………………………………………… Signature of Participant: ……………………………………….… Date: ………………………
EvaluationUnit,TEDI,UniversityofQueensland|Appendices 33
APPENDIX5:FURTHERTESTINGTable1:MeansandstandarddeviationsofPHYS1001studentspre-survey(n=96)andPHYS1002studentspre-survey(n=75)
wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.
PHYS1001
PHYS1002
Significance
LearningActivityfeedback Pre
Mean
(SD)
N=96
Pre
Mean
(SD)
N=75
Completingonlineproblems(MasteringPhysics) 3.95
(1.09)
3.27
(1.15)
<0.001
Participatingintutorials/conceptlabs 2.97
(1.45)
3.50
(1.17)
0.013
Completingpracticallabs 2.21
(1.72)
3.32
(1.23)
<0.001
Theuseof“clickers”duringthelecture 3.73
(1.33)
3.66
(1.07)
Discussingquestionswiththepersonnexttomeduringthelecture 3.74
(1.27)
3.27
(1.24)
.018
Discussingquestionsasawholeclassduringthelecture. 3.79
(1.12)
3.41
(1.20)
.046
Doingtheassignedreadingtasksandonlinequiz. 3.60
(1.09)
3.36
(1.24)
AttendingPASSclasses. 2.46
(1.74)
2.23
(1.53)
Table2:MeansandstandarddeviationsofPHYS1001studentspost-survey(n=76)andPHYS1002studentspost-survey
(n=53)wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.
PHYS10
01
PHYS10
02
LearningActivityfeedback Post
Mean
(SD)
N=76
Post
Mean
(SD)
N=53
Signific
ance
Completingonlineproblems(MasteringPhysics) 3.99
(1.25)
3.17
(1.13)
<0.001
Participatingintutorials/conceptlabs 3.23
(1.26)
3.34
(1.28)
Completingpracticallabs 3.26
(1.24)
3.30
(1.22)
Theuseof“clickers”duringthelecture 3.62
(1.36)
3.33
(1.21)
Discussingquestionswiththepersonnexttomeduringthelecture 3.65
(1.40)
3.13
(1.33)
0.022
Discussingquestionsasawholeclassduringthelecture. 3.72
(1.29)
3.25
(1.28)
0.046
Doingtheassignedreadingtasksandonlinequiz. 3.56
(1.01)
3.46
(1.21)
AttendingPASSclasses. 2.08
(1.63)
2.44
(1.67)
Table3:MeansandstandarddeviationsofPHYS1001studentspre-survey(n=96)andPHYS1002studentspre-survey(n=75)
wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.
PHYS1001
PHYS1002
GeneralPhysicsfeedback Pre
Mean
(SD)
N=96
Pre
Mean
(SD)
N=75
Signific
ance
Iamenthusiasticaboutphysics 4.04
(.96)
3.35
(1.09)
<0.001
Iaminterestedintakingextraphysicsclasses 3.82
(1.18)
2.84
(1.08)
<0.001
IamconfidentIunderstandthecoursematerial 3.69 3.14 <0.001
EvaluationUnit,TEDI,UniversityofQueensland|Appendices 34
(.92) (1.02)
Ineedtomemoriseformulaetogetagoodmarkinthisclass 2.56
(1.05)
3.42
(1.12)
<0.001
Ineedtounderstandconceptsinphysicstogetagoodmarkinthiscourse 4.66
(.71)
4.35
(.82)
0.010
Ihavecompletedalltheassignedreadinginthiscourse 3.65
(1.38)
3.16
(1.32)
0.024
Table2:MeansandstandarddeviationsofPHYS1001studentspost-survey(n=76)andPHYS1002studentspost-survey
(n=53)wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.
PHYS10
01
PHYS10
02
GeneralPhysicsfeedback Post
Mean
(SD)
N=76
Post
Mean
(SD)
N=53
Signific
ance
Iamenthusiasticaboutphysics 3.88
(1.01)
3.33
(1.18)
0.003
Iaminterestedintakingextraphysicsclasses 3.64
(1.27)
2.70
(1.28)
<0.001
IamconfidentIunderstandthecoursematerial 3.47
(.95)
3.11
(.92)
0.030
Ineedtomemoriseformulaetogetagoodmarkinthisclass 2.77
(1.17)
3.26
(1.14)
.018
Ineedtounderstandconceptsinphysicstogetagoodmarkinthiscourse 4.56
(.64)
4.54
(.66)
Ihavecompletedalltheassignedreadinginthiscourse 3.29
(1.34)
3.38
(1.19)
EvaluationUnit,TEDI,UniversityofQueensland|Appendices 35
APPENDIX6:PHYS1001/1002ObservationGuideCourse:Observer:Criteria CommentsBackground
Summariseyourrationaleforchoosingthismethod,howitisplannedtoinformtheevaluation,anditsroleinsupportingothermethods.WhataretheconstraintsandopportunitiesinfluencingyouraccesstotheL&Tcontextforobservations(e.g.timingoftheobservation,typeofparticipantsinvolved,typeofactivitiesthatcanbeobservedetc...)?
Purpose
Summarisetheevaluationquestionsyouhopetoexplorewiththeobservation.
q Whatinformationisgiventostudentsabout:-• how,when,withwhom,todotheactivityetc?• theeducationalaims,objectivesandexpectedlearningoutcomesoftheactivity?• howtheactivityisexpectedtohelptheirlearningforthecoursewhichitformsapart?• howandfromwhomtheycangethelpiftheyneedit?• whatlearningresourcestheycanusetohelpthemdotheactivityandwhere/howtheycanget
accesstothem?• howtheactivityfitsintoanyassessmentforthecourse,andifitisassessed,themarking
criteria?• therelevanceofthelearningbeyondmeetingtheassessmentrequirements,e.g.infuturelife,
practiceetc.?
q Howistheactivityimplemented?• whoistakingpart?• numberofparticipants• natureoftheactivityandteachingapproach• timingandlocationoftheactivity• howtheactivityisorganised• howtimeisusedduringtheactivity• rolesandresponsibilitiesofparticipants
EvaluationUnit,TEDI,UniversityofQueensland|Appendices 36
• decisionsbeingmadebywhomandforwhom• resourcesmadeavailabletostudentse.g.specialequipment,learningresources,software,
virtuallearningenvironmentsetc• helpavailabletostudents• Howaretheclickersbeingused?
q Howaretheparticipantsbehaving?• Howaretheyundertakingtheactivity?• Howarestudentsusinghelpandresources?• Howarestudentsinteractingwiththelearningenvironment?• Dostudentsappearmoremotivated,engaged,orbetterprepared?
q Howaretheparticipantsinteracting?• Istheredialogue?• Isthedialogueconstructiveforlearning?• Whoistalking/listening?• Whatistheirbodylanguage/non-verbalinformation?• Isthereevidenceinthedialoguethatstudentsarelearning?• Howarestudentslearningfromthedialogue?(e.g.staff-student,peer-peerdiscussion,group
inquiry….etc)• Isthereevidenceinthedialoguethatacademicstaff/supportstaffarerespondingtostudents’
learningneeds?• Howisfeedbackbeinggiventostudents?
q Whatistheevidencethatstudentshaveachievedexpectedlearningoutcomes,• fromtheircompletionofactivities,assessment?• fromtheirbehaviouranddialogue?• fromtheirlevelofachievement?
qOthercomments