on the inside: engaging first year physics students in ... · physics teaches a student how to do...

39
Prepared by Evaluations Service Unit, TEDI /Users/mjd/Documents/Admin/Grants/UQ teaching/Reports/PhysicsReportInterim2012_print.docx 1 On the inside: Engaging First Year Physics Students in Classroom Discussion of High-level Concepts: Pre- reading, Reflection and Personalised Feedback School of Mathematics and Physics Prepared Semester 2, 2012 For Professor Michael Drinkwater Prepared by Karen Sheppard and Deanne Gannaway Evaluation Unit Teaching and Educational Development Institute The University of Queensland

Upload: others

Post on 20-Mar-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

PreparedbyEvaluationsServiceUnit,TEDI

/Users/mjd/Documents/Admin/Grants/UQteaching/Reports/PhysicsReportInterim2012_print.docx

1

Ontheinside:EngagingFirstYearPhysicsStudentsinClassroomDiscussionofHigh-levelConcepts:Pre-reading,ReflectionandPersonalisedFeedbackSchoolofMathematicsandPhysics

PreparedSemester2,2012

ForProfessorMichaelDrinkwater

PreparedbyKarenSheppardandDeanneGannawayEvaluationUnit

TeachingandEducationalDevelopmentInstituteTheUniversityofQueensland

PreparedbyEvaluationsServiceUnit,TEDI

/Users/mjd/Documents/Admin/Grants/UQteaching/Reports/PhysicsReportInterim2012_print.docx

2

ContentsOverview...................................................................................................................................1

Background.......................................................................................................................1

EvaluationOutline.............................................................................................................2

Researchproblem................................................................................................................2

RoleofTEDI..........................................................................................................................2

Reporting.............................................................................................................................2

DataCollectionProcess.....................................................................................................2

Phase1:Surveyingfirst-yearexperienceofPHYS1001/1002..............................................2

Phase2:Talkingtothestudents..........................................................................................3

Phase3:Observingtheclasses............................................................................................3

Phase4:Analysisofsecondarydata....................................................................................3

Meta-Analysis....................................................................................................................4

KeyFindings..............................................................................................................................5

PHYS1001/1002StudentExperienceSurveys...................................................................5

PHYS1001PreandPostsurvey............................................................................................5

PHYS1002PreandPostsurvey............................................................................................5

PHYS1001vs.PHYS1002Pre-surveyandpost-survey..........................................................8

Prevs.Postsurveyresultsforbothcoursesforstudentswhocompletedbothsurveys..10

StudentExperience:Focusgroups..................................................................................11

1. Thewholeprocess....................................................................................................11

2. Individualelements..................................................................................................12

Observations...................................................................................................................14

Standard................................................................................Error!Bookmarknotdefined.

Differences............................................................................Error!Bookmarknotdefined.

StudentInformation..........................................................................................................15

Lectureimplementation-resources..................................................................................15

Lectureimplementation-Process.....................................................................................15

Learningopportunities......................................................................................................15

Studentengagement.........................................................................................................16

Overallobservations..........................................................................................................16

Conclusion...............................................................................................................................18

Areasforfurtherinvestigation........................................................................................19

REFERENCES............................................................................................................................20

PreparedbyEvaluationsServiceUnit,TEDI

/Users/mjd/Documents/Admin/Grants/UQteaching/Reports/PhysicsReportInterim2012_print.docx

3

Appendices..............................................................................................................................21

APPENDIX1:PRE-SURVEY.............................................................................................21

APPENDIX2:Post–Survey..............................................................................................25

APPENDIX3:FOCUSGROUPDISCUSSIONGUIDE............................................................29

APPENDIX4:STUDENTFOCUSGROUPCONSENTFORM................................................32

APPENDIX5:FURTHERTESTING.....................................................................................33

APPENDIX6:PHYS1001/1002ObservationGuide..........................................................35

Acknowledgements

Theinvestigationteamwouldliketogratefullyacknowledgeandthankallstudentswhoparticipated

inthisevaluationfortheirtime,honestyandenthusiasmwhileengagingintheprocess.

EvaluationUnit,TEDI,UniversityofQueensland|Overview 1

Overview

BackgroundActivelearningisatermusedtodescribearangeofmethodsofteachingscience.Thefundamental

principlebehindactivelearningrejectstheviewthatpassivelylisteningtoa50-minutelectureon

physicsteachesastudenthowtodophysics.Instead,ittakestheviewthatthestudentneedstobe

activelydoingphysicswiththeguidanceofaninstructor.1AusefuldefinitionfromPrincedefines

activelearningas:

Activelearningisgenerallydefinedasanyinstructionalmethodthatengagesinthelearningprocess.Inshort,activelearningrequiresstudentstodomeaningfullearningactivitiesandthinkaboutwhattheydoing.Whilethisdefinitioncouldincludetraditionalactivitiessuchashomework,inpracticeactivelearningreferstoactivitiesthatareintroducedintotheclassroom.Thecoreelementsofactivelearningarestudentactivityandengagementinthelearningprocess.Activelearningisoftencontrastedtothetraditionallecturewherestudentspassivelyreceiveinformationfromtheinstructor.

(Prince,2004,p.1)

Anessentialprerequisiteforactivelearningisthatstudentscometoclasswithsomefamiliaritywith

thebasicmaterial.Intheprojectteam’sview,regurgitatinginformationprovidedinatextbookisan

inefficientuseofcontacttimewithaninstructor.Abetteruseofclasstimeistohelpstudents

understandthematerialinthebookandbegintoapplytheconcepts.

Educationalresearchhasdemonstratedthatwhensuccessfullyimplementedstudentstaughtinthis

mannermakegreatergainsintheirphysicsabilitiesasmeasuredbeforeandafterinstruction(Laws

etal.,1999).TheprinciplesofactivelearninghavebeenappliedintheteachingofphysicsatThe

UniversityofQueenslandsince2008.Inboth2009and2010itwasfoundthatthisresultedin

improvementofstudents’abilitiesequaltotheworld’sbestpractice,upto200%greatergain

comparedtotraditionallytaughtcourses.

Morerecentresearchinthisareahasresultedinotherexamplesofimprovedlearning.Asmallstudy

carriedoutattheUniversityofBritishColumbiahighlightedtheadvantagesofaprocessof

‘deliberatepracticeofthinkingscientifically’(activeproblemsolvinginlectures),theactofmoving

the‘simpletransferofknowledgeoutsideofclass’(pre-readingtasksandonlinequiz)andfeedback

fromclassmatesandlecturer(groupandclassdiscussion)(Deslauriersetal2011,p.862).

Thefollowingoutlinesastrategyusedtoevaluatetheimpactoftheimplementationoftheactive

learningprocessusedasthebasisforstimulatingstudentlearningandengagementinfirstyear

physicscoursesin2011and2012.TheactivelearningsystemimplementedinPHYS1001in2011had

thefollowingaimsandanticipatedbenefits:

• toencouragestudentstodevelopself-directedlearningskills,

• toencouragestudentstoreflectontheirreadingmaterial,

• toencouragestudentstoreflectontheirlearninginclassandprovidefeedback,and

• toallowteachingstafftoidentifythemostdifficultconceptsforstudents.

AnevaluationofstudentlearninginPHYS1001conductedin2011byexternalreviewerssuggested

thatthesystemdidachievetheaimsanddidresultintheintendedbenefits.

Asaresultofthesefindings,theprojectteamintheSchoolofPhysicsdecidedtoimplementthe

systemacrossbothPHYS1001andPHYS1002in2012.

1FromPHYS1001–anotsobriefguidefor2011.

EvaluationUnit,TEDI,UniversityofQueensland|Overview 2

EvaluationOutlineWhiletheevaluationstrategyusedin2011soughttoevaluatetheuseofprereadingandonline

quizzesinPHYS1001,theevaluationconductedin2012willfocusontheimplementationofactive

learninginPHYS1002.Itwillalsocontinuetocollectdataabouttheefficacyoftheactivelearning

processinPHY1001andisdesignedtoincreaseacrosstheboardbase-linedatathatcanbeusedin

otherdevelopmentalactivitiesproposedforthePhysicsdepartment.

Theevaluationstrategyusedisafour-phasestrategythatmakesuseoffocusgroups,pre-andpost-

surveysofPHYS1001andPHYS1002students,lectureobservationsandanalysisofsecondarydata.

ResearchproblemTheactivelearningapproachmightbeviewedasasuccessifstudentsarebetterabletolearnand

retainkeyconceptsandifteachingstaffareabletoidentifythemostdifficultconceptsforstudents

andprovidemoreintensiveremediationintheseareas.Additionalbenefitswouldbeincreased

studentengagementandretention,increasedmotivationandmoreindependentlearners.

Thisevaluationsoughttotestwhethertheapproachdoesinfactachievethisengagementacross

twocoursesofstudy,PHYS1001(alreadyusingtheactivelearningprocess)andPHYS1002(theactive

learningprocessrecentlyimplemented)giventhatthecoursesaredeliveredbydifferentteachers,

involvedifferentcurriculaandattractdifferentcohortsofstudentstotheclasses.Indoingthisit

mustberecognisedthatasPrince(2004)commented:

…facultyadoptinginstructionalpracticeswiththeexpectationofseeingresultssimilartothose

reportedintheliteratureshouldbeawareofthepracticallimitationsofeducationalstudies.

Educationalstudiestelluswhatworked,onaverage,forthepopulationsexaminedandlearning

theoriessuggestwhythismightbeso.However,claimingthatfacultywhoadoptaspecificmethod

willseesimilarresultsintheirownclassroomsissimplynotpossible.Eveniffacultymasterthenew

instructionalmethod,theycannotcontrolallothervariablesthataffectlearning.

(Prince2004,p.3)

Thesamemightpossiblybeinferredevenwithinfacultiesandschools,andacrosscourseswithin

thosefacultiesandschools.

RoleofTEDITheEvaluationUnit(EU)withintheTeachingandEducationalDevelopmentInstitute(TEDI)was

commissionedbyProfessorMichaelDrinkwater(Physics)toconducttheevaluationasexternal

evaluators.KarenSheppard(TEDI),projectofficerfromEU,preparedtheanalysisandreport.

ReportingThisreportdescribesthedatacollectionandanalysisprocesses;outlineskeyfindingsfromthedata

collectedandconcludeswithasummaryemergingfromthefindingsofthelearningactivitieswhich

mostengagePHYS1001andPHYS1002students.Italsoidentifiesareasofstrengthandweaknessof

thecurrentprogramsandtherecentimplementationoftheactivelearningprocessinPHYS1002,as

reportedbyPHYS1001andPHYS1002students.

DataCollectionProcess

Phase1:Surveyingfirst-yearexperienceofPHYS1001/1002Pre-andpost-surveyswereconductedwithfirst-yearstudentsenrolledinPHYS1001andPHYS1002

toidentifytheirfamiliaritywiththeactivelearningprocessandtodeterminehowmuchthisprocess

assiststhemtoacquirecoreskillsandcontentandmaintainengagement.Thesurveyinstrument,

whilesimilartothepreviousyear,wasmodifiedandincludeditemsaboutotherpartsoftheprocess.

Sample:ThesampleforthisphaseofdatacollectionincludedallstudentsenrolledinPHYS1001and

PHYS1002inSemester1,2012resultinginacohortsizeof196inPHYS1001andacohortsizeof132

inPHYS1002.Toencourageparticipationratesstudentswereofferedtheequivalentoftwotutorial

EvaluationUnit,TEDI,UniversityofQueensland|Overview 3

participationmarksiftheycompletedboththepreandpostsurveys.Atotalof193studentsagreed

toparticipateinthesurvey,providinga53%responserate.Ofthe193,171studentsactually

completedthesurvey,providinga47%responserate.Forthepostsurvey,allstudentswereinvited

totakepart.ByusingtwocollectorsinthesurveyplatformSurveyMonkey,studentswhohadnot

completedthesurveyforthefirsttimewerecapturedinthedataset.Atotalof147agreedtotake

partinthesurvey,however,only132studentsoutof361finishedthesurvey,providinga37%

responseratethesecondtimearound.

Collectionmethod:Theon-linesurveybasedaroundtheInstructionalMaterialMotivationalSurvey

(IMMS)(Keller,1987)wasadministeredviaSurveyMonkey.Thesurveytestedstudents’perceptions

oftheirexperienceswiththeactivelearningprocess.TheIMMSbasedsurvey(SeeAppendices1/2)

wasdesignedtoassessthemotivationalcharacteristicsofinstructionalmaterialsusingthe

Attention,Relevance,ConfidenceandSatisfaction(ARCS)modelofmotivation(Keller,2006).The

themesfocusedonhowthestudentsfeltinresponsetothelessonstheyhadexperiencedinthefirst

threeweeksofSemesterOne.Thesamesurveywasadministeredinthefinalweeksofthesemester

providinga‘post’measureforcomparisonpurposes.Thesurveyswereavailableonlinefor

approximatelytwoweeksandthestudentsreceivedaninitialinvitationplustworeminderstotake

partinthesurvey.

Analysis:EUcollatedthedatafrombothsurveys.EUcollatedthedatafromthesurveysand

conductedadescriptiveanalysis

Phase2:TalkingtothestudentsSample:ThesampleforthisphaseofdatacollectionconsistedofstudentsenrolledinPHYS1001and

PHYS1002.47studentsagreedtoparticipateinthefocusgroups.Atotalof11students,sixmales

andfivefemales,tookpartintwofocusgroupsconductedoveraperiodofaweek.Sevenofthe

studentswereenrolledinPHYS1001andfourwereenrolledinPHYS1002.

Collectionmethods:ThefocusgroupswereconductedatlunchtimeduringWeek10ofthe

semester,byanindependentinvestigator,KarenSheppard.Studentswereinvitedtothemeeting

andtheirattendance,whilerequested,wasvoluntary.Pizzaanddrinkswereprovidedandeach

participantreceiveda$30giftcard.Thefocusgroupswereconductedusingasemi-structured

questioningtechniqueandranforapproximatelyonehour(seeAppendix3forthefocusgroup

guidelines).

Analysis:Athematicanalysisoftherecordingsofthefocusgroupswasconducted.

Phase3:ObservingtheclassesSample:ThisphaseofdatacollectionwascarriedoutinPHYS1001andPHYS1002lecturesoverWeek12and13.ThePHYS1001lecturewasobservedoncebytwoinvestigatorsandthePHYS1002

lecturewasobservedtwice,inthefirstinstancebytwoinvestigatorsandinthesecondbyasingle

investigator.Atotalof233studentswereobserved,96inthePHYS1001lectureand65inthefirst

PHYS1002lectureand72inthesecondPHYS1002lecture.Atotalof233studentswereobserved,

96inthefirstlectureand65inthesecondlectureand72inthethirdlecture.

Collectionmethods:Thelectureobservationswereconductedduringthe11amtimeslotingeneral

lecturetheatres.TheobservationswerecarriedoutbytwoinvestigatorsfromtheEU.Thedatawas

collectedusingamodifiedobservationguideandranforapproximately50minutes(seeAppendix4

fortheguide).Theobserversmetpriortotheobservationstoagreeonthemethodofrecordingand

usedthesameobservationformforallobservations.

Analysis:Analysisoftheobservationaldatawascarriedoutbybothobserverstoimprove

consistencyandreliability.

Phase4:AnalysisofsecondarydataWillbeprovidedwiththefinalreport.

EvaluationUnit,TEDI,UniversityofQueensland|Overview 4

Meta-AnalysisThesurveyresultsandthethemesdevelopedfromthefocusgroupsandobservationdatawere

analysedtoexplorethelinksbetweenhowthestudentsactuallyengageintheactivelearning

processandtheirsuccessindevelopingthefollowing:

• Self-directedlearningskills

• Reflectiononreadingmaterial

• Reflectiononlearninginclassandprovidingfeedback.

Datawasdrawntogethertoidentifyotherpossibleindicatorsforthesuccessoftheactivelearning

systemandthefeaturesoftheimplementationoftheprogramthatencouragedimprovementin

physicsabilitiesinstudentsenrolledinPHYS1001andnowinPHYS1002.

Despitetheincreasednumberofinformantsfromthepreviousyear,thefindingsreportedhereneed

tobeexaminedandinterpretedwithsomecaution.Thefindingscanonlybeviewedasasnapshotof

thecurrentcohort.However,patternsmaybeevidentwhenfindingspresentedherearetakeninto

accountwithotherreports.

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 5

KeyFindings

PHYS1001/1002StudentExperienceSurveysStudentswererequiredtorespondtoanumberofitemswhichinvestigatedthestudents’responses

tothelearningactivitiesconductedinWeekTwoandThreeofSemesterOne.Studentswereasked

torateagreementrelatedtoeachofthestatementsineachitem.Responseswerecategorisedona

5-pointLikertscale,including1(nottrue),2(slightlytrue),3(moderatelytrue),4(mostlytrue)to5

(verytrue),withhigherscoresindicatinghigherlevelsofbelief.Inotherinstances,theywereasked

toratethelearningactivitiesas1(nohelp),2(alittlehelp),3(moderatehelp),4(muchhelp)or5

(greathelp).Includedbelowaretablessummarisingthedatacollectedoverthepre-andpost-

surveysbrokendownforcourseandthedifferentlearningactivitiesthatarepartofthecourses.

Nocomparisonofdatafromthe2012surveysismadewithpreviousyear’sresponsesforeither

PHYS1001orPHYS1002;insteadcomparisonismadebetweenresponsesfromPHYS1001students

andresponsesfromPHYS1002studentsfrom2012.Theevaluationprojectteamhasmadethe

assumptionfrom2011findingsthattheactivelearningprocessappearstoprovideadditional

learningopportunitiesforstudentsenrolledinPHYS1001.Thesecondphaseoftheevaluationnow

seekstogatherdataabouttheimplementationofactivelearningintoasecondcourseinthe

department.

PHYS1001PreandPostsurveyThedatasuggestedthatPHYS1001studentshadsimilarratingsorslightlyhigherratingsforallitems

inthepost-surveywhencomparedtothepre-survey.

Therewasaslightdropinagreementinthefollowingitems

• Thevarietyofreadingpassages,exercises,illustrationsetc.Intheassignedreadinghelpedkeepmyattentiononthelesson,

• IcouldrelatethecontentoftheassignedreadingtothingsIhaveseen,doneorthoughtaboutinmyownlife,

• Theonlinequizmotivatesmetocompletetheassignedreadingbeforeclass;

• Ifindtheinclasslectureactivitiesenjoyable;

• Thecontentoftheassignedreadingandthelecturewillbeusefultome;

• Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthecourse.

Therewasahigherlevelofagreement(≥4.00)inoneiteminthepostsurvey;thisbeingThecontentoftheassignedreadinghelpedmeunderstandthelecture.StudentsenrolledinPHYS1001alsorespondedinapositivewayfortheitemOverallhowyouwouldratethecourse.

PHYS1002PreandPostsurveyFewerstudentsstudyingPHYS1002respondedtothesurvey;howeveranumberofitemsreceived

lowerratings(≤3.0slightlyormoderatelytrue)onboththepre-andpost-surveythantheratingsfor

PHYS1001.Thesedifferences;whilenotalwayssignificantaswewillseeinlateranalysis;indicate

quiteadifferentresponsetotheactivelearningprocessinstudentsenrolledinPHYS1002.The

reasonsbehindthislesspositiveresponsewillbearfurtherinvestigation.Theitemsthatwerearted

lessinPHYS1002thanPHYS1001included:

• Therewassomethinginterestingintheassignedreading;

• Thequalityofthewritingintheassignedreadinghelpedtoholdmyattention;

• IcouldrelatethecontentoftheassignedreadingtothingsIhaveseen,doneorthoughtaboutinmyownlife;

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 6

• Theassignedreadingmademewanttoknowmoreaboutthistopic.

• Ireallyenjoyedtheassignedreadingactivities.

• Aftertheintroductoryinformation,IfeltconfidentthatIknewwhatIwassupposedtolearnfromthisassignedmaterial

• Icanunderstandtheimportantbitsoftheassignedreadingwithoutreadingallofit.

• Thewordingoffeedbackaftertheonlinequiz,orothercommentsinthislecturemademefeelrewardedformyeffort.

• Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture

• Afterworkingonthelessonforawhile,IwasconfidentthatIwouldbeabletopassatestonit.

Inthepostsurveyforsomeitemstherewasnoincreaseinpositiveresponse;infact,students

indicatedsomeitemswerelesstrueinthelatterpartofthePHYS1002courseincluding(seeTable1)

viz:

• Thevarietyofreadingpassages,exercises,illustrationsetc.intheassignedreadinghelpedkeepmyattentiononthelesson;

• Ifindithardtoidentifytheimportantbitsofthereadingmaterial;

• Ireallyenjoyedtheassignedreadingactivities;

• Thewordingoffeedbackaftertheonlinequiz,orothercommentsinthislecturemademefeelrewardedformyeffort;

• Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture;Ifindtheinclasslectureactivitiesenjoyable;

• Thecontentoftheassignedreadingandthelecturewillbeusefultome;andTheactivitiesinthelectureweretoodifficult.

Table1:Overallmeansandstandarddeviationsforallresponsestopre-andpost-surveysforPHYS1001andPHYS1002forallitems.

PHYS1001 PHYS1002

PreReadingandLessonfeedback Pre

Mean

(SD)

N=96

Post

Mean

(SD)

N=76

Pre

Mean

(SD)

N=75

Post

Mean

(SD)

N=53

Overallhowwouldyouratethiscourse? 3.93

(.73)

3.94

(.78)

3.44

(.86)

3.33

(.93)

Therewassomethinginterestingintheassignedreadingthatcaughtmy

attention.

3.10

(.92)

3.25

(.97)

3.00

(1.00)

3.13

(1.02)

Thequalityofthewritinghelpedtoholdmyattention 3.01

(1.04)

3.04

(1.07)

2.92

(1.06)

3.04

(1.19)

Thevarietyofreadingpassages,exercises,illustrationsetc.Intheassigned

readinghelpedkeepmyattentiononthelesson.

3.31

(1.07)

3.28

(1.05)

3.27

(.96)

3.25

(1.02)

IcouldrelatethecontentoftheassignedreadingtothingsIhaveseen,doneor

thoughtaboutinmyownlife.

3.67

(1.08)

3.64

(1.03)

2.89

(0.94)

3.23

(1.07)

Theassignedreadingmademewanttoknowmoreaboutthistopic. 3.01

(1.02)

3.21

(1.07)

2.75

(1.22)

3.11

(1.19)

Ifindithardtoidentifytheimportantbitsofthereadingmaterial. 2.54

(1.26)

2.66

(1.33)

3.25

(1.23)

3.19

(1.16)

Ireallyenjoyedtheassignedreadingactivities. 2.33

(0.98)

2.50

(1.01)

2.43

(1.09)

2.42

(1.14)

Thecontentoftheassignedreadinghelpedmeunderstandthelecture 3.90

(.90)

4.10

(.87)

3.55

(.93)

3.53

(1.07)

Theassignedreadingissoabstractthatitwashardtokeepmyattentiononit. 1.96

(1.02)

2.12

(1.08)

2.64

(1.18)

2.98

(1.23)

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 7

Icouldnotreallyunderstandquiteabitoftheassignedreadingmaterialforthis

lecture.

1.91

(.99)

2.28

(1.16)

2.59

(1.09)

2.66

(1.22)

Aftertheintroductoryinformation,IfeltconfidentthatIknewwhatIwas

supposedtolearnfromthisassignedmaterial.

3.16

(.95)

3.23

(1.11)

2.95

(.90)

2.93

(.96)

Theonlinequizmotivatesmetocompletetheassignedreadingbeforeclass. 3.93

(1.30)

3.63

(1.23)

3.32

(1.29)

3.34

(1.34)

Icanunderstandtheimportantbitsoftheassignedreadingwithoutreadingallof

it.

3.04

(1.20)

3.08

(1.07)

2.73

(1.07)

2.98

(1.08)

Igainedunderstandingbydoingtheassignedreading. 3.68

(1.07)

3.82

(.87)

3.37

(1.02)

3.44

(1.04)

IwouldnotfeelcomfortabletellingmyentireclasswhatIfoundmostdifficult

withthereading(e.g.onadiscussionboard).

2.46

(1.37)

2.46

(1.38)

2.80

(1.19)

2.62

(1.26)

Thewordingoffeedbackaftertheonlinequiz,orothercommentsinthislecture

mademefeelrewardedformyeffort.

2.92

(1.10)

3.00

(1.06)

2.85

(.99)

2.71

(1.23)

Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture. 3.44

(1.15)

3.45

(1.17)

3.12

(1.18

2.96

(1.36)

Ifindtheinclasslectureactivitiesenjoyable. 3.99

(0.98)

3.80

(.94)

3.25

(1.14)

3.06

(1.21)

ItiscleartomehowthecontentofthelecturesisrelatedtothingsIalready

know.

3.73

(.94)

3.73

(.97)

3.15

(.89)

3.21

(.99)

Therewerestories,picturesorexamplesinthelecturesthatshowedmehow

thismaterialcouldbeimportanttopeople.

3.49

(.93)

3.72

(.93)

3.32

(.92)

3.29

(1.09)

Thecontentoftheassignedreadingandthelecturewillbeusefultome. 3.70

(.80)

3.68

(.99)

3.35

(.91)

3.12

(1.08)

Theactivitiesinthelectureweretoodifficult. 1.69

(.87)

1.92

(.96)

2.28

(1.04)

2.23

(1.16)

Afterworkingonthelessonforawhile,IwasconfidentthatIwouldbeableto

passatestonit.

3.47

(1.08)

3.47

(1.04)

2.90

(1.08)

3.11

(1.08)

Theamountofrepetitioninthelessoncausedmetogetboredsometimes. 1.81

(.97)

2.07

(1.13)

1.99

(1.07)

2.40

(1.05)

Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthe

course.

3.76

(1.02)

3.72

(.97)

3.16

(1.03)

3.23

(1.13)

PHYS1001 PHYS1002

LearningActivityfeedback Pre

Mean

(SD)

N=96

Post

Mean

(SD)

N=76

Pre

Mean

(SD)

N=75

Post

Mean

(SD)

N=53

Completingonlineproblems(MasteringPhysics) 3.95

(1.09)

3.99

(1.25)

3.27

(1.15)

3.17

(1.13)

Participatingintutorials/conceptlabs 2.97

(1.45)

3.23

(1.26)

3.50

(1.17)

3.34

(1.28)

Completingpracticallabs 2.21

(1.72)

3.26

(1.24)

3.32

(1.23)

3.30

(1.22)

Theuseof“clickers”duringthelecture 3.73

(1.33)

3.62

(1.36)

3.66

(1.07)

3.33

(1.21)

Discussingquestionswiththepersonnexttomeduringthelecture 3.74

(1.27)

3.65

(1.40)

3.27

(1.24)

3.13

(1.33)

Discussingquestionsasawholeclassduringthelecture. 3.79

(1.12)

3.72

(1.29)

3.41

(1.20)

3.25

(1.28)

Doingtheassignedreadingtasksandonlinequiz. 3.60

(1.09)

3.56

(1.01)

3.36

(1.24)

3.46

(1.21)

AttendingPASSclasses. 2.46

(1.74)

2.08

(1.63)

2.23

(1.53)

2.44

(1.67)

PHYS1001 PHYS1002

GeneralPhysicsfeedback Pre

Mean

(SD)

N=96

Post

Mean

(SD)

N=76

Pre

Mean

(SD)

N=75

Post

Mean

(SD)

N=53

Iamenthusiasticaboutphysics 4.04

(.96)

3.88

(1.01)

3.35

(1.09)

3.33

(1.18)

Iaminterestedintakingextraphysicsclasses 3.82

(1.18)

3.64

(1.27)

2.84

(1.08)

2.70

(1.28)

IamconfidentIunderstandthecoursematerial 3.69

(.92)

3.47

(.95)

3.14

(1.02)

3.11

(.92)

Ineedtomemoriseformulaetogetagoodmarkinthisclass 2.56

(1.05)

2.77

(1.17)

3.42

(1.12)

3.26

(1.14)

Ineedtounderstandconceptsinphysicstogetagoodmarkinthiscourse 4.66

(.71)

4.56

(.64)

4.35

(.82)

4.54

(.66)

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 8

Ihavecompletedalltheassignedreadinginthiscourse 3.65

(1.38)

3.29

(1.34)

3.16

(1.32)

3.38

(1.19)

1=Nottrue,2=slightlytrue,3=moderatelytrue,4=Mostlytrue,5=Verytrue.Note:Figuresinparenthesesrepresentthestandarddeviations.

PHYS1001vs.PHYS1002Pre-surveyandpost-surveyWhencomparingPHYS1001,studentresponseswithPHYS1002students’responsestothepre-and

post-survey,itisevidentthatoverallPHYS1001studentsdemonstratedhigherratingsthan

PHYS1002acrossalloftheitems.Whenthesurveyitemswerebrokendownforlearningactivity

andgeneralphysicsfeedbackthegeneraltrendremainedthesame.Again,itshouldberemembered

thatthepostsurveysamplesizeissmallerthanthepre;however,aseriesofindependentsamplet-

testsindicatedsomestatisticallysignificantdifferencesbetweenthetwocourses.Itemstestedthat

werestatisticallysignificantwhichareofparticularinteresttothestudyincludedthefollowinginthe

pre-survey:

• Theonlinequizmotivatesmetocompletetheassignedreadingbeforeclass,

• Thecontentoftheassignedreadinghelpedmeunderstandthelecture,

• Theassignedreadingissoabstractthatitwashardtokeepmyattentiononit,

• Ifindthein-classlectureactivitiesenjoyable,

• ItiscleartomehowthecontentofthelecturesisrelatedtothingsIalreadyknow,

• Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthecourse.

ThesedifferencesareillustratedinTable2below

Table2:MeansandstandarddeviationsofPHYS1001studentspre-survey(n=96)andPHYS1002studentspre-survey(n=75)wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.Pissignificantat≤0.05

PHYS1001 PHYS1002

PreReadingandLessonfeedback PreMean

(SD)

N=96

PreMean

(SD)

N=75

Significance

Overallhowwouldyouratethiscourse? 3.93

(.73)

3.44

(.86)

<.001

Therewassomethinginterestingintheassignedreadingthatcaughtmy

attention.

3.10

(.92)

3.00

(1.00)

Thequalityofthewritinghelpedtoholdmyattention 3.01

(1.04)

2.92

(1.06)

Thevarietyofreadingpassages,exercises,illustrationsetc.Intheassigned

readinghelpedkeepmyattentiononthelesson.

3.31

(1.07)

3.27

(.96)

IcouldrelatethecontentoftheassignedreadingtothingsIhaveseen,doneor

thoughtaboutinmyownlife.

3.67

(1.08)

2.89

(0.94)

<.001

Theassignedreadingmademewanttoknowmoreaboutthistopic. 3.01

(1.02)

2.75

(1.22)

Ifindithardtoidentifytheimportantbitsofthereadingmaterial. 2.54

(1.26)

3.25

(1.23)

<.001

Ireallyenjoyedtheassignedreadingactivities. 2.33

(0.98)

2.43

(1.09)

Thecontentoftheassignedreadinghelpedmeunderstandthelecture 3.90

(.90)

3.55

(.93)

.014

Theassignedreadingissoabstractthatitwashardtokeepmyattentiononit. 1.96

(1.02)

2.64

(1.18)

<.001

Icouldnotreallyunderstandquiteabitoftheassignedreadingmaterialforthis

lecture.

1.91

(.99)

2.59

(1.09)

<.001

Aftertheintroductoryinformation,IfeltconfidentthatIknewwhatIwas

supposedtolearnfromthisassignedmaterial.

3.16

(.95)

2.95

(.90)

Theonlinequizmotivatesmetocompletetheassignedreadingbeforeclass. 3.93

(1.30)

3.32

(1.29)

.004

Icanunderstandtheimportantbitsoftheassignedreadingwithoutreadingallof

it.

3.04

(1.20)

2.73

(1.07)

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 9

Igainedunderstandingbydoingtheassignedreading. 3.68

(1.07)

3.37

(1.02)

IwouldnotfeelcomfortabletellingmyentireclasswhatIfoundmostdifficult

withthereading(e.g.onadiscussionboard).

2.46

(1.37)

2.80

(1.19)

Thewordingoffeedbackaftertheonlinequiz,orothercommentsinthislecture

mademefeelrewardedformyeffort.

2.92

(1.10)

2.85

(.99)

Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture. 3.44

(1.15)

3.12

(1.18

Ifindtheinclasslectureactivitiesenjoyable. 3.99

(0.98)

3.25

(1.14)

<.001

ItiscleartomehowthecontentofthelecturesisrelatedtothingsIalready

know.

3.73

(.94)

3.15

(.89)

<.001

Therewerestories,picturesorexamplesinthelecturesthatshowedmehow

thismaterialcouldbeimportanttopeople.

3.49

(.93)

3.32

(.92)

Thecontentoftheassignedreadingandthelecturewillbeusefultome. 3.70

(.80)

3.35

(.91)

.010

Theactivitiesinthelectureweretoodifficult. 1.69

(.87)

2.28

(1.04)

<.001

Afterworkingonthelessonforawhile,IwasconfidentthatIwouldbeableto

passatestonit.

3.47

(1.08)

2.90

(1.08)

.001

Theamountofrepetitioninthelessoncausedmetogetboredsometimes. 1.81

(.97)

1.99

(1.07)

Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthe

course.

3.76

(1.02)

3.16

(1.03)

<.001

Pissignificantat≤0.05

Similarly,itemsofparticularinteresttothestudythattestedasstatisticallysignificantincludedthe

followinginthecomparisonbetweenPHYS1001andPHYS1002post-surveys:

• Thecontentoftheassignedreadinghelpedmeunderstandthelecture,

• Igainedunderstandingbydoingtheassignedreading,

• Ifindtheinclasslectureactivitiesenjoyable,

• Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture,

• Thecontentoftheassignedreadingandthelecturewillbeusefultome,

• Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthecourse.

ThesedifferencesareillustratedinTable3below:

Table3:MeansandstandarddeviationsofPHYS1001studentspost-survey(n=76)andPHYS1002studentspost-survey(n=53)wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.

PHYS1001 PHYS1002

PreReadingandLessonfeedback PostMean

(SD)

N=76

PostMean

(SD)

N=53

Significance

Overallhowwouldyouratethiscourse? 3.94

(.78)

3.33

(.93)

<.001

Therewassomethinginterestingintheassignedreadingthatcaughtmyattention. 3.25

(.97)

3.13

(1.02)

Thequalityofthewritinghelpedtoholdmyattention 3.04

(1.07)

3.04

(1.19)

Thevarietyofreadingpassages,exercises,illustrationsetc.Intheassignedreadinghelped

keepmyattentiononthelesson.

3.28

(1.05)

3.25

(1.02)

IcouldrelatethecontentoftheassignedreadingtothingsIhaveseen,doneorthought

aboutinmyownlife.

3.64

(1.03)

3.23

(1.07)

.027

Theassignedreadingmademewanttoknowmoreaboutthistopic. 3.21

(1.07)

3.11

(1.19)

Ifindithardtoidentifytheimportantbitsofthereadingmaterial. 2.66

(1.33)

3.19

(1.16)

.018

Ireallyenjoyedtheassignedreadingactivities. 2.50

(1.01)

2.42

(1.14)

Thecontentoftheassignedreadinghelpedmeunderstandthelecture 4.10

(.87)

3.53

(1.07)

.002

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 10

Theassignedreadingissoabstractthatitwashardtokeepmyattentiononit. 2.12

(1.08)

2.98

(1.23)

<.001

Icouldnotreallyunderstandquiteabitoftheassignedreadingmaterialforthislecture. 2.28

(1.16)

2.66

(1.22)

Aftertheintroductoryinformation,IfeltconfidentthatIknewwhatIwassupposedtolearn

fromthisassignedmaterial.

3.23

(1.11)

2.93

(.96)

Theonlinequizmotivatesmetocompletetheassignedreadingbeforeclass. 3.63

(1.23)

3.34

(1.34)

Icanunderstandtheimportantbitsoftheassignedreadingwithoutreadingallofit. 3.08

(1.07)

2.98

(1.08)

Igainedunderstandingbydoingtheassignedreading. 3.82

(.87)

3.44

(1.04)

.030

IwouldnotfeelcomfortabletellingmyentireclasswhatIfoundmostdifficultwiththe

reading(e.g.onadiscussionboard).

2.46

(1.38)

2.62

(1.26)

Thewordingoffeedbackaftertheonlinequiz,orothercommentsinthislecturemademe

feelrewardedformyeffort.

3.00

(1.06)

2.71

(1.23)

Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture. 3.45

(1.17)

2.96

(1.36)

.034

Ifindtheinclasslectureactivitiesenjoyable. 3.80

(.94)

3.06

(1.21)

<.001

ItiscleartomehowthecontentofthelecturesisrelatedtothingsIalreadyknow. 3.73

(.97)

3.21

(.99)

.004

Therewerestories,picturesorexamplesinthelecturesthatshowedmehowthismaterial

couldbeimportanttopeople.

3.72

(.93)

3.29

(1.09)

.019

Thecontentoftheassignedreadingandthelecturewillbeusefultome. 3.68

(.99)

3.12

(1.08)

.003

Theactivitiesinthelectureweretoodifficult. 1.92

(.96)

2.23

(1.16)

Afterworkingonthelessonforawhile,IwasconfidentthatIwouldbeabletopassatest

onit.

3.47

(1.04)

3.11

(1.08)

Theamountofrepetitioninthelessoncausedmetogetboredsometimes. 2.07

(1.13)

2.40

(1.05)

Itiscleartomehowthecontentofthislessoniscontributingtotheaimsofthecourse. 3.72

(.97)

3.23

(1.13)

.010

* Pissignificantat≤0.05

Prevs.PostsurveyresultsforbothcoursesforstudentswhocompletedbothsurveysResponsesfromstudentswhocompletedboththepreandpostsurveysforeitherofthecourses

werecomparedbymean.Whencomparingthesamestudents’responsesthesecondtimearound

therewasfewsignificantdifferences.PairedT-testswereappliedtoaselectionofitemsfromthe

survey.Itemsthatweresignificantincludedapositiveincreasefortheitems

• Thecontentoftheassignedreadinghelpedmeunderstandthelecture• Completingpracticallabs

AmorenegativeresponsewasfoundtotheitemIfindtheinclasslectureactivitiesenjoyableforPHYS1001inthepostsurvey.Therewerenosignificantdifferenceswhencomparingtheitems

selectedfromthepreandpostPHYS1002surveys.Thesedifferencesareillustratedintable4below.

Table4:StudentswhocompletedbothpreandpostsurveysforPHYS1001andPHYS1002testedforitemsofsignificance(* Pissignificantat≤0.05).

PrePHYS1001

n=62

PostPHYS1001

n=62

Sig. 1002n=43

1002n=43

Sig.

PreReadingandLessonfeedback

Overallhowwouldyouratethiscourse? 3.97

(.70)

3.93

(.77)

.742 3.28

(.83)

3.35

(.95)

.570

Therewassomethinginterestingintheassignedreadingthatcaughtmy

attention.

3.06

(.92)

3.29

(.91)

.066 2.84

(.99)

3.09

(99)

.086

Thecontentoftheassignedreadinghelpedmeunderstandthelecture 3.83

(.87)

4.10

(.82)

.020

*3.42

(.88)

3.53

(1.08)

.499

Theonlinequizmotivatesmetocompletetheassignedreadingbefore

class.

4.00

(1.25)

3.69

(1.24)

.055 3.49

(1.18)

3.28

(1.32)

.391

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 11

Igainedunderstandingbydoingtheassignedreading. 3.75

(1.25)

3.87

(.85)

.382 3.24

(.93)

3.36

.96)

.482

Ilearnbestwhendiscussingproblemswithotherstudentsinthelecture. 3.42

(1.15)

3.40

(1.15)

.899 2.97

(1.30)

2.90

(1.34)

.674

Ifindtheinclasslectureactivitiesenjoyable. 4.05

(.91)

3.75

(.97)

.008

*3.05

(1.13)

3.05

(1.17)

1.0

Thecontentoftheassignedreadingandthelecturewillbeusefultome. 3.74

(.79)

3.66

(1.01)

.505 3.22

(.94)

3.05

(1.05)

.227

LearningActivityfeedback

Completingonlineproblems(MasteringPhysics) 3.97

(1.10)

4.14

(1.11)

.213 3.12

(1.06)

3.14

(1.03)

.888

Completingpracticallabs 2.44

(1.73)

3.16

(1.23)

.002

*3.17

(1.25)

3.29

(1.15)

.548

Theuseof“clickers”duringthelecture 3.82

(1.26)

3.63

(1.39)

.242 3.57

(1.08)

3.52

(1.17)

.793

Discussingquestionswiththepersonnexttomeduringthelecture 3.70

(1.20)

3.70

(1.35)

1.00

0

3.14

(1.18)

3.14

(1.28)

1.000

Discussingquestionsasawholeclassduringthelecture. 3.84

(1.28)

3.76

(1.28)

.653 3.34

(1.04)

3.30

(1.27)

.824

Doingtheassignedreadingtasksandonlinequiz. 3.60

(1.12)

3.56

(1.05)

.829 3.36

(1.25)

3.55

(.99)

.345

Thedatacollectedfromthepreviousyearalsoindicateddifferencesinstudentresponsesbetween

PHYS1001andPHYS1002andtheassumptionmadewasthatstudentsinPHYS1001wereresponding

positivelytotheactivelearningprocess.Coursecoordinatorsdecidedtotestthisassumptionby

implementingtheactivelearningprocessinPHYS1002in2012.However,surveyresponsesstill

indicateagapinratingsbetweenPHYS1001andPHYS1002formostitems.

Insummary,itappearsfromthesurveydata;thatdespitetheimplementationoftheactivelearning

processinPHYS1002;thatsomesignificantdifferencesstillexistinthestudents’perceptionsand

responsestothecoursewhencomparedtothoseexperiencedbythestudentsinPHYS1001.

StudentExperience:FocusgroupsThefollowingfindingsemergedasaresultofathematicanalysisofthequalitativeresponsesand

contentdiscussedinthefocusgroups.

1. ThewholeprocessStudentsinthefocusgroupsappearabletorecognisethebenefitsoftheprocess.Somestudentsare

notnecessarilyabletoarticulatewhytheyfeltitimprovedtheirlearning,whileothersweremore

reflectiveintheirunderstanding.Thereflectivestudentsweremoreabletoconnecttheprocesswith

improvedlearningoutcomes,asapparentinthefollowingextractsfromthediscussions.

IwrotelecturesandactivitiesfirstbecauseitislikeactivelearningIguessaskingyougetspeoplethinking,lecturersaskingquestionsandinsteadofjustsittingthereyouareinterested,inotherlecturesyoujustsitthereandthereisnointeraction…Yeh…andsomeonethrowsinformationatyou.Andthenthereisnotimetoactuallythinkaboutit.Andgrouppartneractivitiestheycanbeexplainingitandiftheyknowitwillhelptoexplainitinadifferentway.Andthenthepersonlearningwillhaveanotherperson’sviewonitandthatalwayshelps.SameforthelecturesandIdidprereadingbecausewithouttheprereadingyou’renormallyabitlostandthenthathelpsinthelectures.

Ithinkithelpsuslearnfasterbecausewedon’tspendthelecturetimelookingthroughthebookwehavealreadydonethat.Andwerefineourunderstandinginthelectureandinsteadofjusttryingtounderstandittostartwith,itsavestimeandwegetmoreworkdoneinashorteramountoftime.Thereadingquizzeshelpustofocusthelearningfromtheprereadingbecausesometimethetextisabitwordy.

Iagreewithyouthatitiscalledactivelearningbecauseweareallparticipatinginthecourse,wearenotpassivelylearning,sittingthereinlecturesjustdaydreamingweareactuallysittingthereinthelectureanddoingclickingquestionsandtalkingtopeopleandIthinkthatisgoodbecauseyoucanironoutanymisconceptions.

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 12

Ithinkitisbetterthanbeingtraditionallytaughtbecauseinthesetypesofcoursesweareactivelylearningwearenotpassivelytaughtwedotheprereadingwedotheproblemsheetswegothelecturesweusetheclickersandwearenotsittingbackandobserving,weareinsidenotoutside.

2. IndividualelementsInthepreviousstudyconductedin2011,emphasiswasplacedoninvestigatingtheprocessofpre-

readingandonlinequizzesasamethodofengagingstudentsinthelearningprocess.Further

investigationwasconsideredwarrantedinordertodeterminetheeffectofotheraspectsofthe

processonstudents.Inthefocusgroupsconductedin2012,allelementsofthelearningprocess

wereinvestigated,asillustratedinthefocusgroupguide(Appendix4).Students’commentsforeach

oftheelementsareexploredbelow.

Pre-readingandonlinequizTheparticipantsofthefocusgroupscontinuedtorecognisethevalueoftheprereadingtasksand

on-linequiz.Havingapreviousencounterwiththeconceptsbeforedealingwiththeminthelecture

orexaminationallowedthestudentstoscaffoldtheirlearning.Theprereadingandquizassiststhe

studentstodevelopconceptualframeworkswhichinturnmakelearningandproblemsolvinginthe

lectureeasierandmoreaccessible.Thefeedbackfromtheonlinequizalsoprovidesalevelof

responsethatwasmoreimmediateratherthanwaitingfortheresultsfromanassignmentor

laboratoryreport.

Theexplanationandthereadingithelpedmealot…ImeanIstartedoffwithskimming….becausetheworkloadwassohugebeforethesemesterbreakIjustskimmeditandthenjustreadingthesummariesandthendoingthequestionsandIfailedthemidsemesterandthenIstartedlikereadingitmoreindepthandthenwritingnotesasIwentalong.IfoundthatIactuallyneededtodothatbecauseitdoesn’tstickotherwisehoweveritisalwaysnicetohavesomethingtogobacktoandyoucanputitinyourownwords.

Soinothersubjectswherethereisnopre-readingyoufindyourselftryingtoreadhalfatextbookforexamprepbecauseyouhaven’tlookedatitbefore.

Constantinteractionwiththetextbookmeansyouarenotconfrontedwithitwhenyougotodotheexam.

Havingthetextbookintheexamisonlyusefulifyouhavealreadyengagedwithitduringthesemester.Youcan’tfakeit.

Sotherearetwotypesoffeedbackonewhereyouaredoingquizzesandyoucanmarkstraightandtheotherwhereyourelyonthelecturertomarkandthenyougetfeedbackinthelectures.

Sothefeedbackallowsthelecturerstocorrectanymisconceptionsyoumighthaveacrossthecohort.Itkindofhelps…

Theygooverthereadinganditusuallytakesaboutfiveminutestogothroughandthentheygoontothelecture.Theyusuallyidentifythemainconcernsinthelecture.Michaelusuallyaddressestheseandthenhewillputupthebestanswersorananswerthatshouldbeansweredinanotherway.

Theparticipantsagainidentifiedtheissueoftheamountoftimerequiredtocompletethepre-

readingandonlinequizactivitiesinbothfocusgroups.Whiletheyrecognisedthebenefitsofthe

practice,theyalsoindicatedthattheworkrequiredtocompletetheactivitieswasmorethanin

othercourses.

Sometimesithardtogetorganisedaroundtheworkload.TheprereadingisusefulwhenI’mlike…whenyoudoitproperlyhoweveritisnotsoeasywhenthereareotherformsofassessmentaswell.Sointhebeginningitwasfinewhenwedidnothavemuchworkfromtheothercoursesaswell.Itwaseasytofindthetimetositdownandgothroughthereadingandtakenotesandtoworkinthephysicsroomandtaketimetodoit…notsomuchnow.

ItisprettyintenseinPhysics.

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 13

Prereadingexercisesaregreatbutthereisfartoomuch.Itisreallygreathowtheygiveustheclickerquestionsandalltheexampleshowevertheydon’tgiveusmuchdetailandtheydon’tactuallyteachthestuff(inthelecture).AndIgetthattheywantustogetthatfromthereadinghoweverIfinditsodifficulttogetthroughthereading.

ClickerquestionsandlectureactivitiesStudentscontinuetorecognisethevalueoftheinclusionofclickerquestionsandinteractive

activitiesinthelecture.Studentsalsofeltmoreconfidentinlectures,havingalreadyengagedwith

thelearningmaterial,whenrespondingtoquestionsbothwithinasmallgrouporwhenaskeddirect

questionsbythelecturer.Theuseofinclassactivitiesandclickerquestionspreparedthestudents

for“expertexplanations”andthedevelopmentofproblemsolvingskills.

TheselectureactivitieswithclickeractivitiesaregreatwhenIturnupbecauseitforcesyoutothinkaboutitandsometimesIcantuneoutduringotherlecturesbutinPhysicsIstayontrack.

Lectureactivitieswiththeclickerstheyarereallygoodbecausethereismoreexamplesandthenyougetachancetotryitthenyougetachancetoseeifyourattemptwascorrectorincorrectandthenthereisanexplanationbutasforlecturesthereisnotmuchcontentcoveredbecauseliketheybarelyteachyouitsoitisallfromthereadingactually.Participationinlectureslikewhenthereisanevenmixonthevotesandyougetpeopletryingtoexplainit.GrouppartneractivitiesareOKbutwhatifyougetstuckwithsomebodywhoisreallydumbordoesn’tevencarethenitisjustfrustrating.

Ithinktheclickerquestionsareprettyexcellentandtheycanhelpyouworkoutwhatisgoingon.Likeifyoumisunderstandsomethingthenyoutalktopeoplearoundyouandtheyhaveadifferentunderstandingofwhatishappeningandthenbecausepeopledothingsdifferentlyandtheymighthaveaclearerwaytodosomething.Thenyoutalktothemanditmakesitbetter.I’mnotsureifyouwouldcallitteamworkhoweveritisbetterthandoingitalone.

Soifyouhaveaquestionandthenyoufindoutsomeoneelsehasthesamequestionyoufeellikeyouarenotalone.

Ithinktheclickerquestionsareprettyexcellentandtheycanhelpyouworkoutwhatisgoingon.Likeifyoumisunderstandsomethingthenyoutalktopeoplearoundyouandtheyhaveadifferentunderstandingofwhatishappeningandthenbecausepeopledothingsdifferentlyandtheymighthaveaclearerwaytodosomething.Thenyoutalktothemanditmakesitbetter.I’mnotsureifyouwouldcallitteamworkhoweveritisbetterthandoingitalone.

Itshouldberecognisedthatnotallstudentsrespondedinapositivewaytotheseteachingmethods.

Somestudentspreferrednottoengagewithanyoneelseintheclassandfelt“pressured”whenthe

lecturerrequestedthattheypairupwithsomeoneelse.Otherssuggestedthatchoosingtheright

partnerwhowasalsoengagedintheprocesswasimportanttoensureappropriatelearning

outcomes.Therewerealsoindicationsfromsomestudentsthatclickerswerenotbeingfullyutilised

bysomeoftheirpeersandthatbecauseparticipationwastheonlyrequirementsomestudentswere

justclickingarandomresponse.Onestudentsuggestedthatmarksshouldbegivenforcorrect

answersnotjustanyresponsetotheproblem.

SoinlecturestherearepeoplewhoaretalkingandontheInternetanddoingotherthingsandthentheyrealisethereisaclickerquestionandtheyclickinsomethingandthentheygobacktodoingsomethingotherthanPhysics.Ithinkthereshouldbeanextrabonusforgettingthequestionright.

IthinkgroupandpartneractivitiesarethebestthingsIfindforlearningandreinforcingknowledgebutnotthatyouareforcedintobeingingroupsbecauseyoumightgetapoorgroupanditisnotbeneficialanditcanturnyouawayfromthecourse.Butifyoudogetagroupthatworksreallyitisgreat.SoIhaveabunchoffriendsandwegoandhavelunchafterthelectureandwebasicallytalkaboutthephysicsproblemswejusthadoraboutthepraccomingupanditjustreinforceseverythinganditjuststicksbetter…Peoplesometimesanswerbeforethequestionisevenupmeaningtheyarenotpayinganyattention.

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 14

Anotheridentifiedissuewasiftheactivelearningprocesswasnotadheredtoinitsentiretythen

thereweretimeswhenshortcutsweremade.Forexamplewhenstudentsindicatedthattheydid

notknowtheanswerviatheclickersthenitbecameeasierforthelecturertojusttellthemrather

thanletthemproblemsolveindependentlyorwithinagroup.Thefocusgroupparticipants

suggestedthistobeanissuewithclasssizeandpossiblytheabilitytomanagelargegroups.Further

probingwasnotabletoelicitamoredetailedanswer.

Ithinkthatsomehowwhathappensinthelecturetheatre…whathappensisthatthelecturer…ifpeopledon’tgetittheysayifwellyoudon’tknowtheanswerthenIwilljusttellyouwhatitis.Anditisprobablyjustharderwiththatmanypeopleinthelecture.

MasteringPhysicsiswhereyoucaneitherdoitornot.Thepre-readingisgreatbutuntilIgetintothelectureIdon’tstartunderstandingituntiltheyexplainitandthenIunderstandwhatwearefocussingon,thepartnerandgroupparticipationactivitiesaregood.

I’menjoyingthemall,howeverPhysicsIthinkI’mlearningmuchfasterbecauseyouhavetodotheprereadingbecauseitcountsasmarksandyouhavetoyouareencouragedtodothatandthewholegoingtolecturesandyoumakesureyougettherebecauseyoumissoutonmarkswhereastheothersyoumissonehereandthere.TheprereadingitdoesforceyoutodoitisandtheteachingstyleisdefinitivelyalotbetterthanthebiologyandchemistrybecauseeveryonehasdonetheprereadingandyoudohavethewholeinteractivityofthelecturebutthenagaintheclassismuchsmallerthanBioandChem.HowmuchsmallerIthinkmaybe1500kidsinChemistryandI’mnotsureabouttheothers.

Feedbackforteacherandstudent-theclosingoftheloopOneofthestrengthsofthelearningprocessistheconstantfeedbackintothelearningloopbyall

stakeholders.Throughthereadingandonlinequizthestudentsareabletoindicatewhatareasthey

arehavingdifficultywithinthecourseprogram.Thelecturersareabletorespondtotheseissuesvia

emailorinthelecturesituation.Astandardpartoftheteachingprocessisasegmentatthe

beginningofeachlecturewherethelecturerhasidentifiedaproblemthatmanyoftheclassare

experiencing.Theimmediacyoftheresponseisthekeyhere;students,whomayhaveexperienced

somedifficultystudyingindependently,knowthattheirquestionwillbeansweredinthebeginning

ofthelecture.Thecontinuousfeedbackfromtheonlinequizandthroughthelectureactivitiesis

alsoaverypowerfultoolforthelecturer.Itallowsthelecturertotailorthelearningprogramto

meettheneedsofthestudents.

IhavelectureactivitiesandclickersonthetopofthelistIjustfinditgoodinclassitisgoodtogetfeedbackanditisimmediatefeedbackforexamplelikeifIgetitwrongthenwellnotsomuchifIgetinwrongbutiftheclassgetsitwrongthentheyexplainitorwehavetotalktothepeoplearoundussoeitherexplainingittosomeoneorhavingthemexplainittomehelpsitquiteabitanditincreasesmyunderstandingofwhatisgoingon.Whatwasreallygoodformeheactuallyemailedmebackandgavetheresponsestothequestionsitisreallygoodtosee…thatpersonalfeedback.ThelecturerhasemailedandtoldmemyresponsesarenotgoodenoughandthatIwouldneedtowriteabitmoredetailinthefuturetoreceiveamark.Youneedtobemorespecificabouttheformulashesaid.Thelecturergoesthroughthethingsthatpeoplearehavingproblemswithatthebeginningofthelecture.Sometimesitseemslikeit’snotthatdifficultuntilafterwardsafterthelectureIwouldliketoasksomequestions.

ObservationsThefollowingsectionisinformedbythestructureoftheObservationGuide(seeAppendix6)and

outlinestheactivelearningprocessesasobservedinthelecturetheatresofPHYS1001and

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 15

PHYS1002inSemesterOne(seePhaseThree-DataCollectionProcess)intableform.Thetable

identifiesprocessesthatseemedtobefollowingamoretraditionallecturestyleinthefirstinstance

andprocessesthatseemtoexemplifyactivelearninginthesecondinstance:

Moretraditionallecturestyle Activelearninglecturestyle

StudentInformation

PowerPoint, learning outcomes for the

day,feedbackfrompreviouslecture

Structured PowerPoint including details

about learning outcomes, feedback from

thequiz,recognitionofissuesexperienced

bystudents,studentquote,connectionto

outside learning, references to pre

readingandacartoon.

Lectureimplementation-resources

Radiomicrophone,PowerPoint

presentation,visualiser,clickers

Reallifepropsandexamples,multimedia

examples

Lectureimplementation-Process

Onscreenenvironmentvisualiser

Seriesofproblemswithsomeexplanation

inbetweentheproblemsolving.

Somedefinitionsandthentoaconcept

problemthatthestudentsweredirected

todiscusswitheachotheriftheydidnot

understanditandthentovote.

Everyonehadpolledtheanswerthen

workedthroughtheproblemonthe

visualiserprovidingalongexplanationand

thenthestudentsre-polled.Some

discussionfollowed.

Introductionofideausingapractical

examplebothonthescreenandusing

props.

Conceptquestionposed.

Studentworkingontheproblem

individuallyandvoting.

Resultswerepolledbythelecturer

howevernotmadepublic.

Discussedtheresultsbriefly;students

weretheninstructedtotalkwithpeers.

Thelecturetheatrelightswentupwhen

studentsweresupposedtotalkand

dimmedslightlywhenthelecturerhadthe

floor.Thelecturermadeuseofthis

momentoffreedomandmovedaround

thetheatretoengagewithsinglestudents

andsmallgroups.

Afteraprescribedamountoftimethe

studentsre-polledandtheresultswere

discussedasawholegroup.Atdifferent

pointsofthelecturethelecturerasked

questionsofthewholegroupand

receivedresponsesfromavarietyof

students.

Throughoutthelecturetherewere

referencestothereadingthatthe

studentshadcarriedoutpriortothe

lecture.

Learningopportunities

• listening and taking notes while

lecturerexplainsproblem

• answering questions posed by

thelecturertothewholeclass

• answering questions using the

clickersafterpeerdiscussion

• workingwithapartnerorasmall

group.

• answering questions posed by the

lecturertothewholeclass

• answering questions using the

clickers,individually

• opportunity to work independently

first

• abletoworkwithapartnerorasmall

groupand

• some opportunities to interact with

thelectureronetoone.

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 16

Studentengagement

Whenaskedtotalkabouttheproblem

studentsrespondedquietlyinmostcases.

Somestudentswhowereactively

engaged,conversingwitheachotherand

pointinganddrawingdiagramsonpaper.

Studentsoftenappearedtobewaitingfor

thefirstpolltorespond,ratherthan

makinguptheirownmindsorworking

throughtheproblem.

Levelofengagementacrosstheclass

highermostofthetime.

Studentsbecamemoreinvolvedwhenthe

lecturerdrewthemintotheproblem

solvingstage.

Studentsworkedontheproblemsas

individuals.

Studentsengagedwiththestudents

aroundthemduringthepeertopeer

discussion

Allthestudentswhohadclickers

respondedtothequestions.

Studentsengagedwiththelecturerasa

wholeclass.

Table5:Similaritiesanddifferencesbetweenactivelearningandotherlecturingprocesses

OverallobservationsThissectionsummariseswhatwasparticularlyeffectiveaboutthelecturesincoveringkeyconcepts,

thequalityofinterpersonalinteractionbetweenthestudentsandlecturersandgeneralobservation.

Theactivelearningnatureofthelectureswasparticularlyeffectiveinensuringthelearning

outcomesofthelecturewereachieved.Thelecturesweresuccessfulbecause:

• Thebeginningofthelecturewassituatedinthecontextoftheprereading,theonlinequiz

andthelearningfortheday.

• Theprereadingandonlinequizwerehighlightedasimportantaspectsoflessonpreparation

andthelecturerrespondedtoquestionsandtheresultsoftheonlinequiz.Theimmediacyof

thisresponsetothereadingandonlinequizisapowerfultoolforstudentengagement.

• Thepreviouslecturewasreferredtoandthelinkswithpreviouslearningmade.

• Thelearningoutcomesoftheday’slecturewerehighlightedandreferencewasmadeto

howtheconceptsmightworkintheworld.

• Theslideswereclear,informativeandengaging.

• Thestudentsunderstoodtheirroleinthelectureprocessandrespondedtothevisualand

verbalcuesasprovidedbythelecturer.Theydiscussedproblemswhenrequired,responded

toquestionswhenaskedandgenerallylistenedtothelecturerwhenappropriate.Itis

importanttonotethattherewastimeallowedfortheindividualtoworkonaquestionprior

topollinganddiscussionwithpeers.

• Theuseofthetablettocontrolthescreenandaddtoslidesallowedthelecturertointeract

withthestudentsmorereadily;theconstant“eyeball”contactisastrengthofthismodeof

communication.

• Theradiomikeallowedthelecturerthefreedomtomovearoundthelecturetheatregiving

himcontactwithindividualsandsmallgroups.

• Theuseofsimpleprops(abikepump)groundedthelectureintherealworldallowingthe

studentstoscaffoldtheirlearningandtobuildaframeworkinwhichtodevelopthe

necessaryconcepts.

• Intheclosingstagesofthelecture,referencewasmadetofuturelecturesandtheir

relevancetotheforthcomingexaminations.

• Therewasaveryclearendtothelecture.

Otherbehavioursobservedduringthelectureswerenotasclearlypartoftheactivelearningprocess

astheobserversunderstoodit.Somedifferencesnotedbytheobserversincluded:

• Whilethelecturerreferredtoapreviouslectureandcompletedaproblemfromthatsection

ofwork,therewaslessconnectionmadewiththelearningthathadtakenplacebefore.

EvaluationUnit,TEDI,UniversityofQueensland|KeyFindings 17

• Therewaslessemphasisbeingplacedonthepre-readingandonlinequiz.Studentsare

requiredtocompletetheprereadingandonlinequizonthebasisthatthelecturewill

developtheideaandprovidethemwithmoreinsight.Withouttheemphasis,thepre

readingmaybecomedevaluedinthestudents’eyes.Mentionwasmadeofithowevernot

tothesameextentandnotvisuallyonaslide.

• Ratherthantheclickerquestionbeingposedafterthedevelopmentofaconceptitwas

oftenusedasthecatalyst.Studentsweregiventhequestion,askedtopolland/or“discussit

withtheirpeersiftheywantedto”andthenthelecturerwouldgothroughtheproblemon

thevisualiser.Therewasnottimeallowedforindividualcontemplationoraspecified

“discusswithyourneighbour”time.Asaconsequencetherewaslessanimationand

engagementinthelectureaudienceduringthistime.

• Theuseofthevisualiserastheprimarymodeofcommunicationapartthetraditionallecture

styleapproachreducedtheamountoffacetofacecontactthatthestudentshadwiththe

lecturer.Withoutthefocusofthelecturer,studentsbecameeasilydistractedbypeersand

communicationtechnology.

EvaluationUnit,TEDI,UniversityofQueensland|Conclusion 18

Conclusion

Thisinterimreportsoughttoevaluatetheimpactoftheimplementationofanactivelearning

processthatisbeingusedasthebasisforstimulatingstudentlearningandengagementinPHYS1001

in2011andnowinbothPHYS1001andPHYS1002in2012.

Theexpectedaimsandbenefitsofthissysteminclude:

• toencouragestudentstodevelopself-directedlearningskills,

• toencouragestudentstoreflectontheirreadingmaterial,

• toencouragestudentstoreflectontheirlearninginclassandprovidefeedback,and

• toallowteachingstafftoidentifythemostdifficultconceptsforstudents,

• Toincreasestudentengagementintheirlearning

Thereisevidenceinthedatacollectedthatthecontinuedimplementationandrefinementofthe

activelearningprocessdoesimprovelearningoutcomesforstudents.

1. Thenatureoftheactivelearningprocessallowsforstudentstointeractwitheachotherinthelecturetheatre.Thestudentswhoattendedthefocusgroupsessionsweregenerallyverypositiveaboutthe

activelearningprocess.Havingbeenexposedtoanumberofteachingmethodsintheir

shorttimeattheuniversity,theywereclearaboutwhatwasworkingforthem.

2. Thedatacollectedfromthesurveysprovidesalessclearpictureofthebenefitofutilisingactivelearningprocessesinfirstyearphysicscourses.Comparisonsbetweenpreandpostsurveyforbothcohorts’datadoesnotprovideany

clearerunderstandinginchangesinattitudeacrossthesemester.Somechangesare

statisticallysignificant;however,theyarenotconsistentacrossallthedata.Overall,

PHYS1001studentsaremorepositiveacrosstheentiresetofitemsthanPHYS1002students.

Thedifferencesintheresultsbetweenthetwocoursescouldbeforanumberofreasons.

ThemorerecentutilisationoftheactivelearningprocessesinPHYS1002maymean

variationsinhowtheapproachisbeingimplemented.Anecdotally,therearesuggestions

thatstudentsperceivePHYS1002courseasmorecomplexanddifficultandthusrespond

accordingly.Moreinvestigationintothesedifferencesiswarranted;adeeperunderstanding

ofthetwocoursesandhowtheyarepresentedmayprovideanswers.

3. Onecannotidentifyasingleaspectoftheapproachthatworksaboveallothersinthelecture;itistheintegrationofallthepracticesintoacoherentprocessthatmakesitsuchapowerfulteachingandlearningintervention.Moststudentsareengagedfromthefirstmomentbyaprocessthatutilises;theirpreviously

acquiredknowledgeandknowledgeapplication(theprereadingandquizstage),the

challengeofproblemsolving(clickerquestions),theoptionofthegroupthink(sharingwith

fellowstudents)andthegeneralinteractionsbetweenstudentandlecturer.

4. Observationsoflecturessuggestthatmoreexperiencedpractitionersofactivelearningteachingarebetterabletoengagethestudents.Lessexperiencedpractitionersarestillworkingtocapturetheessenceoftheactivelearning

processandtherebyfallsomewhereinthemiddle,notquiteabletoprovidethefullbenefit

ofa“traditionallecture”andnotyetabletoachievewhatbestpracticeintheactivelearning

processdelivers.

Inconclusion,thedatagatheredsuggeststhattheactivelearningteachingframeworkisavaluable

oneandthat,whentheintegratedinstructionalpracticesareadheredto,theactivelearningprocess

EvaluationUnit,TEDI,UniversityofQueensland|Conclusion 19

hasthepotentialtoreducetheatmosphereinafulllecturetheatretoonethatresemblesthatofa

smalltutorialwithallitsinherentintimacy,thepromiseofstudentengagementandimproved

learningoutcomes.

AreasforfurtherinvestigationThefindingsfromthesurveys,observationsandfocusgroupidentifiedthefollowingareasthatmay

needsomefurtherinvestigation.

1. Oneoftheissuesraisedbystudentsinthepreviousphaseoftheevaluationwastheamountoftimerequiredtocompletetheprereadingandquizsectionofthework.Thisviewwasidentifiedin2011andcontinuedtobeathemein2012.Whilethestudents

agreedthatthepre-readingwasanintegralpartoftheprocess,theyalsofounditdifficultto

finishsometimes,particularlythestudentswhowereenrolledinsomeofthemoreintensive

programs.Furtherinvestigationcouldprovidedataastowhethertheissuewasoneoftime

managementonthestudents’behalforifindeedthereadingdemandswerebeyondthe

timeprescribedbyuniversitypolicy.

2. Therewasasuggestioninthefocusgroupsthatthisaspectofbeing“stronglyencouraged”or“madeto”tositwithotherstudentswasnotalwayswelcomeandthattheforcednatureofthegroupdiscussionengenderednegativefeelsforsomerespondents.However,somestudentswereverypositiveaboutthisaspectofthelecture,tothepoint

wheretheycarriedthediscussionsoutofthelecturehallandintowhatappearedtobean

informal‘communityofpractice’(LaveandWenger,1991).Theseextremesinresponse

couldbeinterestingtoinvestigategiventhepotentiallypowerfuleffectspeerinteractioncan

haveonstudentlearning(Webb,1989).

3. Thereisaneedtosupportthedisseminationandadoptionoftheteachingpracticesinothercourses.Aswiththeintroductionofanynewapproach,regularmonitoringandreflectionallowsfor

therefiningofthedifferentaspectsoftheapproach.Iftheactivelearningapproachistobe

embeddedinfirstyearphysicscourses,thensomestandardisationoftheprocessacross

lecturersandcoursescouldbeuseful.Theproductionofabrief“howto”guideandsome

peerobservationofwhatisdeemedbestpracticemightbeadvantageous.

4. Aninstructionalpracticethatresultsinimprovedlearningoutcomesinonecoursemaynotnecessarilyprovidethesameoutcomesinasecondsituationformanyreasons.Furtherinvestigationintothecurriculumdesign,studentpopulationandpedagogyof

PHYS1001andPHYS1002mayprovideinsightintohowactivelearningprinciplesmaybest

supportthedeliveryofthesetwocourses.

Finally,thismustbeconsideredaninterimreport,bothprimaryandsecondarydataisstill

beinganalysedandwillbeincludedinasummaryreportatalaterdate.

EvaluationUnit,TEDI,UniversityofQueensland|REFERENCES 20

REFERENCES

Deslauriers,L,Schelew,E.,Wieman,C.(2011)‘ImprovedLearninginaLarge-EnrolmentPhysics

Class’,Science332,862

Keller,J.M.(1987).IMMS:Instructionalmaterialmotivationsurvey,FloridaStateUniversity

Keller,J.M.(1987).‘Strategiesforstimulatingthemotivationtolearn’PerformanceandInstruction,26,No.8,pp362-632

Lave,J.,&Wenger,E.(1991).Situatedlearning:Legitimateperipheralparticipation.Cambridge

UniversityPress.

Laws,P.,Sokoloff,D.andThornton,R.(1999).‘PromotingActiveLearningUsingtheResultsofPhysic

EducationResearch’,UniServeScienceNews,Vol13

Prince,M.(2004).‘DoesActiveLearningWork?AReviewoftheResearch’JournalofEngineeringEducation93(3),pp223-231

Webb,N.M.(1989).Peerinteractionandlearninginsmallgroups.InternationalJournalofEducationalResearch,13(1),21-39.

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 21

Appendices

APPENDIX1:PRE-SURVEY

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 22

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 23

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 24

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 25

APPENDIX2:Post–Survey

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 26

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 27

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 28

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 29

APPENDIX3:FOCUSGROUPDISCUSSIONGUIDE

Engaging first year students in classroom discussion of high-level concepts: pre-reading, reflection and personalised feedback.

1. Introduction 1a. Welcomeandintroductionofmoderator1b. ObjectiveTheobjectiveofthisresearchistoinvestigatetheeffectivenessofanalternativemethodofteaching

introducedinMechanics&ThermalPhysicsI(PHYS1001)andElectromagnetismandModernPhysics

(PHYS1002).Thisinvolvesusinglecturetimeforin-classdiscussionbasedonpre-reading.

1c. ProcessWewillbeconductinga1.5hourfocusgroupwith5–7studentswhoareenrolledon

PHYS1001/1002insemester1,2012.

Discussionwillfocuson

• Youropinionofthetextbookandpre-readingactivities

• Yourconfidencelevelinunderstandingphysics

• YouropinionsonthelearningactivitiessuchasLabs,tutorialsandweeklyproblemsheetsin

PHYS1001/1002

• Yourexperiencesoflearningthroughdoing

Pizzaandsoftdrinksareavailable.Giftcardstothevalueof$30willbeprovidedinappreciationof

yourinvolvement.

Yourpermissionhasbeensoughttoaudiotapetheproceedingstoenabletheresearchertorevisit

theinformationtoensureaccurateanalysis.

Anyinformationthatisobtainedinconnectionwiththisstudyandthatcouldbeidentifiedas

relatingtoyouwillremainconfidentialandisdisclosedonlywithyourpermission.

Remember,youarefreetowithdrawyourconsentandtodiscontinueparticipationatanytime

withoutprejudice.Informationprovidedisfedbacktotheteachingteaminreportthatwillinclude

anoverviewofwhathasbeensaid,inade-identifiedformat.Yourlecturerswillneverbeableto

attributeyourresponsetoyouandyouarewelcometoaskforacopyofthereportaswell.

1d. Rules–honestopinions,oneatatime,lotsofgroundtocover,OKtohavedifferentopinions

1e. Introductionofparticipants-name,course,careeraspiration

Questions

1. OVERALLIMPRESSIONS

‘I’minterestedfirstofftofindoutabitaboutyourperceptionaboutPHYS1001/1002…’

PROMPTS:

• Howwouldyoudescribethiscourse?

• Whatwerethe“learningobjectives”ofthiscourse,meaningwhatwereyoumeanttolearn,

inthebigpicturesense,fromthiscourse?

TASK

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 30

• Whatgradedoyouthinkthatyouwillgetforthiscourseattheendofthesemester?Write

downwhatyouthinkonthepostitinfrontofyou–don’tletanyonesee,folditupandhand

themintome.

• We’llhavealookattheminafewminutes

2. THEPRE-READINGTASKS

WhatI’dliketoturntonowisthetaskofthepre-reading.I’mnotsurewhatthatactuallyentails.

Cansomeonepleasedescribetomewhatthepre-readingandquizisactuallyabout?

• Doyoureallydoitbeforehand?Howlongbeforethelecture?Whathappensifyoudon’tdo

it?

• Howmuchtimedoesthepre-readingtaskandquiz?

• Isthattimewellspent?

• Sowhatisitthatyouactuallydo?

Let’stalkabouthowyougoaboutthetaskofdoingthepre-reading.TASK

I’vegotacopyofoneofthepre-readingtasksandquizzeshere.Canyoudescribetomehowyoudo

thisthing?

Prompts:

• Doyounotedownthepointsthatyouhaveproblemswith?

• Doyouwritedownthethingsthatyouhavetroublewith?Whogoesthoroughthemwith

you?Doesithelp?

• Whatdoyouthinkaboutthematerial–doesitreallyhelpyoumakesenseofcomplex

things?

3. COURSEDESIGN

I’dliketolookatthevariousactivitiesthatyoudointhecourse

TASK:

Firstly,I’dlikeyoutoarrangethevariouselementsofyourcoursethatwarewrittendown

onthesecardsinorderofthecontributionthattheymaketohelpingyouunderstand

physics.Puttheonethathelpsthemostupthetopandranktherestdowntowhathelps

theleast

Let’stalkabouthowthepre-readingfitsinwiththerestofthethingsthatyoudo

PROMPT:

• Howdoesthereadingassistyouin:

o Lectureactivities

o Groupactivities

o Tutorialworksheets

o Weeklyproblemsheets

o Labs

o Pre-readingexercises

• Doyougotothelectures?Why/whynot?

• Whathelpsyou‘do”science?

4. ASSESSMENTANDGRADING

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 31

‘I’dliketohearaboutyourassessmentitemsnowinPHYS1001/1002…’

• WhatformsofassessmentareworkingwellforyouinPHYS1001/1002?Why?

• Doyouthinkit’sfairthatthepre-readingquizisgraded?

• Let’slookatthegradesyouwrotedownonthepost-its.Iseethatyourscoresare....

o Doyouthinkthatyouwouldbeachievingthiswithoutthepre-readingtasks?

o Whatcontributiontothegradedoyouthinkthere-readingtaskmakes?

Conclusion

Weareprettymuchdone;Ireallyappreciateyourcandidfeedback.Beforewegettothegiftvouchers,isthereanythingyouwouldliketosayaboutPHYS1001/1002?Anythingwedidnotcoverinthissessionthatyoufeelwouldbenefitustohearabout?

• Thankyouforyourparticipation

• Pleasesignforandcollectyourdebitcard

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 32

APPENDIX4:STUDENTFOCUSGROUPCONSENTFORM

Project title: Engaging first year students in classroom discussion of high-level concepts: pre-reading, reflection and personalised feedback.

Youareinvitedtoparticipateinresearchinvestigatingtheeffectivenessofanalternativemethodof

teachingintroducedinMechanics&ThermalPhysicsI(PHYS1001)andElectromagnetismand

ModernPhysics(1002).Thisinvolvesusinglecturetimeforin-classdiscussionbasedonpre-reading.

TheresearchisbeingundertakenbyKarenSheppardandDeanneGannawayfromTEDIonbehalfof

M.Drinkwater,M.Davis,W.Bowen,J.Corney,T.McIntyre&M.WegenerfromtheSchoolof

MathematicsandPhysicsatTheUniversityofQueensland

Ifyoudecidetoparticipate,youwilltakepartinoneofaseriesofone-and-a-halfhourfocusgroups

withfivetosevenPHYS1001/1002UniversityofQueenslandstudents.

Studentfocusgroupswillexplorethefollowing:

• Youropinionofthetextbookandpre-readingactivities

• Yourconfidencelevelinunderstandingphysics

• YouropinionsonthelearningactivitiessuchasLabs,tutorialsandweeklyproblemsheetsin

PHYS1001/1002

• Yourexperiencesoflearningthroughdoing

Pizzawillbeprovidedduringthefocusgroup.Yourpermissionwillbesoughttoaudiotapethe

proceedingstoenabletheresearchertorevisittheinformationtoensureaccurateanalysis.A$30

DebitGiftCardwillbeprovidedtoparticipantstoacknowledgeyourparticipation.

Werespectyourrighttoprivacy.OurPrivacyPolicyanditsprocessesensurethatanyinformation

thatisobtainedinconnectionwiththisstudyandthatcouldbeidentifiedasrelatingtoyouwill

remainconfidentialandwillbedisclosedonlywithyourpermission.Yourindividualresponseswill

neverberevealedtoyourlecturers.

Ifyoudecidetoparticipate,youarefreetowithdrawyourconsentandtodiscontinueparticipation

atanytimewithoutprejudice.

Ifyouhaveanyquestions,pleasefeelfreetoaskus.Ifyouhaveanyadditionalquestionslater,

KarenSheppard,ResearchOfficer,EvaluationServicesUnit,TEDI(email:[email protected])

willbehappytoanswerthem.

I have read the information above, and agree to participate in this study. Name of Participant: ……………………………………………… Signature of Participant: ……………………………………….… Date: ………………………

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 33

APPENDIX5:FURTHERTESTINGTable1:MeansandstandarddeviationsofPHYS1001studentspre-survey(n=96)andPHYS1002studentspre-survey(n=75)

wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.

PHYS1001

PHYS1002

Significance

LearningActivityfeedback Pre

Mean

(SD)

N=96

Pre

Mean

(SD)

N=75

Completingonlineproblems(MasteringPhysics) 3.95

(1.09)

3.27

(1.15)

<0.001

Participatingintutorials/conceptlabs 2.97

(1.45)

3.50

(1.17)

0.013

Completingpracticallabs 2.21

(1.72)

3.32

(1.23)

<0.001

Theuseof“clickers”duringthelecture 3.73

(1.33)

3.66

(1.07)

Discussingquestionswiththepersonnexttomeduringthelecture 3.74

(1.27)

3.27

(1.24)

.018

Discussingquestionsasawholeclassduringthelecture. 3.79

(1.12)

3.41

(1.20)

.046

Doingtheassignedreadingtasksandonlinequiz. 3.60

(1.09)

3.36

(1.24)

AttendingPASSclasses. 2.46

(1.74)

2.23

(1.53)

Table2:MeansandstandarddeviationsofPHYS1001studentspost-survey(n=76)andPHYS1002studentspost-survey

(n=53)wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.

PHYS10

01

PHYS10

02

LearningActivityfeedback Post

Mean

(SD)

N=76

Post

Mean

(SD)

N=53

Signific

ance

Completingonlineproblems(MasteringPhysics) 3.99

(1.25)

3.17

(1.13)

<0.001

Participatingintutorials/conceptlabs 3.23

(1.26)

3.34

(1.28)

Completingpracticallabs 3.26

(1.24)

3.30

(1.22)

Theuseof“clickers”duringthelecture 3.62

(1.36)

3.33

(1.21)

Discussingquestionswiththepersonnexttomeduringthelecture 3.65

(1.40)

3.13

(1.33)

0.022

Discussingquestionsasawholeclassduringthelecture. 3.72

(1.29)

3.25

(1.28)

0.046

Doingtheassignedreadingtasksandonlinequiz. 3.56

(1.01)

3.46

(1.21)

AttendingPASSclasses. 2.08

(1.63)

2.44

(1.67)

Table3:MeansandstandarddeviationsofPHYS1001studentspre-survey(n=96)andPHYS1002studentspre-survey(n=75)

wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.

PHYS1001

PHYS1002

GeneralPhysicsfeedback Pre

Mean

(SD)

N=96

Pre

Mean

(SD)

N=75

Signific

ance

Iamenthusiasticaboutphysics 4.04

(.96)

3.35

(1.09)

<0.001

Iaminterestedintakingextraphysicsclasses 3.82

(1.18)

2.84

(1.08)

<0.001

IamconfidentIunderstandthecoursematerial 3.69 3.14 <0.001

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 34

(.92) (1.02)

Ineedtomemoriseformulaetogetagoodmarkinthisclass 2.56

(1.05)

3.42

(1.12)

<0.001

Ineedtounderstandconceptsinphysicstogetagoodmarkinthiscourse 4.66

(.71)

4.35

(.82)

0.010

Ihavecompletedalltheassignedreadinginthiscourse 3.65

(1.38)

3.16

(1.32)

0.024

Table2:MeansandstandarddeviationsofPHYS1001studentspost-survey(n=76)andPHYS1002studentspost-survey

(n=53)wherePHYS1001resultswerestatisticallysignificantlydifferentthanPHYS1002results.

PHYS10

01

PHYS10

02

GeneralPhysicsfeedback Post

Mean

(SD)

N=76

Post

Mean

(SD)

N=53

Signific

ance

Iamenthusiasticaboutphysics 3.88

(1.01)

3.33

(1.18)

0.003

Iaminterestedintakingextraphysicsclasses 3.64

(1.27)

2.70

(1.28)

<0.001

IamconfidentIunderstandthecoursematerial 3.47

(.95)

3.11

(.92)

0.030

Ineedtomemoriseformulaetogetagoodmarkinthisclass 2.77

(1.17)

3.26

(1.14)

.018

Ineedtounderstandconceptsinphysicstogetagoodmarkinthiscourse 4.56

(.64)

4.54

(.66)

Ihavecompletedalltheassignedreadinginthiscourse 3.29

(1.34)

3.38

(1.19)

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 35

APPENDIX6:PHYS1001/1002ObservationGuideCourse:Observer:Criteria CommentsBackground

Summariseyourrationaleforchoosingthismethod,howitisplannedtoinformtheevaluation,anditsroleinsupportingothermethods.WhataretheconstraintsandopportunitiesinfluencingyouraccesstotheL&Tcontextforobservations(e.g.timingoftheobservation,typeofparticipantsinvolved,typeofactivitiesthatcanbeobservedetc...)?

Purpose

Summarisetheevaluationquestionsyouhopetoexplorewiththeobservation.

q Whatinformationisgiventostudentsabout:-• how,when,withwhom,todotheactivityetc?• theeducationalaims,objectivesandexpectedlearningoutcomesoftheactivity?• howtheactivityisexpectedtohelptheirlearningforthecoursewhichitformsapart?• howandfromwhomtheycangethelpiftheyneedit?• whatlearningresourcestheycanusetohelpthemdotheactivityandwhere/howtheycanget

accesstothem?• howtheactivityfitsintoanyassessmentforthecourse,andifitisassessed,themarking

criteria?• therelevanceofthelearningbeyondmeetingtheassessmentrequirements,e.g.infuturelife,

practiceetc.?

q Howistheactivityimplemented?• whoistakingpart?• numberofparticipants• natureoftheactivityandteachingapproach• timingandlocationoftheactivity• howtheactivityisorganised• howtimeisusedduringtheactivity• rolesandresponsibilitiesofparticipants

EvaluationUnit,TEDI,UniversityofQueensland|Appendices 36

• decisionsbeingmadebywhomandforwhom• resourcesmadeavailabletostudentse.g.specialequipment,learningresources,software,

virtuallearningenvironmentsetc• helpavailabletostudents• Howaretheclickersbeingused?

q Howaretheparticipantsbehaving?• Howaretheyundertakingtheactivity?• Howarestudentsusinghelpandresources?• Howarestudentsinteractingwiththelearningenvironment?• Dostudentsappearmoremotivated,engaged,orbetterprepared?

q Howaretheparticipantsinteracting?• Istheredialogue?• Isthedialogueconstructiveforlearning?• Whoistalking/listening?• Whatistheirbodylanguage/non-verbalinformation?• Isthereevidenceinthedialoguethatstudentsarelearning?• Howarestudentslearningfromthedialogue?(e.g.staff-student,peer-peerdiscussion,group

inquiry….etc)• Isthereevidenceinthedialoguethatacademicstaff/supportstaffarerespondingtostudents’

learningneeds?• Howisfeedbackbeinggiventostudents?

q Whatistheevidencethatstudentshaveachievedexpectedlearningoutcomes,• fromtheircompletionofactivities,assessment?• fromtheirbehaviouranddialogue?• fromtheirlevelofachievement?

qOthercomments