scalable, scaffolded writing assignments with peer...

Scalable, Scaffolded Writing Assignments with Peer Assessment in an Economic Literacy Course

AviJ.Cohen(Economics,UniversityofTorontoandYorkUniversity)

AndreaWilliams(WritingInstruction,UniversityofToronto)

ABSTRACT

Thispaperdescribesacollaborationbetweenaneconomistandawritingspecialistinthecontextofafaculty-widewritingacrossthecurriculumprogram,WIT(WritingInstructionforTAs).Thecollaborationinvolveddevelopingscaffoldedwritingassignmentsforalarge(800students)introductorycoursewithaneconomicliteracyapproach.Aneconomicliteracyapproachfocusesoncoreconceptsstudentscanusetomakesmartchoicesasconsumers,businesspeople,andinformedcitizens.Informedcitizenswillmakebetterdecisionsabouteconomicissuesthatmattertothemandtheircommunities,andcommunicatethoseviewseffectivelytoothers.Theop-edassignmentgivesstudentspracticedevelopingaconvincingargumentaboutaneconomicissueforgeneralreaders.

Theassignmentisscalablebecauseofitslength(300-500words),detailed,yeteasy-to-applygradingrubrics,andpeerassessmentsoftware.Studentsgiveeachotherdetailedfeedbackthattheyusetorevisetheirinitialdraft,whilethegraduateteachingassistantsgradethefinalversion.

Sincestudentslackexperiencewritingargumentsforgeneralreaders,wedevelopedtwoscaffoldedassignmentstopreparethem.ThefirstisanonlinemoduleonhowtoreadcriticallythatmodelstheprocessofreadinganarticlefromTheEconomist.Studentsthenwritea250-300wordabstractforanarticlefromanapprovedlist.Thesecondonlinemoduleteachesstudentshowtowriteanabstract,emphasizingthetransitionfromwriter-basedtoreader-basedprose,andtheimportanceofrevision.Thelow-stakesabstractassignmentalsohasstudentsdopeerreviewbeforetheyrevisetheirwriting,alongwithasimplerubrictohelpTAsgrademoreconsistently.Afinalmodulehasstudentswritetheirownargumentasanop-edassignment.

Likeanearlierpaper(CohenandSpencer1993)aboutintegratingwritingintotheeconomicscurriculum,thispaperincludesassignmentdetails,processes,andlessonslearnedforinstructorsinterestedinimplementingsimilarassignments.

Cohen&Williams,May2017

1

INTRODUCTION

Writingiswidelyconsideredbothanessentialskillforcollegegraduatesandakeypart

oftheundergraduatecurriculum.TheWritingAcrosstheCurriculum(WAC)movementandits

closecounterpart,WritingintheDisciplines(WID),advocateforcollegeresponsibilitytoteach

studentstowriteintheirchosenmajor,usingwritingasakeytoolforthinkingandlearning

throughwriting-to-learnactivities.TheInternationalNetworkofWACPrograms(INWAC)--

asubgroupofthelargestpostsecondaryorganizationofwritingteachers,theConferenceon

CollegeCompositionandCommunication--in2014issuedaStatementofWACPrinciples

assertingthat“writingshouldbeanintegralpartofthelearningprocessthroughoutastudent’s

education,notmerelyinrequiredwritingcoursesbutacrosstheentirecurriculum.”1Because

waysofknowingandwritingarediscipline-specific,writinginstructionismosteffectivewhen

guidedbythosewithdisciplinaryexpertise.Whilethecentralroleofwritingisobviousin

English,history,andphilosophy,commonlyreferredtoas“writing-intensivedisciplines,”the

roleofwritingislessvisibleinmorequantitativedisciplinessuchaseconomics.Yetwritingisan

essentialskillineconomics,andwritinginstructionandwritingassignments,asCohenand

Spence(1993)demonstrated,canbeintegratedintoeconomicscoursesinwaysthatenhance

ratherthandetractfromthecorecoursecontentandlearningobjectives.

Thispaperreportsonasuccessfulcollaborationbetweenaneconomistandawriting

specialistindevelopingmulti-partreadingandwritingassignmentsinalarge(over500

students),full-year,introductoryeconomicscoursethattakesaneconomicliteracyapproach,

ECO105IntroductiontoEconomicsforNon-Specialists.2Thecoursestructureandmaterials

describedherecouldbeworkinavarietyofcontexts,includinglarge,publicuniversities(like


2

theonewhereweteach)whereclasssizesarelargeandinstructorsmustrelyonacadreof

graduatestudentswithlittletonoexperienceinteachingwritingtorespondtoandevaluate

studentwriting.Wewillshowhowwritingassignmentsandrelatedinstructionalmaterialscan

helpstudentslearnfundamentaleconomicreasoning.

Althoughwebelievethatwriting,becauseitiscloselylinkedtothinking,deservesa

placeinmosteconomicscourses,writingisaparticularlygoodfitforaneconomicliteracy

coursewheretheaimistopreparestudentstobecomeeconomicallyliteratecitizens–

thoughtfulandengagedpeoplecapablenotonlyofunderstandingbasiceconomicconcepts

andargumentsbutwhocanalsoconstructpersuasiveeconomicargumentsoftheirown.Hall

andPodemska-Mikluch(2015)showthebenefitsofop-edwritingassignmentsinencouraging

studentstoengagewithkeyeconomicconcepts.Similarly,theassignmentswepresenthere,

becausetheyinvolvecriticalthinking,reading,andwriting,helpstudentsengagewithkey

economicconceptsappliedtopolicyissues.Theiterativestructureoftheseassignments

requiresstudentstoread,draft,reviseandedit,andthenreceivefeedbackandread,draft,

revise,andeditagain,teachingstudentsfundamentaltransferableskillssuchascriticalreading

andwritingprocess.

Wepresenttheseteachingmaterialssothatinstructorsinterestedinintegratingwriting

intotheireconomicscourseshavepre-existingassignments,rubrics,andrelatedinstructional

activitiesandresourcestoadoptoradapt,ratherthanhavingtoreinventthewheel.The

assignmentandrelatedresourcesincludedherehavebeentestedforsixyearsonover3000

undergraduatesand,aswewillshow,areefficientandscalable.3


3

Effectiveinstructionmustbealignedwithdisciplinary,curricular,andcoursegoalsand

contexts.Wefirstdiscussthecourseobjectivesrelatedtowritingandhowtheassignments,

activities,andresourcesmeetthecourselearninggoalsandfitourparticulardepartmentaland

institutionalcontexts.Efficiencyisnotonlyakeyconceptineconomics,butincreasinglya

centralconcerninhighereducationasinstitutionsareexpectedtoteachmorestudentswith

fewerdollars.Second,weaddresshowtheassignmentsmakeefficientuseoflimitedresources,

particularlyteachingassistant(TA)hours.Thethirdwayourcourseservesasapotentialmodel

isitsscalability:theseassignments,rubrics,andrelatedactivitiesworkinhigh-enrolment

courses(thiscourseenrolsapproximately500studentsannuallyintheface-to-faceversionand

300intheonlineversion).Forthosewhohavetheluxuryofteachingsmallernumbersof

students,theseassignmentscouldbeexpandedtoprovidestudentswithevenmore

opportunitiestoread,write,receivefeedback,andrevise.

INSTITUTIONAL,DEPARTMENTAL,ANDCOURSECONTEXTS

Writingisacomplexcognitiveskill.Teachingstudentstowritewellischallenginginany

institutionalcontext,andevenmoresoataninstitutionsuchasours.TheUniversityofToronto

hasahighpercentageofmultilingualinternationalstudentsaswellasalargecontingentof

multilinguallocalstudents.With71,000undergraduatestudents,manyclassesattheUniversity

ofTorontoenrolhundredsofstudentsofwhichover20%areinternationalstudents,mostof

whomaremultilinguallearners,alongwiththemanyCanadianstudentswhoaremultilingual.

Becauseofthelargeclassessizes,TAsplayakeyroleinteachingandevaluatingwriting.The


4

TAsinthecoursediscussedherearemostlyeconomicsgraduatestudents(bothMAandPhD

students),manyofwhomarethemselves,liketheundergraduatestheyteach,multilingual

learners.4Ourinstitution,likemostCanadianuniversities,hasnorequiredfirst-yearwriting

coursesoifstudentsaretolearntowrite,theymustdosoindisciplinarycourses.Althoughthe

vastmajorityofourundergraduatesdonottakededicatedwritingcourses,theyhaveaccessto

writingcentreswheretheyobtainhelponwritingassignmentsinone-on-oneappointments

fromhighlyskilledinstructorsandgroupworkshops.5Ourassignmentsaredesignedtoutilize

theseexistinginstitutionalresources.Weencouragestudentstobookappointmentsatoneof

thecampuswritingcentreswheretheycangetfeedbackonadraftoftheirwritingassignment

inaone-on-oneappointment.TheeconomicsTAsdonotprovideguidanceorfeedbackon

assignmentsbeyondgradingthefinalsubmissions.

TheDepartmentofEconomicsispartoftheFacultyofArtsandSciences.6The

departmentalcontextforECO105issimilartotheinstitutionalcontextexceptthatEconomics

hasanevenhigherpercentageofmultilinguallearnersandalanguageotherthanEnglishas

theirfirstlanguagethantherestoftheFacultyanduniversity.7Economicshasparticipatedina

Faculty-wideWACinitiativecalledWIT(WritingInstructionforTAs)since2012andnowhasten

participatingcourses.Participationinvolvesincludingatleastsomewritinginacourseand

consultingwiththeWITCoordinatorandthetwograduatestudentswhoworkwiththe

CoordinatorasLeadWritingTAs(WITconsultants)ondesigningassignments,rubrics,and

tutorialactivities.ParticipatingintheWITprogramgivesinstructorsguidanceoncourseand

assignmentdesign,aswellasadditionalhoursforcourseTAstoreceivetraininginwriting

instruction,particularlyhowtorespondtoandevaluatestudentwriting.8TheLeadWritingTAs


5

aretrainedandmentoredbytheWITCoordinator(AndreaWilliams)andinturntrainthe

courseTAsinprovidingeffectiveandefficientfeedbacktostudentsontheirwriting.

AviCohenhastaughttheECO105coursesince2011.Asanintroductoryeconomics

coursewithaneconomicliteracyapproach,thecourseexplicitly“introducesmicroeconomics

andmacroeconomicsasthebasisformakingsmartchoicesinlifeasconsumers,

businesspeople,investorsandinformedcitizensjudginggovernmentpolicies.”(SeeAppendixA

forthesyllabus).Althoughlessmathematically-orientedthanitscounterpartcourseformajors,

ECO105isarigorouscoursethatstudentsmayusetowardsaneconomicsmajoraslongasthey

earn80%ormore.9Thecourseistaughtinaface-to-faceandanonlinesection,bothofwhich

haveavarietyofassessments,includingonlinequizzes,groupprojects,termtests,finalexam,

andwritingassignments.Theface-to-facecoursehastwowritingassignment(articleabstract

andoneop-edoneitheramicroormacrotopic),theonlineversionhasthreewriting

assignments:anabstractofanarticleandtwoop-eds(onemicroandonemacro).10

WRITINGASSIGNMENTSANDRUBRICS

Webegancollaboratingonthefirstop-edwritingassignmentin2011andhavesince

revisedtheassignmentstructurealongwithrelatedrubrics,resources,activities,andTA

trainingtobettersupportboththeundergraduatesinthecourseandthegraduateTAswho

gradethestudents’writing.OnekeytenetofWACisthatwritingassignmentsbeclosely

alignedwithcoursegoalsandthattheytaketheformofcommongenresinthedisciplineso

thatstudentsseethewritingasalegitimatepartofthecourseandthefieldratherthanasbusy

work.AppendixAshowsthecoursesyllabus,whichincludestenspecificlearningoutcomes.The


6

mostsalientoutcomeforthewritingassignmentsisforstudenttolearnto“Writepersuasive,

informedopinionpiecesaboutmicroeconomicandmacroeconomicpolicyissuesforageneral

audience.”Beyondthelearningoutcomes,thesyllabusalsoprovidesarationaleforthewriting

assignments:

Partofyourroleasaninformedcitizeninvolvesmakingsoundargumentsabouteconomicissuesthatmattertoyouandcommunicatingtheseargumentseffectivelytoothers.Theseassignmentswillgiveyoupracticeidentifyinganddevelopingconvincingwrittenargumentsaboutaneconomicissueforageneralaudience.

Theassignmentsasksstudentstowriteforgeneralratherthanexpertaudiencesbothtobetter

assesstheirunderstanding(it’seasytohidemisunderstandingbehindjargon)andgivethema

writingsituationmorecloselyresemblesthewritingtheywilldoaftergraduation:explaining

complex(economicorotherwise)ideastogeneralreadersratherthantoeconomicexperts.11

Scaffolding

Scaffoldingassignmentsinvolvesstructuringinwaysthatbreaklarger,morecomplex

tasksintosmaller,moremanageableonesthatsupportstudentlearning.Theideaisto

sequenceassignmentssothatstudentsgraduallyprogresstomorecomplexanddifficulttasks

withpracticeandformativefeedback.Thearticleabstractandop-edwritingassignmentsare

scaffoldedinmultipleways.Theop-edassignmentismostcloselyalignedwiththelearning

objectiveto“Writepersuasive,informedopinionpiecesaboutmicroeconomicand

macroeconomicpolicyissuesforageneralaudience.”Butundergraduates,especiallyfirstyear

students,areoftendaunted,ifnotparalyzed,attheprospectofmakinganoriginalargument.

Thearticleabstractassignmentprecedestheop-edassignment,providingpracticein

identifyingsomeoneelse’sargument,andgivingasenseofthestructureandcompositionof


7

whatthestudentwillhavetodoherselfintheop-ed.Writinganeffectiveabstractinvolves

knowinghowtoreadcritically–toreadforargument.Beforethearticleabstractassignment,

wewalkstudentsthroughthestepsofreadingcriticallyanarticlefromTheEconomistabout

Ubersurgepricing.

Theassignmentsequencingtakestheformofthreemodulesthatwedescribeinmore

detailbelow.

1. HowtoReadCritically

2. Howto(Re-)WriteanAbstract

3. Howto(Re-)WriteanOp-Ed

Noticetheemphasisonre-writing.Thereisasecondformofscaffoldingwithineach

writingassignment.Aspartofteachingstudentstowrite,weprovideanexplicitthree-step

processforthewritingprocessthatasksthemto(1)draft,(2)revise,and(3),edittheirwriting.

Thereisanexplicit,grade-incentiverationaleforthisprocessintheassignmentdescription

(AppendixD):“Manynovicestudentwritersoftensubmitfirstdraftsandwonderwhytheyearn

poormarks.However,goodwritingalmostalwaysrequiresrewriting.”

Becauseoneofouraimsistogivestudentsaframeworkandvocabularyforwritingthat

theycantransfertoothercourses,disciplines,andcontexts,theassignmentdescriptionnot

onlynamesthisthree-stageprocess,butexplainsitinconsiderabledetail.Providingstudents

withaframeworkforwritingisparticularlyimportantforthemanymultilinguallearnersinthe

coursewhostrugglewithreadingandwritinguniversity-levelEnglish.Wedevelopedavideo

modulethatnotonlydescribesbutalsomodelsthecriticalreadingprocessandgivesstudents

practicereadingcriticallyasamplearticlefromTheEconomist(showninAppendixC).Aspartof

thewrittenassignmentforHowto(Re-)WriteanAbstractandrelatedvideomodule,weinclude


8

ourownversionsofabstractsforthesamplearticle,includingaversionwithAviCohen’sre-

draftingandrevisionsshownintrackchanges(AppendixE).Bypresentingtwodifferent

abstractsofthesamearticlealongwiththerevisionprocess,weshowthatthereismorethan

onewaytowriteagoodabstractandthatevenexpertwritersmustrevisetheirdraftsto

producehighqualitywriting.ThethirdmoduleonHowto(Re-)WriteanOp-Ed,buildsonthe

processesinthepreviousmodules.

HowtoReadCritically12

Manystudentsentercollegewithinadequatereadingstrategies.Becausestudent

successonthewritingassignments(andinthecourseasawhole)requireseffectivereading

strategies,wedevelopedamoduletoteachthiscrucialskill.Themoduleconsistsofavideo

tutorialthatbothexplainscriticalreadinganddemonstrateshowexactlytodoitusingThe

Economistarticle,“AFareShake.”Thatarticleissimilartotheonesstudentsmustwritetheir

ownabstractsfor.AswereadthroughTheEconomistarticleandbeganseparatelywriting

respectiveabstracts,wecametoseejusthowcomplexthestepsinvolvedincriticalreadingare.

Ourvideooncriticalreadingbeginsbydefiningtheterm,distinguishingitfrom

criticizingordisagreeingwithatext,acommonmisconceptionthatmanystudentshavewhen

askedtocriticallyreadoranalyzeatext,event,orphenomenon.13Ratherthanemphasizing

criticizing,criticalreading(mostcollege-levelreading)involvesseveralkeymovesonthepartof

readers.Thefirstisunderstandingtheoverarchingargumentthatisbeingpresentedrather

thansimplyidentifyingtheparticularinformation,factsordatapresented.Thesecondis

identifyingthespecificclaimsbeingmadeandhoweffectivelytheseclaimsaresupportedwith


9

evidence.Thethirdmoveisaskingquestionsofthetext.Thismayinvolvechallenging

assumptionstheauthormakes,identifyinggapsorcontradictionsintheargumentorevidence,

questioningsourcesofdataorevidence(asinthearticlewheredataprovidedbyUberisused

tosupportaclaim),challengingthetestimonyofexperts(suchasUberemployees),andnoting

counter-argumentstheauthorhasprovided(ornot).

Thesecriticalreadingstepsapplytomostdisciplines,soweexplainwhatcriticalreading

meansspecificallyineconomics,whichwedescribeas“beingabletoreadandidentifythe

economicconcepts,models,assumptions,reasoning,andevidenceanauthorpresents”

(AppendixG).Becausemoststudentshavereadlittleabouteconomicsbeforetakingthis

course,itisnotenoughtomerelyexplainwhattheconceptofcriticalreadingis.Werealized

thatweneedtoshowstudentshowtoreadcriticallyaneconomicstext.Wedecidedthebest

waytodothiswastopayattentiontoourownreadingprocessandAndreaWilliam’sin

particular.Althoughunlikethestudentssheisanexpertreader,likethestudents,sheisnotan

economist.

SoAndreaWilliamsdemonstratesherowncriticalreadingprocess:howsheapproaches

anunfamiliartextinanunfamiliardisciplineandmakessenseofit.Becauseofthemany

multilinguallearners,wethoughtitbestnottoassumethatanyreadingstrategywastoo

obviousorbasic.Andreabeginshercriticalreadingdemonstrationatthebeginning,withthe

title,whichshepointsoutissometimesoverlookedbyhurriedreadersbutwhichtheauthor

haschosencarefullyandwhichusuallyforecaststhearticle’sargument.Thetitleofoursample

articleonUber(“AFareShake”)involvesapunontheword“fare,”whichwepointoutforthe

benefitoftheEnglishlanguagelearnerswhomaynotrecognizethehomonym.Andrea


10

William’sdemonstrationofcriticalreadinganalyzeschronologicallythemovestheauthor

makesandaddressesbothlocalissuessuchastopicsentencesthatforecastthesubjectofthe

paragraph(“Agoodtopicsentenceshouldtellyouwhataparagraphisabout”)andmoreglobal

observationsaboutthetext’sorganization(e.g.,thatitisstructuredasaproblem-solution).But

morethanmerelycommentoneachrhetoricalmovetheauthormakes,AndreaWilliam’s

criticalreadingdemonstrationincludesquestionsshehasofthetextsuchas:“Asa...

consumerwho...usesUberI’maskingofthearticle,‘Whynot?’Whyshouldn’tlocal

governmentsbansurgepricing?I’mexpectingthenextparagraphtoanswerthisquestion.”

(AppendixG).

Themodelofcriticalreadingwegivethestudentsincludesanalyzingthelanguageofthe

article.Forexample,AndreaWilliamspointsoutthatdescribingpayingdriversmoreduring

periodsofsurgepricingas“sharingwithdrivers”istheauthor’sattempttomakethecompany

seemgeneroustowardsitsdriversratherthanmerelyfairlycompensatingthem.Inadditionto

encouragingstudentstoattendcarefullytolanguageastheyread,ourdemonstration

addressestheuseoftwographs,whichlikeallvisualdisplaysofdata,playakeyroleinmost

economicarguments.Althoughthegraphsusedinthesampleargumentarequite

straightforward,asAndreapointsout,othersourcesofevidencearelessso.Forexample,

astudythearticleusestomakeacaseforsurgepricingwassuppliedbytheless-than-

independentsourceofUberitself,whichAndreaexplainsmakesitalessconvincingsource

hadthisevidencebeenprovidedbyaneutralthirdparty.


11

Howto(Re-)WriteanAbstract14

Followingthecriticalreadingmodule,thefirstwritingassignmentsasksstudentsto

chooseoneofsixdifferentarticlesfromTheEconomistandwriteashort(250-300words)

abstractforitthatisunderstandabletoageneralaudience.Becausethegoalistoteach

studentshowtoanalyzeandconstructbasiceconomicarguments,wedecidedthatanabstract

wouldhelppreparestudentstowritethemorechallenginganalysisassignment(theop-ed

writingassignment).Theabstractassignmentisworth5%tomakeitalow-stakesassignment

thatgivesstudentspracticereadingandwritingwithoutunduepressure.Theassignment

containsthegradingrubric,providingadditionalguidancetostudentsaboutwhatweexpect,

andwhattheyneedtodotoreceivehighmarks(AppendixB).

Weallowstudentschoicetoensurethattheyhavethefreedomtoexploreatopicthey

findinteresting,butlimitthemtoarticlesthataregoodmodelsofwriting–TheEconomistis

thegoldstandardofeconomicsjournalism.TheselectionofsixarticlesalsomakestheTAs’

gradingtaskeasier.Allabstractswrittenaboutoneofthesixarticlesweregradedbyasingle

TA.

Tofurtherscaffoldthisassignment,weusepeerfeedbacktoensurestudentstheygive

andreceivecommentsontheirinitialdraftssincetheTAhoursareonlysufficienttogradeone

versionofeachassignmentandbecausestudentslearnfromreadingandrespondingtoeach

other’swork.UsingpeerScholarsoftware(www.peerScholar.com),eachstudentresponds

onlineanonymouslytothreepeers’assignmentsandinturnreceivesfeedbackfromthree

differentpeersontheirownassignment.Weaskstudentstoratetheassignmentona4point

scale,identifyonethingthattheirpeerdidwell,onethingthatneedsimprovement,andthen


12

giveanoverallcomment.ThepeerScholarsoftwarealsohasahighlightingandcommenting

featuresostudentscanidentifyspecificwords,phrases,andsentencesthatareproblematic.

TheTAsmarkstudentsonthequalityofthefeedbacktheyprovide,assigningoneof

fourlevelsonasimplerubric:(1)Strengthsandweaknessesforeachassignedabstract,with

specifics;(2)Fairattemptbutweak,lackingspecifics;(3)Norealeffort;and(4)Missing.This

fostersaccountability,asthequalityoffeedbackstudentsprovidecountsaspartoftheiroverall

gradeonthewritingassignment.

UsingpeerScholaralsomakesiteasyforTAsnotonlytogradestudents’assignments

usingthescoringrubric,butalsoreviewthefeedbackeachstudenthasprovidedtotheirpeers.

Studentsdescribethefeedbackprocessasextremelyvaluableinmultipleways.Notonlydo

theyreceivesuggestionsforimprovingtheirabstracts,theylearnfromreadingothers’

assignmentsandprovidingfeedback.Manycommentongainingperspectivefromseeingother

students’work,ofbothhighandlowquality.

StudentssubmitafinaldraftoftheirabstracttopeerScholar,incorporatingsuggestions

frompeersoranyotherrevisions.TominimizethevariabilityacrossthedifferentTAgraders

andmaximizetheuseofthelimitedTAshours,wedevelopedarubricthatgivesstudentsas

muchinformationaspossibletoexplaintheirperformanceandjustifytheirgrade,yetiseasy

forTAstouse.AsAppendixBshows,therubricfortheabstracthasthreemaincriteria(main

economicargument,structure,andqualityofwriting),whichareassignedoneoffourlevelsof

competence(excellent,good,competent,andproblematic).Inweightingmostheavilythe

economicargument,therubricsignalstostudentsthattheirunderstandingoftheissuesis


13

whatmattersmost;yetbyalsoincluding“structure”and“qualityofwriting”ascriteriathe

rubricremindsstudentsthathowclearlytheyexpresstheirunderstandingofeconomicissues

matters.

Todiscourageplagiarism,studentsmustsubmittheirfinaldraftstoTurnitin,the

software-matchingprogramourinstitutionsubscribesto.WeusedtheTurnitinsettingsthat

allowstudents,whentheysubmit,toseetheoriginalityreporthighlightingpassagesthat

containtextualsimilaritiestopapersinthedatabase.Theonusisonthestudentforfixingany

suspiciouspassages,anddraftscanberesubmitted(withneworiginalityreports)untilthedue

date.Thisdrasticallyreducesmarkingtimeforplagiarismdetection.15

Withtherubricsforpeerassessmentandthefinaldraft,markingtimeaveragedabout

5-6minutesperabstract.MarksareentereddirectlyintopeerScholar,andTAssimplyhaveto

pickonenumberforassessment,and3numbersforthefinaldraft.Themarksaretransferred

directlytotheLMS(inthiscase,Blackboard)gradebook.

Howto(Re-)WriteanOp-Ed16

Buildingontheabstractassignment,thesecondwritingassignment(AppendixF)and

associatedvideomodule(AppendixI)asksstudentstomovefromsummarizingtoanalyzingby

identifyingacontroversialeconomicargumentfromarecentarticle,editorial,oropinionpiece

(thepiecemusthavebeenpublishedsincethestartdateofthecourse).Studentsthenwritea

commentary(300-500words)inresponsetothearticleoropinionpieceinwhichtheymakean

argumentaboutaneconomicissue.Allowingstudentstobasetheirop-edasareactiontoa

publishedstoryisanotherformofscaffolding.Wedecideditwastoodifficultforstudentsto


14

comeupwithanargumentontheirown,andusingapublishedsource(whichtheyhadto

includeintheirsubmission)minimizedtheneedforadditionalresearch.

Aswiththeabstractassignment,studentsareinstructedtofollowthethree-stage

writingprocess(draft,revise,andedit)andpeerreviewisarequiredcomponenttoprovide

studentswithformativefeedbackontheirinitialdraftandguidetheminmakingtheirrevisions.

Liketheassignmentitself,therubricfortheop-edassignment(showninAppendixF)ismore

complexthantheabstract.Themaincriteriaoftherubricare(1)Title,introduction,audience

(howappropriatethewritingisforageneralreader),andbackgroundinformation;(2)

Economicargument,reasoning,counter-argument,andevidence;(3)Structureandclosing;and

(4)Qualityofwriting.

Thisop-edassignmentiscloselyalignedwiththecorelearningoutcomesofthecourse

becauseitasksstudentstoengagewithkeyeconomicissuessuchasgainsfromtrade,

governmentversusmarketfailures,andenvironmentalpoliciesastheyexplainandargueforor

againstvariouspolicies.Byhavingtopersuadeanimaginedgeneralratherthanexpertreader,

studentsmustdomorethanmerelyparrotlecturesorthetextbookandinsteaddemonstrate

theirunderstandingintheirownwords.

Despitethedetailedassignmentinstructions,videotutorials,andpeerresponse,some

studentsstillstrugglewiththereadingandwritingelementsofthecourseandrequire

additionalsupport.Werecommendinallofourwrittenandvideoinstructionsthatstudents

seekhelpontheirwritingassignmentsatoneofourcampuswritingcentres.Inthisway,we

provideadditionalsupporttostudentswhoneedthiswithoutexceedingourcourseTAbudget.


15

Sincemostcollegesanduniversitieshavewritingcentres,thisstrategywouldworkwellina

varietyofcontexts.

TATRAININGANDRESOURCES

OnedistinctivefeaturesoftheWITprogramattheUniversityofToronto,whichthe

initiative’snamesuggests--WritingInstructionforTAs--isitsdualfocusonimproving

undergraduatewritingandprovidingGraduateTeachingAssistantswithtrainingand

professionaldevelopmentinwritingpedagogy.WhereasmanyWACprogramsexportTAsfrom

Englishtoserveaswritingfellowsinotherunits,theapproachofWITistoprovideTAswiththe

necessarytrainingandsupporttoteachwritingintheirhomedisciplines.TAsarenotteaching

designatedwritingcourses;rather,theyareteachingwritingskillswithindisciplinarycourses

suchaseconomics.

Nowinitstenthyear,akeyfeatureofWITisitsuseofbenchmarkingorgrading

calibrationsessionstoimprovethequalityoffeedbackstudentsreceiveontheirwritingandthe

consistencyofthegrading,butalsoprovideprofessionaldevelopmentinteachingwritingfor

TAs.Foreachwritingassignment,thecourseinstructorandWITCoordinatorco-facilitate

sessionswiththeTAswhereweanalyzestudentwritingandscoreitaccordingtotherubric.

ThesesessionsgiveTAsachancetoaskquestionsabouttheassignmentandclarifythe

expectationsoftheinstructor.


16

GradingCalibration

Thereweregradingcalibrationsessionsforeverywritingassignment–thearticle

abstract,andeachop-ed.Thesesessionslastedabout2hourseach,withtheTAs,Andrea

WilliamsandAviCohenaroundabigtablewithalargewhiteboard.Everyonebringsalaptop

foraccessingpeerScholar,wherethegradingtakesplace.Wejointlyselect5-6studentpapers

thatcovertherangeofqualityandgradingissues–strong/weakarguments,structure,writing,

andprosewrittenbymultilinguallearners.AfterAndreaWilliamsandAviCohengothrough

therubriccategories,everyonereadsandprivatelygradesthesameop-ed.Weputamatrixon

theboard,withtherubriccategoriesasrows,andthenamesofeachgrader(TAs,Andrea,Avi)

acrossthecolumns.

Wegoaroundthetable,haveeachpersonstatetheirmarksforeachrubriccategory,

andmoreimportantly,theirthinkingbehindthatmark.Wedon’tallowhalfwaymarksbetween

rubricgrades(excellent,good,competent,problematic),forcingquickerdecisionsandless

agonizing.Therewasoftenconsensusonmarksforindividualcategories,andmoreoften,close

totalmarksdespitedifferencesinindividualcategories.Butsituationswithlargedifferencesin

markswereoftenmostvaluable,fortheensuingdiscussionwouldclarifywhatwewerelooking

for.Wedonotprivilegeourownmarks(whichoftendisagreed),andusuallyspokeafterthe

TAs.WhenTAsweretornbetweenmarksforanindividualcategory,weencouragedthemto

compensateinadifferentcategory.Forexample,ifagraderwasequivocalbetweenexcellent

andgoodforthecategoryofEconomicArgument,shemightgiveexcellent,butthengofora

lowergradeforthecategoryofQualityofWritingwhereshewasequivocalbetweenexcellent

andgood.ThesediscussionshelptheTAsfeelcomfortableandconfidentintheirmarking.


17

Theop-edassignmenthasanadditionalrubric(notpresentintheabstractassignment)

fortheReflection(“onwhatyoulearnedfromthewritingandassessmentprocess”)that

studentsmustsubmitwiththerevisedversionoftheirop-ed.TheReflectionrubrichasfour

levels:(1)Thoughtfulanddetailed,referstopeercomments,incorporatedrevisions;(2)Honest

attemptbutlackingspecifics—maximumifmissedassessments;(3)Norealeffort;and(4)

Missing.AccordingtothecognitivepsychologyresearchbehindpeerScholar,thecombination

ofcriticallyassessingothers,receivingandincorporatingassessments,andreflectingonthe

processenhanceslearning.17

HavingusedpeerScholarforyears,AviCohendevelopedanefficientworkflowforthe

orderofactivitiesinvolvedingradingeachassignment.Fortheop-eds,thoseactivitiesinclude

lookingatthemediastorylinkonwhichtheop-edisbased,readingthefinaldraftoftheop-ed

andassigninggradesineachcategory,readingthepeerassessmentsgiventothestudent,

comparingtheoriginalandfinaldrafts(thesoftwaremakesthiseasy,similarlytotrack

changes),andgradingtheReflection.Wethenlookatthepeerassessmentsgivenbythis

student,andassignamarkusingtheAssessmentrubric.Hecreatedashortscreencapture

video18walkingtheTAsthroughtheworkflow.WeestimatethatittakesaTA7-9minutesto

gradeeachop-edassignment.

OneofthemostchallengingissuesourTAsfaceingradingstudentwritingconcernshow

toevaluatethewritingofEnglishlanguagelearners(ELL).Tocounteractthewidespreadbut

falsenotionsthatallgoodwritingiserror-freeandthaterror-freewritingisalwaysgood,atthe

gradingcalibrationsessionweintroducetheideathatmanygoodwriterswriteEnglishwithan

“accent.”Inconversation,someonemaybearticulateandeasilyunderstoodbylistenerseven


18

thoughtheyspeakwithanaccent(notlikeanativespeaker).Similarly,goodwritersmaywrite

withanaccentandmakeminorerrorsthatdon’timpedemeaning.Althoughweencourage

studentstoeditandproofreadcarefullytheirwritingassignments,wealsorecognizethatfor

mostEnglishlanguagelearnersitisimpossibletoproduce100%error-freeproseontheirown.

DuringthesessionwediscussstudentwritingsamplesthathelpTAsdistinguishbetweenmajor,

meaning-impedingerrors(suchasentencefragmentsandmajorproblemswithverbtense)and

moretrivialones(suchasmissingarticles).ManyoftheTAsarethemselvesmultilingual

learnerswhohavenevergradedwritingbefore.

SeeinghowtheTAsusetherubricandgettingtheirfeedbackonitenablesusto

improveitovertheyears.Forexample,thisyearwenoticedthatmanystudentsandTAsalike

werestrugglingwitharubriccategorythatincludedthephrase“useofeconomicconcepts.”

Therubricwasintendedtogetstudentstoactuallyapplykeycourseconceptslikeelasticityto

explainrealeventslikepricingdecisions.Instead,studentsinterpretedtherubricasrewarding

thementionofeconomicconcepts,whichtheythensprinkledinwithoutnecessarilyexplaining

them.Asaresult,werevisedtherubrictosay“useofeconomicreasoning”andthewritinghas

sinceimproved,withfewerstudentsresortingtousingeconomicjargon.

TAHours

Whatdothesewritingassignments“cost”intermsofTAhours?Beyondtheestimates

of5-6minutespergradedabstract,and7-9minutespergradedop-ed,wecanonlygiverough

approximationsbecauseTAwritingresponsibilitiesarecombinedwithtraditional


19

responsibilitieslikeinvigilatingandmarkingtestsandexams.Eventheonlinesectionhasan

in-personmidtermandfinalexam.

Forthethreeacademicyearsof2013/14–2015/16,theECO105full-yearcoursewas

allocated1050TAhoursfor500students,or2.1TAhoursperstudentperyear.The2016/17

academicyearwasthefirstwithbotha500studentface-to-facesectionanda300studentfully

onlinesection.TAresourceswerecombinedforthesections,with1505TAhoursfor800

students,or1.88TAhoursperstudentperyear.SomeoftheTAhourswereunused,sowe

estimatethatthecourses,includingtheabstractandop-edwritingassignments,need

approximately1.75TAhoursperstudentyear.

OtherTAresponsibilitiesincludeinvigilating6hoursoftermtests(4onehourtestsfor

theface-to-facesection,1twohourmidtermfortheonlinesection)and5hoursoffinalexams

(2hoursforface-to-face,3hoursforonline).Termtestsandexamsconsistofacombinationof

true/false,multiplechoice,andshortanswerproblems.WhiletheheadTAdealswith

computerizedmarkingoftrue/falseandmultiplechoicequestions,TAsgradetheshortanswer

problems,whichconstituteabout25%oftotaltest/exammarks.Theothermajorresponsibility

isansweringstudentquestionsoncoursediscussionboards(forbothsections).Wehave

eliminatedin-personofficehoursforlackofdemand,andtherearenotutorialsforthese

courses.

Wehopethesenumbersgiveyouatleastsomeabilitytojudgethe“affordability”of

implementingwritingassignmentsinyourowncourses.Suggestionsaboutwhatother

informationwouldbeusefultohavearemostwelcome.


20

CONCLUSION

Readingandwritinginstructionandassignmentscanbeseamlesslyintegratedintoan

economicliteracycoursetohelpstudentslearnhowtoapplykeyeconomicconceptswhile

improvingtheirwriting.Weprovidestrategiesforrealizingpedagogicaleffectivenessand

efficienciesinwritingpedagogies—short,scaffoldedassignmentswithpeerfeedbackandarich

arrayofwrittenandvideotapedinstructionsandguidedpractice—thatcanbeusedinavariety

ofcontexts.Theseassignmentscanbeadoptedinlargeclassesinotherinstitutionalcontexts,

orusedinsmallerclasseswhereinstructorscouldintroduceadditionalassignmentsormeet

one-on-onewithstudentstoimprovetheirwriting.Wehavepresentedherepracticalstrategies

forintegratingwritingintolargecourses.

Anecdotally,weseesignificantimprovementsinthestudents’writingbetweenthe

draftsoftheabstractandthefinalversions,butitwouldbeusefultoanalyzestudentwriting

moresystematicallytoseeinwhatareasstudentsmostandleastimproveandhowwecan

adapttheassignmentsandactivitiesaccordingly.Althoughwehavedevelopedrobustcourse

andwritingassignments,goodteaching,likegoodwriting,involvesongoingrevision.Having

developedeffectiveassignmentsandrubricsweplantopilothavingTAsaddageneral

commenttostudentsontheirwritingassignmentasmeansofprovidingadditionalformative

feedbacktostudents.Weanticipatecontinuingtorefinethewritingelementstoensurethat

studentsgetasmuchguidedpracticeandfeedbackpossiblegiventheconsiderableresource

constraints.


21

REFERENCES

Cohen,A.andJ.Spencer.1993UsingWritingacrossthecurriculumineconomics:IsTakingthePlungeWorthIt?TheJournalofEconomicEducation.24,no.3,(Summer):219-230.http://links.jstor.org/sici?sici=00220485%28199322%2924%2924%3A3%3C219%3AUWTCI%3E2.0.CO%3B2-P

Hall,J.andM.Podemska-Mikluch.2015.TeachingtheEconomicWayofThinkingthroughOp-eds.InternationalReviewofEconomicsEducation,19,(May):13-21https://doi.org/10.1016/j.iree.2015.05.002

TheInternationalNetworkofWACPrograms(INWAC),StatementofWACPrinciplesandPracticeshttps://wac.colostate.edu/principles/statement.pdf

King,A.2002.Structuringpeerinteractiontopromotehigh-levelcognitiveprocessing.TheoryintoPractice,41,33-39.

McLeod,SusanH.1992.WritingAcrosstheCurriculum:AnIntroduction.InWritingAcrosstheCurriculum:AGuidetoDevelopingPrograms.Eds.SusanH.McLeodandMargotSoven.NewburyPark:SagePublications.1-11.

Russell,DavidR.2002.WritingintheAcademicDisciplines:ACurricularHistory.2ndEd.Carbondale:SouthernIllinoisUniversityPress.

Venables,A.,&Summit,R.2003.Enhancingscientificessaywritingusingpeerassessment.InnovationsinEducationandTeachingInternational,40,281-290.

Vgotsky,L.S.1978.Mindinsociety:Thedevelopmentofhigherpsychologicalprocesses.Cambridge,MA:HarvardUniversityPress.


22

ENDNOTES

1ForanoverviewofWACandWIDseeRussell(2002)andMcLeod(1992).2AttheUniversityofToronto,studentsmustchooseeitheranintensivespecialistmajor,adoublemajor,oramajorandtwominors.With40one-semestercoursesrequiredforadegree,aspecialistmajorhas24courses,amajorhas14courses,andaminorhas8courses.Specialistmajorsgenerallyarepreparingforgraduateschool.StudentswhotakeECO100IntroductiontoEconomics,areeligibleforalleconomicsdegrees.TheunsexynameforECO105,IntroductiontoEconomicsforNon-Specialists,isaccurateinthatitallowsstudentstopursueaneconomicsmajororminor,butnotaspecialistdegree.ECO105attractsstudentsdoingmajorssuchasinternationalrelationsandpublicpolicy,whichrequireatleastoneeconomicscourse,andstudentsinotherscience,socialscience,andhumanitiesdepartmentslooksforanelectiveeconomicscourse.http://calendar.artsci.utoronto.ca/crs_eco.htm#ASSPE14783AppendicesA-Fcontainallprintmaterials.AppendicesG-Icontaintheslidesfromthevideomodules.4MostTAsareeconomicsMAorPhDstudents,butbetween¼and½oftheTAsaregraduatestudentsinthe2yearMaster’sofPublicPolicy(MPP)program,whichrequiresafullyearmicro/macrocoursetaughtbyanEconomicsfacultymember.WhiletheEconomicsqualificationsoftheseMPPstudentsarenotasstrong,theyhaveturnedouttobeexcellentTAs–moreversedinwriting,policyapplications,andtheexperienceoflearningsomeeconomicsforitsapplicability.Atthestartofeachyear,AviCohengatherspreferencesfromtheTAsformarkingproblemsontests/examsversuswritingassignments.MPPTAsarefarmoreinterestedin,andusuallybetterat,markingwritingassignments.5TheFacultyofArtsandScience,wherethedepartmentofeconomicsissituated,hassevencollegewritingcenterswithapproximately30full-timeequivalentinstructorsandtwiceasmanyadjunctinstructors.TheUniversityofTorontoasawholehas9undergraduateWritingCenterson3campuses.http://writing.utoronto.ca/writing-centres/6TheDepartmentofEconomicshas62full-timetenuretrackfaculty,and7full-timeteachingstreamfaculty.7It’simpossibletogetaprecisenumberforthepercentageofEnglishLanguageLearners,sincethereisnodefinitionofsuchacategoryandtheUniversitydoesnotadministeralanguageaptitudetesttoallincomingstudents.WehavedataonthenumberofincomingstudentsrequiredtoshowproofofEnglishlanguageproficiency(i.e.TOEFLorothertestscore),butthisappliesonlytopeoplewhodidn'tcomplete4yearsofhighschoolinOntarioanddoesnotcoverthemanystudentswhoaremorecomfortableinalanguageotherthanEnglish,whethertheyareCanadians,permanentresidents,orvisastudents. 8NormallyWIT-designatedcoursesreceiveupto280additionalTAhoursovertheirbasehoursforteachingandgradingwriting.9Studentswhoreceiveagradeof67%inECO100or80%inECO105areeligibletogoontoupper-levelcoursesandtomajororminorineconomics.Allmajorsmustalsoreceiveagradeof63%inafull-yearcalculuscourseasapre-requisiteforintermediatemicroormacro.


23

10Theonlinesectionwasofferedforfirsttimethisyear.Differencesinthestructureoronlinecoursescontributedtodecisiontoaddasecondop-edwritingassignment.11AccordingtoCohenandSpencer(1993,221),“Studentsfaceauniquedifficultyinclarifyingtheaudienceforwhomtheyarewriting.Inmostwritingsituations,thewriterwritestoinformareaderwhoisgenerallylessknowledgeable.Insharpcontrast,studentsareaskedtowriteforareader(theinstructor)whoisclearlymoreknowledgeable.Thisplacesthestudentwriterinanartificialandhencedifficultsituation.’TheECO105writingassignmentsallspecifytheaudience:“Youraudienceisthegeneralreadingpublic.Assumeyouraudiencehassomeeducationandbackgroundincurrentaffairsandunderstandsbasiceconomicconcepts,butisnotknowledgeableaboutthedetailsofeconomictheoryorpolicy.Inotherwords,you’llneedtoexplainanyeconomicconceptsandspecializedvocabularyinawaythatkeepstheirinterestandrespectstheirintelligence.YouraudienceisnottheprofessorortheTAs.Youneedtowritemorelikeajournalistthananacademic.”Becauseofthepeerreviewprocess,theactualaudienceisfellowstudents.12SlidesforthevideoareinAppendixG.13Weallowthatcriticalreadingsometimesinvolvesdisagreeing.14SlidesforthevideoareinAppendixH.15Theinstructionstostudentsare:“TurnitingeneratesanOriginalityReporthighlightingsuspiciouspassages.Clickonhighlightedpassagestolinktotherelatedinternetsource.Ifthesourceisastudentpaperfromyourclassthiswillbeindicated.Ifthesourceisastudentpaperfromanotherclass,yourinstructorwillreceivecontactinformationforthatinstructor.EachOriginalityReporthasascorebasedonthepercentageofmatchingtext.ThesereportsdoNOTindicatewhetheryouhaveplagiarized,butsimplyhelptoidentifysourcesthatcontaintextualsimilaritiesinsubmittedpapers.Ifyoufindsignificanttexturalsimilaritiestouncreditedsources,youshouldmodifyyourpaper.YoumaysubmityourpaperasmanytimesasyoulikebeforetheduedatetorechecktheOriginalityReport.”16SlidesforthevideoareinAppendixI.17ThepowerofpeerassessmenttoenhancelearningfitswellwiththeviewsofVygotsky(1978)andhasbeensupportedempiricallyandtheoreticallyviamanyresearchstudies(e.g.,King,2002;Venables&Summit,2003).18https://play.library.utoronto.ca/o8oeVCYLTXRn

scalable, scaffolded writing assignments with peer...

Documents