peer review in an economics writing course: the pen is still mightier than the sword

Peer review in an economics writing course:

the Pen is still mightier than the SWoRD

Jennifer ImazekiDepartment of Economics

jimazeki@gmail.comhttp://economicsforteachers.blogspot.com

SWoRD

• Scaffolded Writing and Reviewing in the Disciplines– Students submit papers– Students review papers of 4-6 peers

• Comments and 7-point numeric rating– Students ‘back evaluate’ reviews they receive

• 5-point numeric rating

What makes SWoRD different?

• Student review and backevaluation scores grades for both writing and reviewing– Reviewing grade: “Accuracy” + “Helpfulness”

• Accuracy: Consistency compared to other reviewers• Helpfulness: Back evaluation ratings

– Writing grade: adjusted average of numeric ratings from reviewers

– Scoring algorithm to remove bias

Econ 449W: Economic Literacy

• “The goal of this course is to make you better economists by making you better writers… This course will focus on how to apply and use [economic] knowledge, by incorporating economic material and the economic way of thinking into writing for a more general audience.”

Econ 449W: Structure

2010:• 6 peer-reviewed papers,

exchanged in class• 1 reviewer for each

paper, plus instructor feedback

• No feedback for reviewers, participation points only

Potential benefits of SWoRD(vs. other peer review tools)

• Students read multiple papers, receive feedback from multiple reviewers

• Back evaluations provide incentive to take reviews seriously, give students feedback for improving reviews (without requiring instructor intervention)

• Rubric scores converted to grades with adjustment for bias

Econ 449W: 2010 vs. 2011

2010:• 6 peer-reviewed papers,

exchanged in class• 1 reviewer for each

paper, plus instructor feedback

• No feedback for reviewers, participation points only

2011:• 4 peer reviewed papers,

all online submission• 4-5 reviewers for each

paper, no/less instructor feedback on draft 1

• Writers back-evaluate reviewers, reviewing score based 50% on those back evaluations

Assignment timeline

First Draft 5 days

Reviews (5) 5 days

Final draft 5 days

Final reviews

(5)

Back reviews by Thurs

Back reviews by Wednesday

9pm Sunday 9pm Friday 9pm Wednesday 9pm Monday

Issues

• Lack of flexibility• Lack of transparency• Scoring algorithm questionable

Peer review in an economics writing course:

the Pen is still mightier than the SWoRD

Jennifer ImazekiDepartment of Economics

jimazeki@gmail.comhttp://economicsforteachers.blogspot.com