peer review in an economics writing course: the pen is still mightier than the sword,
DESCRIPTION
Jennifer Imazeki, Economics Scaffolded Writing and Reviewing in the Disciplines(SWoRD) is a web-based peer-review system. One of the primary innovations of SWoRD, relative to other peer review tools, is the scoring algorithm through which peer review scores are converted into student grades for both writing and reviewing. In this session, I will discuss my experience with SWoRD, which I used in Spring 2011 for an upper-division writing course for economics majors, replacing my previous system of ‘manual’ peer review (i.e., students swapping papers)TRANSCRIPT
![Page 1: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/1.jpg)
Peer review in an economics writing course:
the Pen is still mightier than the SWoRD
Jennifer Imazeki
Department of Economics
http://economicsforteachers.blogspot.com
![Page 2: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/2.jpg)
SWoRD
• Scaffolded Writing and Reviewing in the Disciplines– Students submit papers– Students review papers of 4-6 peers
• Comments and 7-point numeric rating
– Students ‘back evaluate’ reviews they receive• 5-point numeric rating
![Page 3: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/3.jpg)
What makes SWoRD different?
• Student review and backevaluation scores grades for both writing and reviewing– Reviewing grade: “Accuracy” + “Helpfulness”
• Accuracy: Consistency compared to other reviewers• Helpfulness: Back evaluation ratings
– Writing grade: adjusted average of numeric ratings from reviewers
– Scoring algorithm to remove bias
![Page 4: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/4.jpg)
Econ 449W: Economic Literacy
• “The goal of this course is to make you better economists by making you better writers… This course will focus on how to apply and use [economic] knowledge, by incorporating economic material and the economic way of thinking into writing for a more general audience.”
![Page 5: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/5.jpg)
Econ 449W: Structure
2010:• 6 peer-reviewed papers,
exchanged in class• 1 reviewer for each
paper, plus instructor feedback
• No feedback for reviewers, participation points only
![Page 6: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/6.jpg)
Potential benefits of SWoRD(vs. other peer review tools)
• Students read multiple papers, receive feedback from multiple reviewers
• Back evaluations provide incentive to take reviews seriously, give students feedback for improving reviews (without requiring instructor intervention)
• Rubric scores converted to grades with adjustment for bias
![Page 7: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/7.jpg)
Econ 449W: 2010 vs. 2011
2010:• 6 peer-reviewed papers,
exchanged in class• 1 reviewer for each
paper, plus instructor feedback
• No feedback for reviewers, participation points only
2011:• 4 peer reviewed papers,
all online submission• 4-5 reviewers for each
paper, no/less instructor feedback on draft 1
• Writers back-evaluate reviewers, reviewing score based 50% on those back evaluations
![Page 8: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/8.jpg)
Assignment timeline
First Draft 5 days
Reviews (5) 5 days
Final draft 5 days
Final reviews
(5)
Back reviews by Thurs
Back reviews by Wednesday
9pm Sunday 9pm Friday 9pm Wednesday 9pm Monday
![Page 9: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/9.jpg)
Issues
• Lack of flexibility• Lack of transparency• Scoring algorithm questionable
![Page 10: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/10.jpg)
![Page 11: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/11.jpg)
![Page 12: Peer review in an economics writing course: the Pen is still mightier than the SWoRD,](https://reader034.vdocuments.us/reader034/viewer/2022042813/5479c74cb4af9faf158b48fb/html5/thumbnails/12.jpg)
Peer review in an economics writing course:
the Pen is still mightier than the SWoRD
Jennifer Imazeki
Department of Economics
http://economicsforteachers.blogspot.com