http//www.cytojournal.com

1st Peer-Reviewer Retreat 2006During USCAP Annual Meeting, Hyatt Regency Atlanta, Atlanta, Georgia

February 12, 2006

Vinod B. Shidham, MD, FIAC, FRCPathExecutive editor & coeditor-in-chief, CytoJournal (www.cytojournal.com)Director of Cytopathology Fellowship Training Program & FNAB ServiceDirector of Fine Needle Aspiration Biopsy ServiceMedical College of Wisconsin (MCW)Milwaukee, WI 53226vshidham@mcw.edu

CytoJournal Peer-Review process: goals and resources

Goals

►Maintain high standards of peer-review process for CytoJournal

► Create a pool of best peer-reviewers for CytoJournal

► Appreciate the pivotal role of reviewers in publishing good science

► Emphasize significance of efficient quick turnover

Online submission of the manuscript to the editorial office

Abstract directed to the executive editor by Managing editor with academic support / participation by trainee and faculty colleagues

Monitoring/Academic editor is identified by the Executive editor (Managing editor will follow the progress)

Abstract sent to Monitoring/Academic Editor

At least 2 ad hoc reviewers are identified and communicated to Managing editor

Manuscript is forwarded for Peer Review

The comments by the reviewers sent to Editorial office are forwarded by Managing editor to be communicated to the Monitoring/Academic editor or the Executive editor

Final decision (Revise*/Accept/Decline) is taken by both or one of the Editors-in-chief **, based on the reviewers comments and monitoring editor’s comments.

If revision is indicated the manuscript is re-evaluated based on reviewer’s and / or Monitoring/Academic editor’s comments and final decision as to Accept/Decline by both or one of the Editors-in-chief ** is communicated to the author.

Shidham VB, Cafaro A, Atkinson BF. CytoJournal joins 'open access' philosophy. Cytojournal. 2004 Jul 29;1(1):1.Free full text available at: http://www.cytojournal.com/content/1/1/1

Limitations

►Although fundamental to scientific literature, PR has been largely an amateur process (1)

►Only a few biomedical experts are formally trained in critical analysis of manuscript (2)

1. BMJ training for peer-reviewers. BMJ 2004;328:658. 2. Callaham M, Schiger D, Copper RJ. An instructional guide for peer-review

of Biomedical manuscripts.

http://www3.us.elsevierhealth.com/extractor/graphics/em-acep/index.html

History►In place only since 1950’s ►Evolved independently at individual

journals►Traditionally an ‘amateur’ volunteers

without training►Efficacy not proven►Science as we know is product of PR

1. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: randomised controlled trial. BMJ 2004;328:673-5.

2. Davidoff F. Improving peer review: who’s responsible?

BMJ 2004;328:657-8.

Role of PR in modern science

►Credibility to research publication ►Builds careers►Facilitates grants, promotions►Ultimately shapes clinical care of patients►Catalyst for free scientific communication►Quality assurance mechanism-

identify flaws before publication►Editorial tool- filter out ‘bad’ excess information►‘Validation’ of science (in eyes of many readers)

Efficacy of PR►Hyopthesis-

PR is able to steer the quality of research►Scientific scrutiny- little research►Hard to demonstrate objectively-

if it improves quality of published science►Most believe-

PR introduces consistency and quality.►Compare-

high quality journals withself published quack information

►Like democracy- It is not very good, but better than alternatives.

Weaknesses of PR process

►Part time enterprise of volunteers ►Lack of formal training or knowledge about resources►Wide variation in qualifications and motivations►PRs evaluating manuscript are few (1 to 3)-

with low inter-rater agreement►Individual biases of reviewers (pro or against)►Bias favors positive than negative results►Quality of reviewers is not assessed and tracked►Can NOT detect fraud.

Product of PR process

►Detection of errors, omissions, misstatements. ►Monitor ethics in research►Improve quality of data reporting►Enhance readability of article►Improve accuracy of claims and conclusions►But, statistical errors are not usually detected by PR

Peer Reviewers Selection- How?

►Lack of standardized approach ►Usually based on word of mouth recommendations

from EB and other experts in the field►PubMed search►Expertise in science does not equate with reviewer

skills►Retreats such as this may introduce objectivity►Motivation and commitment for steering best

science in the field is pivotal

Peer Reviewers- How are they trained?

►Almost never trained ►Graduate training in epidemiology and study design

is believed to improve PR quality.►Most journals DO NOT screen or train reviewers►Impact of PR Training workshops is not proven►It is recommended, at least to orient PRs and

evaluate them on regular basis.►Regular participation at such retreats appears to be

favorable to achieve PR of high standards.

CytoJournal Peer Reviewers

Motivated and committed for high standards of peer-review process

Efficient with quick but high quality peer review for rapid turn around.

►

►

Resources for Peer-Review guidance►On-line resources: 1. On-line training: Callaham M, Schiger D, Copper RJ. An instructional

guide for peer-review of Biomedical manuscripts.http://www3.us.elsevierhealth.com/extractor/graphics/em-acep/index.html

2. Training package for BMJ peer reviewers: http://bmj.bmjjournals.com/advice/peer_review/

►Reading material:1. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of

training on quality of peer review: randomised controlled trial. BMJ 2004;328:673-5.

2. Davidoff F. Improving peer review: who’s responsible?

BMJ 2004;328:657-8.

3. Purcell G, Donovan SL, Davidoff F. Measuring the quality of editorial peer review. JAMA 2002;287:2786-2790.

4. Peer Review issues of JAMA in March 1990, July 1994, July 1998, July 2002.

Resources for Peer-Review guidance►On-line resources: 1. On-line training: Callaham M, Schiger D, Copper RJ. An instructional

guide for peer-review of Biomedical manuscripts.http://www3.us.elsevierhealth.com/extractor/graphics/em-acep/index.html

2. Training package for BMJ peer reviewers: http://bmj.bmjjournals.com/advice/peer_review/

►Reading material:1. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of

training on quality of peer review: randomised controlled trial. BMJ 2004;328:673-5.

2. Davidoff F. Improving peer review: who’s responsible?

BMJ 2004;328:657-8.

3. Purcell G, Donovan SL, Davidoff F. Measuring the quality of editorial peer review. JAMA 2002;287:2786-2790.

4. Peer Review issues of JAMA in March 1990, July 1994, July 1998, July 2002.

►On-line resources: 1. On-line training: Callaham M, Schiger D, Copper RJ. An instructional

guide for peer-review of Biomedical manuscripts.http://www3.us.elsevierhealth.com/extractor/graphics/em-acep/index.html

http://www3.us.elsevierhealth.com/extractor/graphics/em-acep/index.html

PubMed indexed, peer-reviewed cytopathology journal

Available free on-line at- (www.CytoJournal.com)