Challenges in the selection, design and implementation of an online submission

and peer review system for STM journals

ELPUB 2007 – Vienna, Austria - June 14, 2007

Judy Best – judy.best@nrc-cnrc.gc.ca

NRC CISTI

Overview

• Background

• What is OSPREY?

• Overview of OSPREY’s features

• Build or Buy?

• Methodology

• Technologies – Open Source and Other Technologies

• Implementation

• Impact

• Current Events

• Future Work

Background

• Canada Institute for Scientific and Technical Information (CISTI)

Background

• CISTI– science library and a world leader in document delivery for all

areas of science, technology, engineering and medicine

• NRC Research Press– CISTI’s publishing arm, publishing since 1929– publishes 16 international print and online STM journals + 15

client journals– 1996 - Publish content in PDF format on the Web for its

subscribers– 1997 - Create SGML metadata – OCLC DTD– 2004 - XML publishing system - custom DTD

Background - Transition to Electronic Peer Review

• Traditionally submission of manuscripts, peer review process and manuscript management - manual and paper-based

• Email was introduced for transmission of manuscripts and reviews• 2000 - Purchased commercial software - PaperPath 2000

– helped bridge the gap in the digital world between the peer review process and publication

– client software installed on desktop for editorial office staff– web-based access for authors and reviewers – submission of

manuscripts and reviews– supported 15 journals– vendor closed late 2001

• 2004 - developed Online Submission and Peer Review System (OSPREY)

What is OSPREY?

• Supports web-based submission of manuscripts, reviews and editorial decisions

• Automates and streamlines the peer review process within a configurable automated electronic environment

• Supports 36 STM journals

OSPREY features…

• English and French interface

• Single sign-on

• Automatic editorial workflows for peer review and manuscript management

• Creation of PDF file in real-time for reviewing purposes– accepts up to 300 different file types– accepts multiple files (manuscript, figures, tables)

• Captures Meta Data in XML format

OSPREY features

• Customizable– journal branding– correspondence templates– copyright forms– reviewer forms– roles, tasks, workflow– keywords– role privileges

Build or Buy…again

• Considerations– possibility of collaboration with another publisher– available features in COTS or licensed product– associated costs – our diverse editorial workflows– requirement for an English and French interface– CISTI’s technical skills and infrastructure– storage of confidential data off site– accessibility to data for customer relationship management– integration to publishing and subscription management systems– past experience in purchasing software– ensure our continued access to the source code and systems

Decision....Build

• Develop a system in-house that would:– ensure ease of use– minimal overhead and support costs– flexibility and scalability – future growth of features.

• Develop to current standards– Java and XML for the application– oracle as its database engine– component-based architecture

Methodology

• Collaboration– partnered with Commonwealth Scientific and Industrial

Research Organisation (CSIRO) Publishing, Australia– shared a common vision, priorities and goals– ability to share resources (people and money)– gather broader set of requirements– draw on skills and experiences of two scientific publishers

• Development Approach– iterative– frequent releases– development completed at CISTI

Team

• 2 developers

• System administrator

• Database administrator

• Application architect

• Stakeholders

• Users

• Business analyst

• Project manager

Open Source Technologies

• Java

• Java servlets and Java Server Pages (JSP)

• Model View Controller (MVC) design pattern

• Java Remote Method Invocation (RMI)

• Tomcat

• Apache

• XML

• MikTex

Other Technologies

• Adlib Express Server

• Oracle 9i database

• Network Attached Storage (NAS)

Implementation

• Production Environment – web/application server + failover – database server + failover – two conversion servers + failover – network attached storage + failover

• Training and Support– editorial assistants – onsite 4 day training– editors and associate editors – onsite 2-3 hrs.– helpdesk

• Data Conversion– on average converted 5 yrs data per journal– data analyzed, mapped and validated

Integration to Publishing System

• XML file created and exported to Publishing System

• XML file is created and sent to the publishing system • <?xml version="1.0" encoding="UTF-8"?>• <EXPORT>• <ITEM>• <ITEM_INFO>• <EXPORT_DATE>13/09/2005</EXPORT_DATE>• </ITEM_INFO>• <MANUSCRIPT>• <TITLE>Life history variation among populations of Canadian Toads in Alberta, Canada</TITLE>• <NUMBER>4727</NUMBER>• <TYPE>Article</TYPE>• <DATE_SUBMITTED>31/08/2004</DATE_SUBMITTED>• <DATE_FINALIZED>13/09/2005</DATE_FINALIZED>• <OUTCOME>Accepted</OUTCOME>• <ABSTRACT>Development of appropriate conservation plans relies on life history information and

how life history traits vary across populations of a species. Such data are lacking for

Impact

• Authors receive an automatic acknowledgement of receipt of their submission immediately upon completion

• Paper is in the hands of the Editor faster

• Editors search for reviewers by expertise taxonomy keyword

• Reviewers access manuscripts immediately and submit reviews online enabling Editors to have access to reviews immediately

• Automatic reminders sent out to Reviewers and Editors

• Editors now have access to all data from their desktop regardless of their geographic location

Impact

• Editorial staff had to learn how to work differently

• Check PDF files for completeness and provide support to authors and reviewers using the system

• No longer piles of mail on their desks

• Many e-mails

• Increase in submissions

Submissions

0

1000

2000

3000

4000

5000

6000

1999 2000 2001 2002 2003 2004 2005 2006

Year

Nu

mb

er

of

new

su

bm

issio

ns

Current Events

• Usability Study– more clearly understand the users’ expectations of the system– verify the types of tasks that users’ complete, and identify how

well OSPREY supports those tasks– identify patterns to the types of usability issues that maybe

occurring among users

• Methodology– hired usability expert– analysis to be completed July 2007– heuristic evaluation of interface– surveys, interviews, online interactive exercises

Future Work

• Automatic workflow from OSPREY to Workflow Management System

• Parsing and tagging references to provide a link to abstract databases or full text (e.g. PubMed, CrossRef, and user organization link resolver)

• Option for authors to identify papers for open access and supply payment or funding

• Ability to access tasks directly from e-mail

• Integration with external databases

• Troubleshooting of common graphic file problems

Future Work

OSPREY Publishing System

ManuscriptSubmission

PublishingWorkflow

XML publishingprocess

ApproveProofs

Order Reprints

Printer

Thank You

Judy Best

Judy.best@nrc-cnrc.gc.ca

http://www.nrc.ca/