Research Integrity in ASHA:Scientific Publication Practices

2006 ORI Research on Research Integrity Conference

University of South FloridaSafety Harbor Resort and Spa - Tampa, FLDecember 1-3, 2006

Janis Costello Ingham, Ph.D.University of California – Santa Barbara

Sharon E. Moss, Ph.D.American Speech-Language-Hearing Association

Co-Presenters Dean C. Garstecki, Northwestern University

Jennifer Horner, Medical University of South Carolina

Joanne Jessen, ASHA

Charissa R. Lansing, University of Illinois, Urbana - Champaign

James H. McCartney, California State University Fred D. Minifie, University of Washington (retired) Randall R. Robey, University of Virginia

Sarah Slater, ASHA

This research was supported by the Research on Research Integrity Program, an ORI/NIH collaboration, grant # RO1 N544534-0151

Session Outline

Purpose of StudyMethodology

ResultsConclusionsQuestions???

Purpose of Study

The purpose of this research survey was to --

sample CSD academic and research community

importance and adequacy of research integrity topics

publications in ASHA’s scientific journals

Methodology

Instrument Development

Pilot Testing

Fielding/Survey Population

Methodology: INSTRUMENT DEVELOPMENT

Methodology: INSTRUMENT DEVELOPMENT

Methodology: Instrument Development

Methodology: Instrument Development

Methodology: PILOT TESTING

Cognitive interviews Field tested

Methodology: FIELDING/SURVEY POPULATION

Editor, Associate Editor, Reviewer Publications Board Member Board of Ethics Member Contributing Scientist / Author General Readership Graduate Student

Results:RANKINGS OF IMPORTANCE

The survey Q: In regard to scientific integrity in research publications in general, how important is ______ ?

Scale: 1-10 (1 = Not At All Important and 10 = Critically Important) or, Do Not Know

RESULTS: The TOP TEN (derived from Group 1: Editors, Associate Editors, Pub Board) 1 – Fabrication of data

2 – Falsification of data

3 – Separation of advertising sales from scholarly

content decisions

4 - Plagiarism

5 – Criteria for accepting advertising

The TOP TEN, continued

6 – Maintenance of retracted papers in electronic archives

7 – Publication of retractions and errata

8 – Confidentiality of the peer review process

9 – Disclosure to Editor of author conflicts of interest

10 – Author declaration to Editor of adherence to HIPAA requirements

The TOP TEN, continued

There was high agreement in ranking of items between Group 1 (Editors, Associate Editors, Publication Board members) and Group 2 (reviewers, authors, and Board of Ethics members) (See handout, blue items)

Rank order correlation was 0.94.

Mean highest rankings were 9.71, 9.74 and 9.70, respectively

The TOP TEN, continued

Within the Top Ten, readers judged three items to be considerably less important than the other judge groups:

Criteria for accepting advertising

Confidentiality of the peer review process

Disclosure to the Editor of author conflicts of interest

RESULTS: The “BOTTOM TEN”

56 – Authors’ ID during the review process

55 – Requirement that copyright be transferred to ASHA

54 – Authors’ responsibility to report errors for correction

53 – Explicit statement of humane care and Treatment of animals in the published MS

52 – Evaluation of reviewers

The BOTTOM TEN, continued 51 – Reviewers’ evaluation of MSs previously

reviewed for another journal

50 – Data sharing for meta-analyses

49 – Publication of previously published data

48 – Reviewers’ ID during the review process

47 – Authors’ declaration to Editor of adherence to IRB requirements

The BOTTOM TEN, continued There was good agreement in ranking of

items among the three groups.

Mean lowest rankings were: 4.66, 6.11 and 7.12, for Groups 1, 2, and 3, respectively.

THE “READER” GROUP

Throughout the data, where incongruent

ratings occurred, they were typically generated by the Reader group (18 items out of 56 total). (See red items on the handout.)

Rank order correlations:Group 1 vs. Group 3 = 0.86Group 2 vs. Group 3 = 0.83

ASHA’s Publication Policy Documents Publication Manual of the American

Psychological Association ASHA’s Editor’s Handbook ASHA’s Information for Reviewers ASHA’s Instructions for Authors ASHA’s Reviewer Agreed Letter ASHA’s Code of Ethics

Adequacy of Coverage of Publication Ethics Issues in ASHA’s Policy Documents Expert judges identified all sections judged to

be potentially relevant to the survey’s 56 items from all six policy documents

Panel of 6 expert judges rated each identified item from each document 0 = item not relevant to the given topic 1 = only tangentially relevant 2 = partially relevant, but not completely

adequate 3 = adequate coverage/discussion of the topic

4 or more rankings of “3” = topic adequately covered

Adequacy of Coverage of Publication Ethics Issues in ASHA’s Policy

Documents Eight of the Top Ten items were judged to

be discussed adequately in at least one of ASHA’s policy documents. (See handout.)

However, of the total 56 items, 28 (50%) were judged not to be adequately discussed, or were completely ignored. (See handout.)

Adequate coverage of specific items is distributed across five documents, sometimes singly; sometimes redundantly. (See handout.)

Conclusions

Develop a comprehensive document.

Make recommendations to Publications Board and Board of Ethics.

Develop additional policies.

Educate the research and reader membership.

Use these findings as a baseline for the future.

Questions