us national academies study on reproducibility in scientific results · 2019-04-09 · scnt-somatic...
TRANSCRIPT
Xia & He Publishing
US National Academies Study on Reproducibility in Scientific Results
Harry Hua-Xiang Xia, Ph. D, M. MedFounder, Medjaden Bioscience (2005, Hong Kong; 2009,
Mainland China; 2014, USA)Co-Founder, Xia & He Publishing (2011, Hong Kong; 2015, USA)Editor-in-Chief, Journal of Clinical and Translational Hepatology
(Indexed by PubMed & ESCI)Adjunct Professor, First Affiliated Hospital, Guangdong
Pharmaceutical University, Guangzhou, ChinaPresident, Alliance for Scientific Editing in China (ASEC)
May 31, 2018, Washington DC, USA 1
Xia & He Publishing
1985-1990, Union Hospital, Tongji Medical College, HUST, Wuhan
1990-1995, St. James Hospital, Trinity College, Dublin University, Dublin
1995-2000, Nepean Hospital, The University of Sydney, Sydney
2000-2006, Queen Mary Hospital, The University of Hong Kong, Hong Kong
2006-2014, Novartis Pharmaceuticals Corporation, East Hanover, NJ
2014-Present, Medjaden Bioscience (Hong Kong,China)Xia & He Publishing, TX 2
My Research and Academic Career
Xia & He Publishing
Q1a: Are there specific examples in your country/region where a lack of reproducibility and replicability in research results has led to doubt about reported results more broadly?
www.nature.com/news/replications-ridicule-and-a-recluse-the-controversy-over-ngago-gene-editing-intensifies-1.20387
Xia & He Publishing
Q1a: Are there specific examples in your country/region where a lack of reproducibility and replicability in research results has led to doubt about reported results more broadly?
www.nature.com/news/replications-ridicule-and-a-recluse-the-controversy-over-ngago-gene-editing-intensifies-1.20387
Xia & He Publishing
Crisis or not: reproducibility and replicability in the scientific world
More than 70% of researchers have tried and failed to reproduce another scientist's experiments;
More than 50% have failed to reproduce their own experiments;
52% of those surveyed agree that there is a significant 'crisis' of reproducibility;
Less than 31% think that failure to reproduce published results means that the results are probably wrong.
Q1b: Are reproducibility and replication of research results a global concern or is it a concern focused within specific countries?
Baker M. 1,500 scientists lift the lid on reproducibility: Survey sheds light on the ‘crisis’ rocking research. Nature, 2016 May, 533(7604):452-4
Xia & He Publishing
Reasons for the failure of reproducibility and replicability
“Publish or perish” research environment;
Insufficient training for the researchers (studyprocedures, statistics, etc.);
Selective use of data, inappropriate study conditions, etc.
Academic misconduct: Fraud (Fabrication, falsification, etc.)
Q1b: Are reproducibility and replication of research results a global concern or is it a concern focused within specific countries?
Xia & He Publishing
Analyses of retractions of scientific publications
Q1b: Are reproducibility and replication of research results a global concern or is it a concern focused within specific countries?
Figure 1(A) Number of retracted
articles for specific causes by year of retraction.
(B) Percentage of published articles retracted for fraud or suspected fraud by year of publication.
Fang F. Misconduct accounts for the majority of retracted scientific publications PNAS 2012;109(42):17028
Xia & He Publishing
Q1b: Are reproducibility and replication of research results a global concern or is it a concern focused within specific countries?Analyses of retractions of scientific publications
Figure 2. Country of origin of publications retracted for fraud or suspected fraud (A), plagiarism (B), or duplicate publication (C).
Fang F. Misconduct accounts for the majority of retracted scientific publications, PNAS. 2012;109(42):17028
Xia & He Publishing
Q1b: Are reproducibility and replication of research results a global concern or is it a concern focused within specific countries?
Rank CountryPapers retracted
Retraction rate(per 10,000)
1 USA 1065 2.92 China 313 1.83 UK 174 2.04 Germany 167 1.75 Japan 162 2.06 India 162 3.47 The Netherland 117 3.48 Canada 109 1.89 Italy 102 1.810 Australia 88 1.9
August 2010-November 11, 2016 http://retractionwatch.com/
Retraction by countries
Xia & He Publishing
Q1b: Are reproducibility and replication of research results a global concern or is it a concern focused within specific countries?
Retraction by individual authors
November 11, 2016 http://retractionwatch.com/the-retraction-watch-leaderboard/
Yoshitaka Fujii (藤井 善隆)A Japanese report concludes: “It is as if someone sat at a desk and wrote a novel about a research idea.”
Xia & He Publishing
Q1b: Are reproducibility and replication of research results a global concern or is it a concern focused within specific countries?Dutch former professor of social psychology
Xia & He Publishing
Q1b: Are reproducibility and replication of research results a global concern or is it a concern focused within specific countries?
Hwang Woo-suk (黄禹锡), South KoreaSCNT - Somatic cell nuclear transferHwang WS, et al. Evidence of a pluripotent human embryonic stem cell line derived from a cloned blastocyst. Science. 2004 Mar 12; 303 (5664): 1669–74. doi:10.1126/science.1094515. PMID 14963337. (Retracted, see PMID 16410485)
Haruko Obokata (小保方 晴子), JapanSTAP - Stimulus-triggered acquisition of pluripotency1. Obokata H, et al. Stimulus-triggered fate conversion of somatic cells
into pluripotency. Nature. 2014 Jan 30;505(7485):641-7. doi: 10.1038/nature12968. Retraction in: Nature. 2014 Jul 3;511(7507):112. PMID: 24476887
2. Obokata H, et al. Bidirectional developmental potential in reprogrammed cells with acquired pluripotency. Nature. 2014 Jan 30;505(7485):676-80. doi: 10.1038/nature12969. Retraction in: Nature. 2014 Jul 3;511(7507):112. PMID: 24476891
Xia & He Publishing
Q1b: Are reproducibility and replication of research results a global concern or is it a concern focused within specific countries?
Based on the above evidence, I believe that reproducibility and replication of research results are a global concern!
Xia & He Publishing
Our research on perceptions of Chinese medicalresearchers on misconduct in China
Liao QJ, Zhang YY, Fan YC, Zheng MH, Bai Y, Eslick GD, He XX, Zhang SB, Xia HH, He H. Perceptions of Chinese Biomedical Researchers Towards Academic Misconduct: A Comparison Between 2015 and 2010. Sci Eng Ethics. 2018 Apr;24(2):629-645. doi: 10.1007/s11948-017-9913-3
Xia & He Publishing
5.3
37.6
28.3
19.7
9.0
3.9
42.3
32.6
14.7
6.6 4.4
40.7
31.1
16.4
7.4
0.0
10.0
20.0
30.0
40.0
50.0
Not serious Somewhat serious Serious Very serious Extremely serious
Perc
enta
ge (%
)
Current status of misconduct
2010
2015
total
Our research: perceptions on current academic misconduct in Mainland China
Overall, 55% of participants thought that academic misconduct was serious, very serious or extremely serious.
Xia & He Publishing
22.87
12.78
26.56
15.87
25.27
14.7916.82
33.63
20.07
30.05
18.94
31.30
17.49
24.44
19.23
27.64
18.62
26.53
9.64
16.82
7.45
13.70
8.22
14.79
30.94
10.99
25.96
12.26
27.70
11.82
0.00
10.00
20.00
30.00
40.00
1st Choice 2nd Choice 1st Choice 2nd Choice 1st Choice 2nd Choice
2010 2015 total
The most common form of misconduct
Plagiarism Falsification Fabrication Duplication Authorship
Our research: most common forms of academic misconduct in mainland China
Impropriate authorships and plagiarism were the most common forms of academic misconduct, followed by falsification and fabrication.
Xia & He Publishing
Our research: reasons behind academic misconduct in the Mainland China
52.69
31.61
49.04
31.01
50.31
31.22
7.40
47.53
9.13
43.39
8.53
44.84
33.18
13.23
34.13
16.83
33.80
15.57
0.45 2.020.24 0.96 0.31 1.33
4.26 3.146.13 7.33
5.48 5.87
0.00
20.00
40.00
60.00
1st Choice 2nd Choice 1st Choice 2nd Choice 1st Choice 2nd Choice
2010 2015 total
Reasons behind misconduct
National system Institutional system Social environment Family influence Individual behaviors
The most important reason behind academic misconduct was the academic/research assessment system, followed by the social environment
Xia & He Publishing
Our research: estimation of percentage of publication involving academic misconduct in Mainland China
Participants estimated that 40.1% (39.8 ± 23.5% in 2010; 40.2 ± 24.5% in 2015) of published scientific articles were associated with some form of academic misconduct.
Year Answer (%)
0-20% 20.1-40% 40.1-60% 60.1-80% 80.1-100%
2010 29.9 27.2 25.8 13.2 3.9
2015 28.9 27.9 25.5 13.1 4.7
Total 29.2 27.6 25.6 13.1 4.5
Percentage of publication involving misconduct
Xia & He Publishing
Our research: Conclusions
Conclusions The perceptions of Chinese biomedical
researchers towards academic misconduct had not significantly changed from 2010 to 2015.
Reform of the academic assessment system should be the fundamental approach to tackling this problem in China.
Xia & He Publishing
Q2 Are there particular scientific fields in which lack of reproducibility and replicability is more/less of a concern?
More Life Science;Medicine
Less Engineering Business
Xia & He Publishing
Measures taken by Chinese government to tackle academic misconduct (but not specifically reproducibility and replicability)
Suspend research projects;
Cancel grants and rescind promotion;
Demote, or dismiss from the office;
Reform in the academic assessment system
Q3 Are there any concrete actions that organizations (e.g. funders, publishers, societies) in your country or region have taken to address concerns about reproducibility and replicability? What actions should they take?
http://www.moe.edu.cn/srcsite/A02/s5911/moe_621/201607/t20160718_272156.html
Xia & He Publishing
Proposed actions to tackle the problem Generate the ”reproducibility index” for journals, similar
to an impact factor;
Reproducibility index = ?
Regulate funding bodies
Improve training for researchers
Radom results audits
Q3 Are there any concrete actions that organizations (e.g. funders, publishers, societies) in your country or region have taken to address concerns about reproducibility and replicability? What actions should they take?
https://theconversation.com/science-is-in-a-reproducibility-crisis-how-do-we-resolve-it-16998
Xia & He Publishing
Yes, both regionally and globally.
Priorities
1) Modify the academic assessment system to be less publication-orientated;
2) Increase the awareness of prevention of academic misconduct through courses, workshop and training;
3) Increase the transparency of doing research, which can be referred as open science.
Q4 Should the research community work regionally and/or globally to address concerns about reproducibility and replicability? If so, what should be the priorities?
https://theconversation.com/science-is-in-a-reproducibility-crisis-how-do-we-resolve-it-16998
Xia & He Publishing
A Comprehensive Guide to English Medical Manuscript Writing and Publication
Huaxiang Xia, Yuanyuan Zhang. A Comprehensive Guide to English Medical Manuscript Writing and Publication. Wuhan: HUST Press, 2017
Xia & He Publishing
A Comprehensive Guide to English Medical Manuscript Writing and Publication
Huaxiang Xia, Yuanyuan Zhang, A Comprehensive Guide to English Medical Manuscript Writing and
Publication HUST Press, Wuhan, 2017
Xia & He Publishing
Acknowledgments
Selin Hua He. M. PhilCEO MedjadenXia & HE Publishing
Sarah Yuanyuan Zhang, Ph. DAcademic DirectorMedjadenXia & HE Publishing
Xia & He Publishing
Welcome to China
The Second International Summit on Medical Research and Publication
September 16, 2018, Wuhan, ChinaHosted by Xia & He Publishing, Medjaden & Tongji Medical College
http://www.xiahepublishing.com/MeetingDetail.aspx?sid=3&id=5