final presentation group 2 porto, 2 nd may 2007 medical faculty of oporto

Final PresentationGroup 2

Porto, 2nd May 2007Medical Faculty of Oporto

1. Introduction

2. Methods

3. Results

4. Discussion and Conclusions

5. Acknowledgements

As an extremely dynamic field, medicine evolves each day, integrating new ideas and concepts which come from all over the world

Gastroenterology and hepatology are medical specializations concerned with the diagnosis and treatment of diseases of the digestive tract and organs

Diagnostic tests play a crucial role in the accurate diagnose of diseases

The number of existing diagnostic tests has grown over the past few years1

In January 2003, the guide for reporting studies of diagnostic accuracy Standards for Reporting of Diagnostic Accuracy (STARD) was published1

The objective of the STARD initiative is to improve the quality of reporting of studies of diagnostic accuracy1

1- Bossuyt, P; Reitsma, J; Bruns, D; Gatsonis, C; Glasziou, P; Irwig, L; Lijmer, J; Moher, D; Rennie, D; de Vet, H, Towards complete and accurate reporting o studies of diagnostic accuracy: the STARD initiative; Annals of internal medicine; Volume 138, 2003.

Bossuyt, P; Reitsma, J; Bruns, D; Gatsonis, C; Glasziou, P; Irwig, L; Lijmer, J; Moher, D; Rennie, D; de Vet, H, Towards complete and accurate reporting o studies of diagnostic accuracy: the STARD initiative; Annals of internal medicine; Volume 138, 2003.

Objectives

Evaluate the quality of the articles which evaluate diagnostic tests in Gastroenterology and Hepatology on journals with the highest impact factor in the past 12 years based on the STARD checklist, and unveil if there was evolution in this type of studies

Assess if the STARD initiative had influence in the articles or not

1 - Bossuyt, P; Reitsma, J; Bruns, D; Gatsonis, C; Glasziou, P; Irwig, L; Lijmer, J; Moher, D; Rennie, D; de Vet, H, Towards complete and accurate reporting o studies of diagnostic accuracy: the STARD initiative; Annals of internal medicine; Volume 138, 2003.

1. Introduction

2. Methods

3. Results

4. Discussion and Conclusions

5. Acknowledgements

1. Article search

2. Selection of abstracts

3. Article evaluation

Initial Query for studies of diagnostic accuracy

Definition of inclusion parameters

Restriction to the five journals of gastroenterology and hepatology with the highest impact factor

Restriction to diagnostic tests in humans

Initial Query

(((((((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields)) OR (((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All fields]) OR "roc and"[All fields]) OR "roc area"[All fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields])) OR "likelihood ratio"[All Fields])

Ref:Dévillé WL, Buntins F, Bouter LM, Montori, VM, de Vet H, Windt DAWM, Bezemer PD. Conducting systematic reviews of diagnostic studies: didactic guidelines, BMC medical research methodology 2002, 2:9

Initial Query

(((((((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields)) OR (((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All fields]) OR "roc and"[All fields]) OR "roc area"[All fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields])) OR "likelihood ratio"[All Fields]) AND (Humans[MeSH])

Year 1994

Articles Found – 174

Query parameters:

(…) 1994 [Publication Date] AND (GASTROENTEROLOGY [Journal] OR HEPATOLOGY [Journal] OR GUT [Journal] OR GASTROINTEST ENDOSC [Journal] OR J HEPATOL [Journal])

1994 – 174 1995 – 144

1996 – 168

1997 – 167

1998 – 129

1999 – 180

2000 – 170

2001 – 144

2002 – 165

2003 – 140

2004 – 261

2005 – 299

Journals with highest occurence:

•Gastroenterol•Hepatol•Gut

1. Article search

2. Selection of abstracts

3. Article evaluation

• Non-excluded and non-included abstracts were listed along with reasons of non-inclusion and added to the data base on SPSS

1. Article search

2. Selection of abstracts

3. Article evaluation

1. Introduction

2. Methods

3. Results

4. Discussion and Conclusions

5. Acknowledgements

Does not assess diagnostic testsMissing abstractOther

Reasons for non-inclusion

102 abstracts not excluded from 300 found

Comparison between excluded and included articles

Agreement between abstract reviewers

Rate of agreement between reviewers – 84,3%

Does not assess diagnostic testsMissing article

Exclusion reasons (%)

78 articles included from 102 abstracts not excluded

Comparison between excluded and included articles

Agreement between article reviewers

1 - Smidt, N.; Rutjes, A.; van der Windt, D.; Ostelo, R.; Bossuyt, P; Reitsma, J.; Bouter, L; de Vet, H; Reproducibility of the STARD checklist: an instrument to assess the quality of reporting of diagnostic accuracy studies; BMC medical research methodology; 2006

Obtained ICC value is in accordance with ICC to be expected by Smidt, N.

Expected ICC = 0,79 [95% CI: 0,62 to 0,89]1

Evolution of the article’s score by mean

Evolution of the article’s score by median

Items with highest absence (i13, i24) refer to reproducibility issues

Individual STARD items absence percentage

1. Introduction

2. Methods

3. Results

4. Discussion and Conclusions

5. Acknowledgements

STARD did not had a significant effect in the article’s quality

• Insufficient time span between the STARD publishing (2003) and the current date

• The small evolution in the article’s quality is likely to be a consequence of other factors, not taken into account in this study

This study should be repeated in a wider time span

The quality of the this kind of articles in the journals with the highest impact factors is not as positive as expected

• The credibility of these articles may be compromised

• The results presented may be misleading

The STARD items referring to reproducibility issues are highly absent

• The studies might be difficult to reproduce, which may lead to difficulties in the repeating of the studies by other researchers and therefore may impede verification of results

• There is the possibility of the diagnostic tests assessed to be inaccurate

1. Introduction

2. Methods

3. Results

4. Discussion and Conclusions

5. Acknowledgements

• Cristina Santos, MD

• Altamiro da Costa Pereira, PhD MD

Porto, 2nd May 2007Medical Faculty of Oporto

OrientationCristina Santos, MD

SupervisorAltamiro da Costa Pereira, PhD MD

Authors