spectrum bias of studies on diagnostic tests

I Congresso Brasileiro de Prática Clínica Baseada em Evidências

Viés de espectro em estudos que avaliam a performance de testes diagnósticos.

Spectrum bias in studies on diagnostic tests

Luz M. Letelier MD; FACPAssociate Professor

Pontificia Universidad Católica de Chile

OBJECTIVES

Analyze how an inappropriate spectrum of disease may bias the results of a study on the performance of a diagnostic test.

Understand spectrum of disease.

Spectrum of disease

• When do we use a diagnostic test?

• What do we expect a test to do, when we apply it?

Likelihood of a disease1) Maria A: 26 year old, feeling “nervous” for 2

months, she’s lost 10 kilos, has tremor, increased sweating and 5 bowel movements/day. On exam: tremor, tachycardia 110/min, goiter, thyroid bruit and lid lag.

2) Maria B: 26 year old, feeling “nervous” for 2 months, feels short episodes of palpitations every day and has insomnia. PA 142/88. Thyroid somewhat enlarge.

3) Maria C: 26 year old woman feeling “nervous” for 2 months, no other symptoms, but tells you she is moving to another country due to her husband’s new job. No remarkable findings on exam.

Diagnostic Process

Diagnostic threshold

Therapy thresholdUncertainty area

0% 100%

Diagnostic process for the likelihood of disease

Diagnostic threshold Therapy threshold

Uncertainty area

Maria B Maria A

Maria A Maria B HyperthyroidismMaria CMaria C

Diagnostic test

• We use a diagnostic test to move the probability of having a condition or not having the condition when there is uncertainty.

Diagnostic test

• Assuming we use a test when we are uncertain regarding the condition status.

• We need the test to discriminate between those who do from those who do not have the condition.

Spectrum of disease

Distribution of people without hyperthyroidism

Distribution of people with hyperthyroidism

How can an inappropriate spectrum of patients influence the results of a study on the performance of a diagnostic test?

Let’s assume we want to validate a new test for morbid obesity:

THE RING SIZE

We prospectively recruit 100 women

Compare “the ring size” (our test) to the Gold Standard: Body Mass Index (BMI).If the ring size is > 10 the test is (+)If the ring size is <10 the test is (-)

BMISuzana Luz

40

THE RING SIZE TEST

Of the 100 women recruited: 70 look like Suzana 30 and look like Luz

CONDITION MORBID OBESITY

SI NO

TESTRING SIZE> 10

(+)

(-)

30 70 100

SuzanaLuz

CONDITION MORBID OBESITY

SI NO

TESTRING SIZE> 10

(+) 30 0 30

(-) 0 70 70

30 70 100

CONDITION MORBID OBESITY

SI NO TESTRING SIZE

(+) 30 0 30

(-) 0 70 7030 70

0 False positive and 0 false negative SENSITIVITY = 100%

SPECIFICITY = 100%PERFECT TEST!!!!

RING SIZE TEST

Now we have a perfect test that correctly identifies all morbid obese patients from all of those who are not. BUT will it perform as good in a sample of women with real uncertainty…….?

TRUE OR FALSE

• The ring test is the perfect test.

• We chose an inappropriate spectrum of patients.

F

V

Studies on diagnostic tests might render biased results when:

An inappropriate spectrum of patients is chosen.

ESPECTRUM OF DISEASE AND TEST RESULTS

D-Dimer

Normal individuals Very sick patients

600400

D-Dimer

D Dimer

Possibly sick Not severe sickness

D-Dimer

Possibly sick Non severe sick

600400

Spectrum of disease

BREAST CANCER DIAGNOSIS

Diagnostic threshold

Therapy threshold

TO REMEMBER

Diagnostic tests should be validated on patients with diagnostic uncertainty.

REMEMBER

• The appropriate spectrum of patients is crucial to reduce bias in a study on the performance of a diagnostic test.

• The appropriate spectrum of patients are those who have diagnostic uncertainty regarding the condition under study.

OBRIGADA

Background: Survivin is a member of the family of inhibitor of apoptosis proteins that is overexpressed in several human tumors. Previous studies have found that overexpression of survivin in cancer cells induces an antibody response. Methods: We compared 232 serum samples from colorectal cancer (CRC) patients and 365 samples from healthy volunteers using an in vitro enzyme-linked immunosorbent assay to evaluate the survivin autoantibody response in patients.

Conclusions Our results suggest that analysis of both parameters would assist in screening patients with CRC.

CHEN JS, CHEN KT et al. Combined analysis of survivin autoantibody and carcinoembryonic antigen biomarkers for improved detection of colorectal cancer.Clin Chem Lab Med. 2010 Feb 24.