CONFIDENCE INTERVALSCONFIDENCE INTERVALS

DR.S.SHAFFI AHAMEDDR.S.SHAFFI AHAMED

ASST. PROFESSORASST. PROFESSOR

DEPT. OF F & CMDEPT. OF F & CM

BACKGROUND AND NEED OF CIBACKGROUND AND NEED OF CI

---Statistical analysis of medical ---Statistical analysis of medical studies is based on the key idea studies is based on the key idea that, we make observations on a that, we make observations on a sample of subjects from which the sample of subjects from which the sample is drawn.sample is drawn.

---If the sample is not representative ---If the sample is not representative of the population we may well be of the population we may well be misled and statistical procedures misled and statistical procedures cannot help.cannot help.

. .

---Even a well designed study can ---Even a well designed study can give only an idea of the answer give only an idea of the answer sought , because of random sought , because of random variation in the sample.variation in the sample.

---And the results from a single ---And the results from a single sample are subject to statistical sample are subject to statistical uncertainty, which is strongly uncertainty, which is strongly related to the size of the sample. related to the size of the sample.

-- -- The quantities( single mean, The quantities( single mean, proportion, difference in means, proportion, difference in means, proportions, OR, RR, proportions, OR, RR, Correlation,-----------) will be Correlation,-----------) will be imprecise estimate of the values in imprecise estimate of the values in the overall population, but the overall population, but fortunately the imprecision can fortunately the imprecision can itself be estimated and itself be estimated and incorporated into the presentation incorporated into the presentation of findings.of findings.

--- Presenting study findings directly on the --- Presenting study findings directly on the scale of original measurement together scale of original measurement together with information on the inherent with information on the inherent imprecision due to sampling variability, imprecision due to sampling variability, has distinct advantages over just giving has distinct advantages over just giving ‘p-values’ usually dichotomized into ‘p-values’ usually dichotomized into “significant” or “non-significant”. “significant” or “non-significant”.

““THIS IS THE RATIONALE FOR THIS IS THE RATIONALE FOR USING CI”USING CI”

Confidence Intervals for Reporting Confidence Intervals for Reporting ResultsResults

““[Hypothesis tests] are sometimes overused and [Hypothesis tests] are sometimes overused and their results misinterpreted.”their results misinterpreted.”

““Confidence intervals are of more than Confidence intervals are of more than philosophical interest, because their broader use philosophical interest, because their broader use would help eliminate misinterpretations of would help eliminate misinterpretations of published results.”published results.”

““Frequently, a significance level or pvalue is Frequently, a significance level or pvalue is reduced to a ‘significance test’ by saying that if the reduced to a ‘significance test’ by saying that if the level is greater than 0.05, then the difference is ‘not level is greater than 0.05, then the difference is ‘not significant’ and the null hypothesis is ‘not significant’ and the null hypothesis is ‘not rejected’….The distinction between statistical rejected’….The distinction between statistical significance and clinical significance should not be significance and clinical significance should not be confused.”confused.”

But why do we always see But why do we always see 95% CI’s?95% CI’s? ““Duality” between confidence intervals and Duality” between confidence intervals and

pvaluespvalues ExampleExample: Assume that we are testing that for a : Assume that we are testing that for a

significant change in QOL due to an intervention, where significant change in QOL due to an intervention, where QOL is measured on a scale from 0 to 50. QOL is measured on a scale from 0 to 50. 95% confidence interval: (-2, 13)95% confidence interval: (-2, 13) pvalue = 0.07pvalue = 0.07

It is true that if the 95% confidence interval overlaps 0, It is true that if the 95% confidence interval overlaps 0, then a t-test testing that the treatment effect is 0 will be then a t-test testing that the treatment effect is 0 will be insignificant at the alpha = 0.05 level.insignificant at the alpha = 0.05 level.

It is true that if the 95% confidence interval does not It is true that if the 95% confidence interval does not overlap 0, then a t-test testing that the treatment effect overlap 0, then a t-test testing that the treatment effect is 0 will be significant at the alpha = 0.05 level.is 0 will be significant at the alpha = 0.05 level.

--THE BRITISH MEDICAL JOURNALTHE BRITISH MEDICAL JOURNAL-THE LANCET-THE LANCET-THE MEDICAL JOURNAL OF -THE MEDICAL JOURNAL OF

AUSTRALIAAUSTRALIA-THE AMERICAN JOURNAL OF PUBLIC -THE AMERICAN JOURNAL OF PUBLIC

HEALTHHEALTH-THE BRITISH HEART JOURNAL-THE BRITISH HEART JOURNAL--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Different Interpretations of the Different Interpretations of the 95% confidence interval95% confidence interval

““We are 95% sure that the TRUE parameter We are 95% sure that the TRUE parameter value is in the 95% confidence interval”value is in the 95% confidence interval”

““If we repeated the experiment many many If we repeated the experiment many many times, 95% of the time the TRUE parameter times, 95% of the time the TRUE parameter value would be in the interval”value would be in the interval”

““Before performing the experiment, the Before performing the experiment, the probability that the interval would contain the probability that the interval would contain the true parameter value was 0.95.”true parameter value was 0.95.”

-- -- In a representative sample of 100 observations In a representative sample of 100 observations of heights of men, drawn at random from a large of heights of men, drawn at random from a large population, suppose the sample mean is found population, suppose the sample mean is found to be 175 cm (sd=10cm) .to be 175 cm (sd=10cm) .

-- Can we make any statements about the -- Can we make any statements about the population mean ?population mean ?

-- We cannot say that population mean is 175 cm -- We cannot say that population mean is 175 cm because we are uncertain as to how much because we are uncertain as to how much sampling fluctuation has occurred.sampling fluctuation has occurred.

-- What we do instead is to determine a range of -- What we do instead is to determine a range of possible values for the population mean, with possible values for the population mean, with 95% degree of confidence.95% degree of confidence.

-- -- This range is called the 95% confidence interval This range is called the 95% confidence interval and can be an important adjuvant to a and can be an important adjuvant to a significance testsignificance test..

In general, the 95% confidence interval is given In general, the 95% confidence interval is given by:by:

Statistic ± confidence factor x S.Error of statisticStatistic ± confidence factor x S.Error of statistic

In the previous example, n =100 ,sample mean = In the previous example, n =100 ,sample mean = 175, S.D., =10, and the S.Error =10/√100 = 1.175, S.D., =10, and the S.Error =10/√100 = 1.

Therefore, the 95% confidence interval is,Therefore, the 95% confidence interval is,

175175 ± 1.96 * 1 = 173 to 177”± 1.96 * 1 = 173 to 177”

That is, if numerous random sample of size 100 are That is, if numerous random sample of size 100 are drawn and the 95% confidence interval is drawn and the 95% confidence interval is computed for each sample, the population mean computed for each sample, the population mean will be within the computed intervals in 95% of will be within the computed intervals in 95% of the instances.the instances.

Sampling DistributionsSampling Distributions

2.0 2.5 3.0 3.5 4.0

05

10

15

samps

n = 25

2.0 2.5 3.0 3.5 4.0

05

10

15

20

25

samps

n = 50

2.0 2.5 3.0 3.5 4.0

01

02

03

0

samps

n = 100

2.0 2.5 3.0 3.5 4.0

05

10

15

20

samps

n = 500

sem = 0.47

sem = 0.10

sem = 0.23

sem = 0.17

x sem 1 9 6.

xs

n 1 9 6.

General formula for 95% confidence General formula for 95% confidence interval for single meaninterval for single mean

Notes:Notes: sample size must be sufficiently large for sample size must be sufficiently large for

non-normal variables.non-normal variables. how large is large? depends on how large is large? depends on

skewness of variableskewness of variable VERY often people use 2 instead of 1.96.VERY often people use 2 instead of 1.96.

Example:Example: A study was carried out to determine the A study was carried out to determine the effect, if any, of pesticide exposure on blood effect, if any, of pesticide exposure on blood pressure. A random sample of 100 men was pressure. A random sample of 100 men was selected from a group of agriculture workers selected from a group of agriculture workers known to have been exposed to pesticides. 100 known to have been exposed to pesticides. 100 randomly selected workers with no such randomly selected workers with no such exposure comprised the control group. Their exposure comprised the control group. Their mean SBP and Sd., were given as 145 mm Hg, 20 mean SBP and Sd., were given as 145 mm Hg, 20 mm hg (exposed group) and 120 mm hg, 15 mm mm hg (exposed group) and 120 mm hg, 15 mm Hg (non exposed group). Calculate the 95% and Hg (non exposed group). Calculate the 95% and 90% confidence intervals for the true difference 90% confidence intervals for the true difference in mean SBP between the exposed and non in mean SBP between the exposed and non exposed populations .exposed populations .

95 % C.I. for Difference in 95 % C.I. for Difference in Means Means

( ) .x xs

n

s

n1 212

1

22

2

1 9 6

The 90% confidence interval for the difference in The 90% confidence interval for the difference in means :means :

25 ± 1.645 (2.5) = 20.9 to 29.125 ± 1.645 (2.5) = 20.9 to 29.1

The 95 % confidence interval for the difference in The 95 % confidence interval for the difference in means:means:

25 ± 1.96 (2.5) = 20.1 to 29.925 ± 1.96 (2.5) = 20.1 to 29.9

Thus we can be 90% (or 95%) certain that the true Thus we can be 90% (or 95%) certain that the true mean difference in systolic blood pressure mean difference in systolic blood pressure between the exposed and non-exposed between the exposed and non-exposed populations lies between 21 and 29 mm Hg (20 populations lies between 21 and 29 mm Hg (20 and 20 mm Hg). and 20 mm Hg). Notice that the both the intervals Notice that the both the intervals does not include zero so the data are not does not include zero so the data are not compatible with no difference in mean systolic compatible with no difference in mean systolic blood pressure.blood pressure.

95% Confidence Intervals for 95% Confidence Intervals for ProportionsProportions Socinski et al., Socinski et al., Phase III Trial Comparing a Defined Duration Phase III Trial Comparing a Defined Duration

of Therapy versus Continuous Therapy Followed by of Therapy versus Continuous Therapy Followed by Second-Line Therapy in Advanced-Stage IIIB/IV Non-Small-Second-Line Therapy in Advanced-Stage IIIB/IV Non-Small-Cell Lung Cancer Cell Lung Cancer JCO, March 1, 2002.JCO, March 1, 2002.

Patients and MethodsPatients and Methods: Arm A (4 cycles of carboplatin at an : Arm A (4 cycles of carboplatin at an AUC of 6 and paclitaxel), Arm B (continuous treatment with AUC of 6 and paclitaxel), Arm B (continuous treatment with carboplatin/ paclitaxel until progression). At progression, carboplatin/ paclitaxel until progression). At progression, patients from each arm receive second-line weekly patients from each arm receive second-line weekly paclitaxel at 80mg/m2/week.paclitaxel at 80mg/m2/week.

ResultsResults: 230 Patients were randomized (114 in arm A and : 230 Patients were randomized (114 in arm A and 116 in Arm B). Overall response rates were 22% and 24% 116 in Arm B). Overall response rates were 22% and 24% for arms A and B. Grade 2 to 4 neuropathy was seen in for arms A and B. Grade 2 to 4 neuropathy was seen in 14% and 27% of Arm A and B patients, respectively.14% and 27% of Arm A and B patients, respectively.

95% Confidence Intervals for 95% Confidence Intervals for ProportionsProportions

What are 95% confidence intervals for the response rates What are 95% confidence intervals for the response rates in the two arms?in the two arms?

standard error of a sample proportion isstandard error of a sample proportion is An equation for confidence interval for a proportion:An equation for confidence interval for a proportion:

Assumptions: Assumptions: n is reasonably largen is reasonably large p is not “too” close to 0 or 1p is not “too” close to 0 or 1 rule of thumb: pn > 5rule of thumb: pn > 5

.( )

pp p

n

1 9 6

1

( )p p

n

1

Example: Response Rate to Example: Response Rate to TreatmentTreatment

Arm A:Arm A:

Arm B:Arm B:

.( )

pp p

n

1 9 6

1

0 2 2 1 9 60 2 2 0 7 8

11 40 1 4 0 3 0. .

. ( . )( . , . )

0 2 4 1 9 60 2 4 0 7 6

11 60 1 6 0 3 2. .

. ( . )( . , . )

Example: Grade 2 to 4 Example: Grade 2 to 4 NeuropathyNeuropathy

Arm A:Arm A:

Arm B:Arm B:

.( )

pp p

n

1 9 6

1

0 1 4 1 9 60 1 4 0 8 6

11 40 0 8 0 2 0. .

. ( . )( . , . )

0 2 7 1 9 60 2 7 0 7 3

11 60 1 9 0 3 5. .

. ( . )( . , . )

95% Confidence Interval for Difference 95% Confidence Interval for Difference in Proportionsin Proportions

What is the 95% confidence interval for What is the 95% confidence interval for the difference in rates of neuropathy in the difference in rates of neuropathy in arms A and B?arms A and B?

( ) . ( ) ( )

p pp p

n

p p

n1 21 1

1

2 2

2

1 9 61 1

( . . ) .. ( . ) . ( . )

( . , . )0 2 7 0 1 4 1 9 60 2 7 0 7 3

11 6

0 1 4 0 8 6

11 40 0 3 0 2 3

APPLICATION APPLICATION

OF OF CONFIDENCE CONFIDENCE INTERVALS INTERVALS

EExample:xample: The following finding of non-The following finding of non-significance in a clinicalsignificance in a clinicaltrial on 178 patients.trial on 178 patients.

TreatmentTreatment SuccessSuccess FailureFailure TotalTotal

AA 76 (75%)76 (75%) 2525 101101

BB 51(66%)51(66%) 2626 7777

TotalTotal 127127 5151 178178

Chi-square value = 1.74 ( p > 0.1)Chi-square value = 1.74 ( p > 0.1) ((non –significantnon –significant)) i.e. there is no difference in efficacy between the i.e. there is no difference in efficacy between the

two treatments.two treatments.

--- The observed difference is: --- The observed difference is: 75% - 66% = 9%75% - 66% = 9% and the 95% confidence interval for the and the 95% confidence interval for the

difference is:difference is: - 4% to 22%- 4% to 22%---- This indicates that compared to treatment B, -- This indicates that compared to treatment B,

treatment A has, at best an appreciable treatment A has, at best an appreciable advantage (22%) and at worst , a slight advantage (22%) and at worst , a slight disadvantage (- 4%).disadvantage (- 4%).

--- --- This inference is more informative than just This inference is more informative than just saying that the difference is non significantsaying that the difference is non significant..

Example:Example:

TreatmentTreatment SuccessSuccess FailureFailure TotalTotal

AA 49 (82%)49 (82%) 1111 6060

BB 33 (60%)33 (60%) 2222 5555

TotalTotal 8282 3333 115115

The chi-square value = 6.38The chi-square value = 6.38 p = 0.01 (highly significant)p = 0.01 (highly significant)

The observed difference in efficacy isThe observed difference in efficacy is 82 % - 60% = 22%82 % - 60% = 22% 95% C .I. = 6 % to 38%95% C .I. = 6 % to 38%

This indicates that changing from treatment This indicates that changing from treatment B to treatment A can result in 6% to 38% B to treatment A can result in 6% to 38% more patients being cured.more patients being cured.

Again, this is more informative than just Again, this is more informative than just saying that the two treatments are saying that the two treatments are significantly different.significantly different.

Example: Disease Control ProgramExample: Disease Control ProgramConsider the following findings pertaining to case-Consider the following findings pertaining to case-holding in the National TB control programholding in the National TB control program

YearYear ProgramProgram CompletedCompleted

TreatmentTreatmentFailed toFailed to

CompleteCompleteTotalTotal

19871987 RoutineRoutine 276276

(46%)(46%)

324324 600600

19881988 SpecialSpecial 312312

(52%)(52%)

288288 600600

The chi square = 4.32, p-value = 0.04 (significantThe chi square = 4.32, p-value = 0.04 (significant))

The impact of special motivation programThe impact of special motivation program

= 52 % - 46% = 6%= 52 % - 46% = 6% in terms of improved in terms of improved

case-holding.case-holding.

The 95% C.I. = 0.4 % to 11.6%,The 95% C.I. = 0.4 % to 11.6%, which indicates which indicates that the benefit from the special motivation that the benefit from the special motivation program is not likely to be more than 11.6%.program is not likely to be more than 11.6%.

This information may helps the investigator to This information may helps the investigator to conclude that the special program was not really conclude that the special program was not really worthwhile, and that other strategies need to be worthwhile, and that other strategies need to be explored, to provide a greater magnitude of explored, to provide a greater magnitude of benefit.benefit.

CHARACTERISTICS OF CI’SCHARACTERISTICS OF CI’S----The (im) precision of the estimate is The (im) precision of the estimate is

indicated by the width of the indicated by the width of the confidence interval.confidence interval.

--The wider the interval the less --The wider the interval the less precisionprecision

THE WIDTH OF C.I. DEPENDS ON:THE WIDTH OF C.I. DEPENDS ON: ---- SAMPLE SIZE---- SAMPLE SIZE ---- VAIRABILITY---- VAIRABILITY ---- DEGREE OF CONFIDENCE---- DEGREE OF CONFIDENCE

EFFECT OF VARIABILITYEFFECT OF VARIABILITY

      Properties of errorProperties of error

  1.1.    Error increases with smaller sample sizeError increases with smaller sample size

For any confidence level, large samples reduce the For any confidence level, large samples reduce the margin of error margin of error

  

2.2.    Error increases with larger standard DeviationError increases with larger standard Deviation

          As variation among the individuals in the population As variation among the individuals in the population increases, so does the error of our estimateincreases, so does the error of our estimate

  

3.3.      Error increases with larger z valuesError increases with larger z values

Tradeoff between confidence level and margin of Tradeoff between confidence level and margin of errorerror

    

Not only 95%….Not only 95%…. 90% confidence interval: 90% confidence interval:

NARROWER than 95%NARROWER than 95%

99% confidence interval: 99% confidence interval: WIDER than 95% WIDER than 95%

x sem 1 6 5.

x sem 2 5 8.

C.I. (degree of C.I. (degree of confidence)confidence)

Z -valueZ -value

90% 90% 1.641.64

95%95% 1.961.96

98% 98% 2.332.33

99%99% 2.582.58

Figure 8-10 and 8-11

Interval width (error) increases withIncreased confidence level

Higher confidence levels haveHigher z values

Error is high in small samples

Other Confidence IntervalsOther Confidence Intervals Differences in meansDifferences in means Response ratesResponse rates Differences in response ratesDifferences in response rates

Hazard ratiosHazard ratios median survivalmedian survival difference in median survivaldifference in median survival OR, RR, Correlation, RegressionOR, RR, Correlation, Regression Non – parametric testsNon – parametric tests

RecapRecap 95% confidence intervals are used to quantify 95% confidence intervals are used to quantify

certainty about parameters of interest.certainty about parameters of interest. Confidence intervals can be constructed for any Confidence intervals can be constructed for any

parameter of interest (we have just looked at parameter of interest (we have just looked at some common ones).some common ones).

The general formulas shown here rely on the The general formulas shown here rely on the central limit theoremcentral limit theorem

You can choose level of confidence (does not You can choose level of confidence (does not have to be 95%).have to be 95%).

Confidence intervals are often preferable to Confidence intervals are often preferable to pvalues because they give a “reasonable range” pvalues because they give a “reasonable range” of values for a parameter. of values for a parameter.

Excellent References on Use of Excellent References on Use of Confidence Intervals in Clinical TrialsConfidence Intervals in Clinical Trials Richard Simon, “Confidence Intervals for Richard Simon, “Confidence Intervals for

Reporting Results of Clinical Trials”, Reporting Results of Clinical Trials”, Annals of Annals of Internal MedicineInternal Medicine, v.105, 1986, 429-435., v.105, 1986, 429-435.

Leonard Braitman, “Confidence Intervals Extract Leonard Braitman, “Confidence Intervals Extract Clinically Useful Information from the Data”, Clinically Useful Information from the Data”, Annals of Internal MedicineAnnals of Internal Medicine, v. 108, 1988, 296-298., v. 108, 1988, 296-298.

Leonard Braitman, “Confidence Intervals Assess Leonard Braitman, “Confidence Intervals Assess Both Clinical and Statistical Significance”, Both Clinical and Statistical Significance”, Annals Annals of Internal Medicineof Internal Medicine, v. 114, 1991, 515-517., v. 114, 1991, 515-517.

THANK YOUTHANK YOU