calculation of sampling errors mics3 data analysis and report writing workshop

Calculation of Sampling Errors

MICS3 Data Analysis and Report Writing Workshop

Background

The sample selected in a survey is one of the many samples that could have been selected (with same design and size).

Sampling errors are measures of the variability between all possible samples, which can be estimated from survey results.

Background

Calculation of sampling errors is very important;

-Provides information on the reliability of your results

-Tells you the ranges within which your estimates most possibly fall

-Provide clues as to the sample sizes (and designs) to be selected in forthcoming surveys

Background

MICS3 sample designs are complex designs, usually based on stratified, multi-stage, cluster samples.

It is not possible to use straightforward formulae for the calculation of sampling errors. Sophisticated approaches have to be used

New versions of SPSS (13 or 14) is used for this purpose.

SPSS uses Taylor linearization method of variance estimation for survey estimates that are means or proportions.

This approach is used by most other package programs: Weswar, Sudaan, Systat, EpiInfo, SAS

Background

In MICS3, the objective is to calculate sampling errors for a selection of variables, for the national sample, as well as selected sub-populations, such as urban and rural areas, and regions

Sampling errors will be presented as part of the final report, in an appendix

BackgroundSampling ErrorsStandard errors, coefficients of variation, design effects (DEFF), square root of design effects (DEFT) and confidence intervals for selected variables, Country, Year

Table Value (p)Standard error (SE)

Coefficient of variation

Design effect (DEFF)

Square root of design effect

(DEFT)Unweighted

count p - 2SE p + 2SE

Household availability of insecticide treated nets CH.10 0.000 0.000Iodized salt consumption NU.5 0.000 0.000

Improved source of drinking water EN.1 0.000 0.000Sanitary means of excreta disposal EN.5 0.000 0.000Net primary school attendance rate ED.3 0.000 0.000Net secondary school attendance rate ED.4 0.000 0.000

Skilled attendant at delivery RH.5 0.000 0.000Antenatal care RH.3 0.000 0.000

Underweight prevalence NU.1 0.000 0.000Had acute respitory infection CH.6 0.000 0.000Antibiotic treatment of suspected pneumonia CH.7 0.000 0.000Diarrhoea in last two weeks CH.4 0.000 0.000Received ORT or increased fluids and continued feeding CH.5 0.000 0.000Children sleeping under ITNs CH.11 0.000 0.000Fever in last two weeks CH.12 0.000 0.000Antimalarial treatment CH.12 0.000 0.000Support for learning CD.1 0.000 0.000Birth registration CP.1 0.000 0.000

UNDER-5s

Confidence limits

HOUSEHOLDS

HOUSEHOLD MEMBERS

WOMEN






(DEFT)Unweighted






UNDER-5s

Confidence limits

HOUSEHOLDS

HOUSEHOLD MEMBERS

WOMEN

Value of the estimate should be the same as that in the corresponding table






(DEFT)Unweighted






UNDER-5s

Confidence limits

HOUSEHOLDS

HOUSEHOLD MEMBERS

WOMEN

Standard error is the square root of the variance – a measure of the variability between all possible samples






(DEFT)Unweighted






UNDER-5s

Confidence limits

HOUSEHOLDS

HOUSEHOLD MEMBERS

WOMEN

Coefficient of variation (relative error) is the ratio of SE to the estimate






(DEFT)Unweighted






UNDER-5s

Confidence limits

HOUSEHOLDS

HOUSEHOLD MEMBERS

WOMEN

Design effect is the ratio between the SE using the current design and the SE that would result if a simple random sample was used. A DEFT value of 1.0 indicates that the sample is as efficient as a SRS






(DEFT)Unweighted






UNDER-5s

Confidence limits

HOUSEHOLDS

HOUSEHOLD MEMBERS

WOMEN

Upper and lower confidence limits are calculated as p +/- 2.SE

Indicate the ranges within which the estimate would fall in 95 percent of all possible samples of identical design and size

How SPSS works

COMPLEX SAMPLES module

Can be used to select a sample, or indicate the design of the sample from which the data set comes, so that sampling error estimates can be calculated

Calculations can be done for means and proportions, ratios, frequencies and crosstabs. Also possible to use general linear models and logistic regression.

How SPSS works

Prepare an analysis file to indicate the parameters that define the sample design.CSPLAN ANALYSIS /PLAN FILE='micsplan.csplan' /PLANVARS ANALYSISWEIGHT=hhweight /PRINT PLAN /DESIGN STRATA= strat CLUSTER= HH1 /ESTIMATOR TYPE=WR.

Using the plan file, calculate sampling errors.Complex Samples Descriptives.CSDESCRIPTIVES /PLAN FILE = 'micsplan.csplan' /SUMMARY VARIABLES =treated iodized /MEAN /STATISTICS SE CV COUNT DEFF DEFFSQRT /MISSING SCOPE = ANALYSIS CLASSMISSING = EXCLUDE.

Problems with using SPSS

Need to pair clusters and create pseudo-strata.

Cannot handle normalized weights – multiply the weights by 1,000,000 before analysis.

Provides estimates for subpopulations only when the data file used contains only cases for the subpopulation in question

Provides incorrect confidence limits

Cannot report on sampling errors for variables coming from different data sets

SPSS OutputUnivariate Statistics

.0285 .00608 .213 6.418 2.533 4811

.4722 .01050 .022 2.119 1.456 4786

Household availability ofinsecticide treated nets

Iodized salt consumption

MeanEstimate

StandardError

Coefficientof Variation Design Effect

Square RootDesign Effect

UnweightedCount

Univariate Statistics

.6920 .02211 .032 67.767 8.232 29560

.9306 .00510 .005 11.919 3.452 29560

.6212 .00658 .011 .828 .910 4504

.8157 .00788 .010 2.240 1.497 5422

Improved source ofdrinking water

Sanitary means ofexcreta disposal *

Net primary schoolattendance rate

Net secondary schoolattendance rate

MeanEstimate

StandardError



UnweightedCount

Univariate Statistics

.8309 .01582 .019 2.151 1.467 1209

.7836 .01691 .022 2.036 1.427 1209

Skilled attendantat delivery

Antenatal care

MeanEstimate

StandardError



UnweightedCount

SPSS OutputUnivariate Statistics

.1699 .00786 .046 1.312 1.146 2995

.0132 .00199 .150 .956 .978 3167

.3507 .05078 .145 .487 .698 44

.1340 .00760 .057 1.576 1.255 3167

.3311 .02156 .065 .904 .951 432

.0097 .00404 .416 5.375 2.318 3167

.0743 .00630 .085 1.824 1.350 3167

.0168 .00748 .446 .810 .900 240

.6178 .01103 .018 1.630 1.277 3167

.8688 .00872 .010 2.114 1.454 3167

Underweight prevalence

Had acute respitoryinfection

Antibiotic treatment ofsuspected pneumonia

Diarrhoea in last twoweeks

Received ORT orincreased fluids ANDcontinued feeding

Children sleeping underITNs

Fever in the last twoweeks

Antimalarial treatment

Support for learning

Birth registration

MeanEstimate

StandardError



UnweightedCount

calculation of sampling errors mics3 data analysis and report writing workshop

Documents