2_sources of errors in measurement

8/22/2019 2_Sources of Errors in Measurement

1/29

1

Types and Sources of Errors in Statistical

Data


2/29

2

Types of Errors

In general, there are two types of errors:

a. Non-sampling errors and

b. Sampling errors.

It is important for a researcher to be aware of these

errors, in particular non-sampling errors, so that they

can be either minimised or eliminated from the data

collected.


3/29

3

Non-sampling errors

These are errors that arise during the course of all

data collection activities.

In summary, they have the following

characteristics: exist in both sample surveys and censuses

data.

difficult to measure.


4/29

4

Sources of non-sampling errors

Non-sampling errors arise from:

defects in the sampling frame.

failure to identify the target population.

non response.

responses given by respondents.

data processing and

reporting, among others.


5/29

5

Defects in the sampling frame

This result in coverage errors.

These occur when there is an omission, duplicationor wrongful inclusion of units in the sampling frame.

Omissions are referred to as under coverage whileduplications and wrongful inclusions are called overcoverage.

These errors are caused by defects such asinaccuracy, incompleteness, duplication, inadequacy

and out of date sampling frames.

Coverage errors may also occur in field operations,that is, when an enumerator misses severalhouseholds or persons during the interviewing

process.


6/29

6

Failure to Identify Target Population

This occurs when the target population is not clearly

defined through the use of imprecise definitions or

concepts or when the survey population does not

reflect the target population due to an inadequatesampling frame and poor coverage rules.


7/29

7

Response

They result from the data that have been requested,

provided, received or recorded incorrectly.

They may occur as a result of inefficiencies with

the questionnaire, the interviewer,

the respondent or

the survey process.


8/29

8

a. Poor questionnaire design

The content and wording of the questionnaire may

be misleading and the layout of the questionnaire

may make it difficult to accurately record responses.

As a rule, questions in questionnaire should not beloaded, double-barrelled, misleading or ambiguous,

and should be directly relevant to the objectives of

the survey.

It is essential to pilot test questionnaires to identifyquestionnaire flow and question wording problems,

and allow sufficient time for improvements to be

made to the questionnaire.


9/29

9

Poor questionnaire design contd

The questionnaire should then be re-tested to

ensure changes made do not introduce other

problems.


10/29

10

b. Interviewer bias

An interviewer may influence the way a respondent

answers survey questions.

To prevent this, interviewers must be trained to

remain neutral throughout the interviewing processand must pay close attention to the way they ask

each question.


11/29

11

c. Respondent errors

These arise through the respondent providing

inaccurate or wrong information.

They occur because of memory biases or

respondents giving inaccurate or false informationwhen they believe that they are protecting their

personal interests or integrity.

They can also arise from the way the respondent

interprets the questionnaire and the wording of theanswer that the respondent gives.

Careful questionnaire design and effective

questionnaire testing can overcome these problems

to some extent.


12/29

12

d. Problems with the survey process

Errors can also occur because of problems with the

actual survey process such as using proxy

responses, that is, taking answers from someone

other than the respondent or lacking control over the

survey procedure.


13/29

13

Non-Response Non-response results when data is not collected from

respondents.

The proportion of these non-respondents in the sample is calledthe non-response rate.

Non-response can be eithertotal orpartial.

Total non-response or unit non-response can arise if arespondent cannot be contacted (because the sampling frame isincomplete or out-of-dated) or the respondent is not at home or is

unable to respond because of language difficulties or illness orout rightly refuses to answer any questions or the dwelling unit isvacant.

Other respondents may indicate that they simply don't have the

time to complete the interview or survey form.


14/29

14

Non-response - contd

When conducting surveys it is important to document information

on why a respondent has not responded.

Partial non-response or item non-response can occur when a

respondent replies to some but not all questions of the survey.

This can arise due to memory problems, inadequate information

or an inability to answer a particular question/section of the

questionnaire.

A respondent may refuse to answer if;

a. they find questions particularly sensitive, or if

b. they have been asked too many questions.


15/29

15

Non-response - contd

To reduce non-response, the following approaches can beused:

care should be taken in questionnaire design through the use

of simple questions.

pilot testing of the questionnaire.

explaining survey purposes and uses.

assuring confidentiality of responses.

public awareness activities including discussions with key

organisations and interest groups, news releases, mediainterview and articles.


16/29

16

Processing

These occur at various stages of data processing such as

data cleaning, data capture and editing.

Data cleaning involves taking preliminary checks before

entering the data onto the processing system.

Coder bias is usually a result of poor training or incomplete

instructions, variability in coder performance and data entry

errors.


17/29

17

Processing contd

Inadequate checking and quality management at this stage

can introduce data loss (where data is not entered into the

system) and data duplication (where the same data is entered

into the system more than once) thus introducing errors in

data.

To minimise these errors, processing staff should be given

adequate training, instructions and realistic workloads.


18/29

18

Time Period Bias

This occurs when a survey is conducted during an

unrepresentative time period.

Survey timing is thus important and failure torecognise this introduces errors in data.


19/29

19

Analysis and Estimation

Analysis errors include any errors that occur whenusing wrong analytical tools or when preliminaryresults are used instead of the final ones.

Errors that occur during the publication of the dataresults are also considered as analysis errors.

Estimation errors occur when inappropriate orinaccurate weights are used in the estimationprocedure thus introducing errors to the data.

They also occur when wrong estimators areselected by the analyst.


20/29

20

Reducing non-sampling errors

Can be minimised by adopting any of the following

approaches:

using an up-to-date and accurate sampling frame.

careful selection of the time the survey isconducted.

planning for follow up of non-respondents.

careful questionnaire design.

providing thorough training and periodic retraining

of interviewers and processing staff.


21/29

21

Reducing non-sampling errors contd

- designing good systems to capture errors that occurduring the process of collecting data, sometimes

called Data Quality Assurance Systems.


22/29

22

Sampling error

Refers to the difference between the estimate derived from asample survey and the 'true' value that would result if a

census of the whole population were taken under the same

conditions.

These are errors that arise because data has been collected

from a part, rather than the whole of the population.

Because of the above, sampling errors are restricted to

sample surveys only unlike non-sampling errors that canoccur in both sample surveys and censuses data.


23/29

23

Sampling errors contd

There are no sampling errors in a census because

the calculations are based on the entire population.

They are measurable from the sample data in thecase of probability sampling.


24/29

24

Factors Affecting Sampling Error

It is affected by a number of factors including:

a. sample size

In general, larger sample sizes decrease the sampling error,however this decrease is not directly proportional.

As a rough rule of the thumb, you need to increase the samplesize fourfold to halve the sampling error but bear in mind thatnon sampling errors are likely to increase with large samples.

b. the sampling fraction.

this is of lesser influence but as the sample size increases as afraction of the population, the sampling error should decrease.


25/29

25

Factors Affecting Sampling Error contd

c. the variability within the population.

More variable populations give rise to larger errors as the

samples or the estimates calculated from different samples

are more likely to have greater variation.

The effect of variability within the population can be reduced

by the use of stratification that allows explaining some of thevariability in the population.

d. sample design.

An efficient sampling design will help in reducing samplingerror.


26/29

26

Characteristics of the sampling error

generally decreases in magnitude as the sample size

increases (but not proportionately).

depends on the variability of the characteristic of interest in the

population.

can be accounted for and reduced by an appropriate sample

plan.

can be measured and controlled in probability sample surveys.


27/29

27

Reducing sampling error

If sampling principles are applied carefully within the

constraints of available resources, sampling error

can be kept to a minimum.


28/29

28

Potential Sources of Error in Research Designs

Surrogate Information Error

Measurement Error

Population Definition Error

Sampling Frame Error

Data Analysis Error

Respondent Selection Error

Questioning Error

Recording Error

Cheating Error

Inability Error

Unwillingness Error

Total Error

Non-sampling

Error

Random

Sampling Error

Non-responseError

ResponseError

Interviewer

Error

Respondent

Error

Researcher

Error


29/29

29

End of Topic

2_sources of errors in measurement

Documents