Kumar Abhay , IA&ASDeputy Director

O/o the Director General of Audit

Central Expenditure, New Delhi

Statistical Sampling: An Audit Assurance Mechanism

Emerging Areas of AuditLecture at NAAA, Shimla

September, 2015

Sampling : IntroductionMethod of selecting ‘some’ units from the

‘population’, studying the selected units in detail and make inferences about the whole

Sampling provides a means of gaining information about the population without the need to examine the population in its entirety

Sampling can provide a valid, defensible methodology in Audit but it is important to match the type of sample needed to the type of analysis required

Population: Group of individuals/items under study; aggregate of objects, may be finite or infinite

Sample : A finite subset of statistical individuals in a population ; no. of units in a sample is called the sample size

04/21/232

Sampling cont’d…

Why Sampling ?Less timeReduced cost of surveyGreater accuracy of resultsIf population is too largeIf testing is destructive

If time and money are not important factors or if the population(under consideration) is not too large , a complete census may be better than any sampling method.

04/21/233

Sample vs. Census

04/21/234

Conditions Favoring the Use of

Type of Study

Sample Census

1. Budget

Small

Large

2. Time available

Short Long

3. Population size

Large Small

4. Variance in the characteristic

Small Large

5. Cost of sampling errors

Low High

6. Cost of nonsampling errors

High Low

7. Nature of measurement

Destructive Nondestructive

8. Attention to individual cases Yes No

Principal Steps in Sample Survey

Objective of the survey : clear & unambiguousDefining the population to be sampledSampled population should be same as target

populationSampling frame and sampling units (the sampling

units must cover the entire population and they must be distinct , unambiguous and non-overlapping)

Data collection ( consistent with the objective of the survey)

The Questionnaire ( to be filled in by the respondent) or Schedule ( of enquiry to be completed by the interviewer)

04/21/235

Principal Steps in Sample Survey cont’d…

The method of collecting information (interview /telephone /mail/email/other available records)

Non-response ( how to deal with ?)

The Pretest ( trying out the questionnaire and field method on a small scale )

04/21/236

Principles of Sample Survey

Principle of Statistical Regularity : A moderately large no. of items chosen at random from a large group are almost sure on the average to possess the characteristics of the large group ( larger the sample size more reliable the estimate, other things constant)

Principle of Validity: Conditions to get valid estimate , conclusions, etc. ( requirement is Random Sampling)

Principle of Optimisation : Given level of efficiency at minimum costMaximum possible efficiency with given cost

04/21/237

Errors in Sample SurveySampling Error: As only a part of the population is

used to estimate population parameters ; no sampling error in complete enumeration( larger the sample size smaller the sampling error ).

Sampling Risk : The conclusion drawn on the basis of a particular sample may not always be true in respect of the population. There are four possible situation that can arise based on decision from sample ( two correct decisions and two wrong : Rejecting right thing (Type I Error or Alpha Risk) & Accepting wrong thing (Type II Error or Beta Risk )

Alpha (α) Risk : Client’s/Auditee’s Risk and is related to Audit Efficiency (may lead to further sampling, etc.)

Beta (β) Risk : It relates to Audit Effectiveness (Detection Risk)

04/21/238

Non-sampling ErrorsIt is possible at every stage of planning or

execution of census or sample survey.Faulty planning, definitions , inadequate

data specifications, etc.Response Errors

Accidental ( memory loss , etc. )Prestige biasSelf interestBias due to interviewer

Non-response biasErrors in coverage, compilation ,

publication , etc.

04/21/239

What is Survey ?

Action of ascertaining facts regarding conditions or the condition of something to provide exact information and as a systematic collection and analysis of data on some aspect of an area or group.

A survey is a process – more than mere compiling of data.

To yield relevant information, the data must be analysed, interpreted and evaluated.

04/21/23Kumar Abhay , IAAS10

Types of Survey• Census • complete enumeration• considering the whole population• expensive and time consuming

• Sample Survey• part of the population is studied• sample is a representative part of the

whole population• inferring from the part about the whole

population

04/21/23Kumar Abhay , IAAS11

Steps involved in Survey

• Defining the Purpose of the Survey• Clear and unambiguous• Knowledge of the exact nature of the problem

(objective) would determine exactly what kind of data to collect and what to do with it

• Level of living, status of employment• Performance of Social Sector Schemes – Health,

Education, etc.• Defining the Population • Depends upon the objective of the survey• Target segment should be clearly defined• Geographical (districts, hills, plains, agro-climatic

zones, etc.)• Demographic ( urban/rural, age, sex, etc.)• Socio-economic ( APL/BPL, monthly income/expenditure, etc.)

04/21/23Kumar Abhay , IAAS12

Steps involved in Survey

• Developing the Survey Plan• Survey Methodology :- should it be cross-

sectional or repetitive.• Data collection plan• Analysis Plan :- Which indicator to generate:-

per capita income/expenditure, proportion of beneficiary, proportion of poor.

• Determining the Sampling Frame • Complete and up-to-date list of sampling unit• Area Frame / List Frame• All the beneficiary/ BPL household constitute

a frame

04/21/23Kumar Abhay , IAAS13

Steps involved in Survey

• Questionnaire Design • Questionnaires play a central role in the data collection process.

• Questionnaire is the means for collecting your survey

data.

•A well-designed questionnaire efficiently collects the

required data with a minimum number of errors.

• It facilitates the coding and capture of data.

• A poorly designed questionnaire gives rise to non-sampling

error.

04/21/23Kumar Abhay , IAAS14

Steps involved in Survey:Questionnaire Design

• Key points to note while formulating the questionnaire

• Is the introduction informative ? Does it stimulate respondent’s

interest?

• Are the words simple, direct and familiar to all respondents?

• Do the questions read well? How is flow in the questionnaire?

• Are the questions clear and as specific as possible?

• Does the questionnaire begin with easy and interesting

questions?

• Does the question specify a time reference?

• Should the questions be open/close-ended? If the questions are

close-ended, are the response categories mutually exclusive and

exhaustive?

04/21/23Kumar Abhay , IAAS15

Steps involved in Survey

• Undertaking Fieldwork and Gathering Data

•Resource Planning in terms of manpower and time constraint

•Training of investigators

•Quality monitoring at Field level

•Progress report.

04/21/23Kumar Abhay , IAAS16

Steps involved in Survey

• Data Processing

• Data Entry and Verification

• Data Validation.

• Weight /Multiplier Calculation.

• Table Generation

• Report Writing

04/21/23Kumar Abhay , IAAS17

Types of Statistical Sampling Plan

Attribute Sampling Plans & Variable Sampling PlansAn attribute is a qualitative characteristic which can

not be measured quantitatively but the population may be classified into various classes w.r.t. the attribute under study (simplest being dichotomous classification : Pass-Fail type).

Attribute Sampling is generally used in tests of controls ( to answer questions like how many deviations , proportion of deviations ,no. of invoices paid twice , in cases of rare events) Fixed sample size attribute sampling (objective is to

perform a test of controls to estimate the deviation rate of a population)

Sequential (stop or go )attribute sampling ( not so common cases); it prevents oversampling

Discovery Sampling ( observing at least one deviation , very rare cases); Exploratory Sampling

04/21/2318

Variable Sampling PlansVariable ( or quantitative ) sampling is used when

the objective is to estimate a quantity ( may be total amount of loss to the government , average loss per transaction , etc.); deal with “ How Much”

It is used primarily for substantive testing.Probability Proportional to Size (PPS) Sampling

Plan : It is hybrid plan combining the characteristics of attribute and variable sampling.

Margin of Error/Precision Limit/Precision : Maximum permissible deviation rate ( or maximum degree of acceptable misstatement in either direction )

Reliability (Confidence Coefficient ): Probability that precision interval contains the true value

04/21/2319

Types of SamplingSubjective or Purposive or Judgment Sampling :

Personal bias/ preference/ prejudices may come into play ( validity problem)

Snowball Sampling: An initial group of respondents is selected, usually at randomAfter being interviewed, these respondents are asked to

identify others who belong to the target population of interest

Subsequent respondents are selected based on the referrals

Block Sampling : A block sample includes all items in a selected time period

Probability SamplingMixed Sampling : Partly judgment and partly

probability sampling The aim of the (sampling) design is to achieve a

balance between the required precision and the available resources

04/21/2320

Probability SamplingIt is a scientific method of selecting samples

according to some laws of chance in which each unit in the population has some definite pre-assigned probability of being selected in the sample.

Each unit has an equal chance of being selected Units have different prob. of being selected Prob. of selection of a unit is proportional to the

size/importance of the unit in the population

Probability Sampling permits justifiable inference from the sample to the population, at quantified levels of precision

We need representative sample ( i.e. sample should reflect the population characteristics)

04/21/2321

Types of Probability Sampling

Simple Random Sampling (SRS)SRS with replacementSRS without replacement

Stratified Random SamplingSystematic Random SamplingCluster SamplingMultistage SamplingMultiphase SamplingPPS/ Monetary Unit Sampling , etc.

04/21/2322

Simple Random Sampling

It is the technique of drawing a sample in such a way that each unit in the population has an equal and independent chance of being included in the sample.

Selection of a SRSBy lottery systemBy random numbers method

Merits of SRS:Element of subjectivity completely eliminatedCan be implemented easily

04/21/2323

Simple Random Sampling cont’d…

Limitations of SRS:Requires an up-to-date frameAdministrative inconvenienceAt times give most non random looking resultsFor a given precision SRS generally requires

larger sample size as compared to Stratified Random Sampling

Size of the simple random sample depends on margin of error permissible and confidence coefficient

04/21/2324

Stratified Random Sampling

Stratification means division into layers.Population is divided into various groups such that :

Units within each group are as homogeneous as possible

Between groups units are as widely different as possible

Population is divided into certain no. of relatively homogenous , mutually disjoint sub-groups termed as Strata and from each stratum simple random sample is drawn

Stratifying Factor : Criterion used for classification like Income, Age, Height, Gender , Rural- Urban, Revenue- Capital, Charged-Voted, Plan-Non-Plan, Treasuries, Major Heads, etc.

04/21/2325

Stratified Random Sampling cont’d…

Merits :More representative ( specially when

population is highly heterogeneous)Greater accuracyAdministrative convenience

Allocation of Sample Size:Proportional allocationOptimum allocation

04/21/2326

Stratified Random Sampling cont’d…

Larger sample size required from a stratum if

Stratum size is large

Stratum variability is large

Sampling cost per unit is low in the stratum

04/21/2327

Systematic SamplingUsed when complete and up-to-date list of

sampling units is available. Only the first unit is selected at random and rest of the units get selected automatically according to some predetermined pattern involving regular spacing of units.

Merits :Operationally more convenientEfficient if units in the population are arranged

randomlyProblem of variable sample size in case N ≠ nk [N:

population size; n: sample size & k: sampling interval]

04/21/2328

Cluster Sampling

In this case the total population is divided , depending on problem under study , into some recognisable sub-divisions ( households, village, district , etc.) which are termed as Clusters and a SRS of these clusters is drawn. All the units of the selected clusters are studied.

Block Sample includes all the items in a selected time period. Bills/ vouchers are grouped in months…. selecting say March & September months’ Vouchers for audit .

Merits:Very convenient ( though not very precise)Cost effective

04/21/2329

Multistage Sampling

Instead of enumerating all the units in selected clusters, sub-sampling ( selecting only some not all ) within clusters is done. 2-stage/3-stage/4-stage sampling ( larger the no. of stages more complicated calculations and analysis)

In 2-stage sampling primary units are the clusters and the secondary units are the units within clusters

Merits:We need frame only for those units which are

selected in the first stageCost effective

04/21/2330

Sampling with Varying Probabilities

If units vary considerably in size , SRS(WOR) may not be appropriate since it does not take into account the possible importance of larger units in the population. So attaching unequal prob. of selection to different units in the population becomes logical way-out.

Probability Proportional to Size (PPS): Sampling method in which the units are selected with prob. proportional to some measures of their size is known as PPS Sampling. When monetary value of a unit/item is taken as a measure of size this type of sampling is known as Monetary Unit Sampling.

04/21/2331

04/21/2332

Various Sampling Methods : Comparison

04/21/2333

Various Sampling Methods : Comparison

cont’d….

04/21/2334

Various Sampling Methods : Comparison

cont’d….

Audit Sampling

Application of an audit procedure to less than 100 % of the items or class of transactions for the purpose of evaluating some characteristic of the items of class of transactions under audit.

Use of sampling may not be possible/advisable in all auditing procedures ( scanning accounting records for unusual items (outliers) , inquiry (Satyam case), most analytical procedures, etc.).

04/21/2335

Size of the Sample (Extent of testing)

Important qualitative factors in determining the size of the sample: No. of items in the population (N) Degree of assurance desired from sample evidence

(Confidence coefficient) Margin of error or precision ( a measure of possible

difference between the sample estimate and the actual population value)

Expected variability among the units in the population ( Population Mean Square, S2 )

Resource constraints ( cost of sampling per unit ; time , money, human resource)

Importance of the decision ( extent of penalty for making mistake )

Nature of the analysis intended ( exploratory and conclusive)

Sample sizes used in similar studies Incidence rates

04/21/2336

Reporting the Results

When reporting the results of a Sample it is important to cover the following key factors :

The Sample Size

The Sample selection methodology

The Estimate(s) resulting from the Sample, and

The precision and confidence intervals for the Estimate(s)

04/21/2337

04/21/2338

Thank You all for Your Kind Participation &

Attention

Feedback & Comments

Kumar Abhay, IA&ASDeputy Director (AMG-II)O/o the Director General of

Audit(CE)New Delhi-110002E-mails:

abhaystat@yahoo.comabhayK@cag.gov.in

Phone Nos.:Tel. (O.) : +91-11-23702351Mobile : +91-8130353444

04/21/23 39