Editing and Validation

Prices and PPPs*

ICP-Western Asia Regional MeetingBeirut, LebanonMay 26, 2005

Yonas BiruYBIRU@WORLDBANK.ORG

*Presentation adopted from Chapter 7 of the ICP Handbook

Editing & Validation

The primary objective is to eliminate or reduce the incidence of non-sampling errors & to correct them when detectedThe ICP uses a hierarchy of checking and validation proceduresFirst, outliers of individual prices are identified and screened at item level within a countrySecond, outliers of average prices can be identified and screened for different products within the same basic heading within the same countryThird, outliers of average prices are identified and screened for the same products in different countries.

Tool Pack For Data Validation

The Tool Pack Supports both intra-country (Tables 1 and 2) and inter-country data editing and validation (Quaranta Tables)

Intra-country Validation Process (National Coordinators)

PCM: Screening of individual price observations

PAM: Screening of average prices both at product and at the BH level

Inter-country Validation Process (Regional Coordinators)

PAM: Screening of average prices for the same products in different countries are checked against each other

Prevention is Better than Correction

The amount of data editing and validation work at the regional level depends on the quality of the data collectedGood survey frame and survey management practices are critical to minimize non-sampling errorsWell defined product specifications accompanied with product images and clear instruction and questionnaires help minimize product related errorsData collectors and data entry clerks should be well trained and regularly supervisedClosely monitored and supervised field work helps eliminate or reduce errors

Validation Is An Iterative Process

National Coordinators (NC) send edited price data to the Regional Coordinator (RC)RC computes Basic Heading PPPs, examines Quaranta Tables (QT) & sends feedback to NCs for correction or validationNCs recheck data, correct errors or validate the first submission or withdraw the data in question.RC computes Basic Heading-PPPs using revised dataRC may get back to NCs for further checking & validationRC generates Basic heading PPPs using fully edited pricesEditing is completed when NCs endorse their data as finalAcceptance of the price data implies endorsement of BH-PPP results

Time Is Of The Essence

Running the full sequence of editing and validation for a given quarter can take as long as five to six monthsThe process must be carefully planned and tightly scheduledNCs need to realize that a delay on their end will drag the regional programRCs need to provide feedback to NCs without delayRC, however, cannot start the process before data from at least 50% of the countries are submittedTimely validation of data is crucial to put in place the necessary corrective measures for the subsequent quarter

County Level Editing & Validation

Editing should start immediately after data collection and should be undertaken on a regular basisThe objective Is to identify extreme observations or outliers (Outlier are observations diverge so much from the average)The policy is not to reject outliers outright but to investigate further if they are indeed outliersIt is recommended that a separate editing and validation process should be carried out in each region within a countryCountry level editing are then carried out under the general guidance and supervision of the national coordinatorSteps should be taken to avoid errors detected in the early rounds of price collection from being repeated in later roundsAfter the preliminary editing at the country level the data is transferred to regional coordinator for further checking

Table 1 and 2 of the Tool Pack

Data editing is carried out using standard format and tables Table 1 provides Average Prices, Max and Min prices as well as Standard Deviation and Var. Co.If the Var. Co. for a given product is > 30% we go to the corresponding Table 2 for further investigation Table 2 provides individual price observations, ratio to average price, Min and Max prices, as well as Min Max ratio and “t-value”‘t-values’ measure divergence between an individual price and the average price divided by the standard deviationAt 95% degree of confidence the ‘t-ratio’ falls between 2 & -2 Ratio between individual price and average price should fall between 1.25 and .75

1

2

3

4

After Country Level Editing Data Is Transferred To RC

Regional Editing & Validation

The primary objective is to eliminate or reduce the incidence of non-sampling errors by comparing prices across countriesEditing is done in cooperation with National Coordinators Types of potential error sourcesPrice Error – Wrong price is recorded for the right product, the right UOM and/or the right outlet (this may happen when data is collected or entered into the Tool Pack)Product Error – Price represents the wrong specification, including error in UOM, product specification and outlet type (this is harder to detect at the country level)

The Quaranta TablesQTs represent a set of diagnostic tables for a basic heading and one for each product level details within the BH

QTs consist of data submitted by NCs and other indicators computed by RCs

QTs are useful to screen the estimated national average prices for possible errors by comparing the dispersion of the average prices for the same product in different countriesData editing is done after national average prices have been converted into a common unit of currency by using Exchange Rates (XR) or PPPs.

Data Required for Quaranta Tables

Average Prices Expressed in National currenciesExchange Rate (XR) for the survey periodBasic Heading PPPs (the BH-PPPs are unweighted PPPs calculated by the CPRD method.)Expenditure weights at the BH levelThe number of products and the number of representative products priced by the country under investigationThe BH weights and the number of products are useful indicators of the importance and coverage of the BHThe BH weights and the number of products prices inform decision making in the validation process

Other Indicators

BH-PLI: (Price Level Indexes at the BH level) This is also known as CUP priceBH-PLI is defined as (BH-PPP) / (XR)The BH-PLI facilitate the comparison of price levels both across basic headings and within basic headings.XR-PRICE: These are prices expressed in a common currency PPP-PRICE Are prices expressed in PPP termsCV (Coefficient of Variation)

Coefficient Of Variations

There are 3 different CVs in the Quaranta TablesVar. Co for the individual price variations on which each national average price is basedThe second Var. Co is the average Var. Co that measures the relative dispersion of the PPP prices or the PPP price ratios for the same product in different countriesThe average of CVs for the individual product prices can be used as an indicator of the overall reliability of the basic heading paritiesThe third Var. Co is a country Var. Co. It measures the dispersion in the PPP Price Ratio for the different products in single country.

Quaranta Tables

Quaranta Tables

XR-Price: National Average Prices expressed in a common currency (D) PPP-Price: National Average Prices expressed in PPP (D) PLI: (PPP-Price/EX-Price) shows price relatives – County “A” is most expensive and “E” least expensive Items: No. of items and no. of representative items in the BH Var. Co: Variation Coefficient

Quaranta Tables

1. PLI% for country “A” warrants a closer look, but the Var. Co seems OK2. Var. Co for country “C” warrants a closer look, but the PLI% seems OK. 3. Both PLI% & Var. Co of Country “E” warrant closer investigation

Item Level Details

XR- Ratio: Ratio of XR-Price to GeoMean of XR-PricePPP-Ratio: Ratio of PPP-Price to GeoMean of PPP-Price

Var. Co for country “A” shows Var. Co for individual price observationsThe second type of variation (75.63) measures the relative dispersion of the PPP prices (or PPP Price Ratios) for the same product in different countries

Flagging and Editing Outliers

XR-PRICE and the BHPPP-PRICE are used to obtain product price level indices (XR-PLI and BHPPP-PLI)Product price level indices that fall outside specified ranges are flagged. Usually 50 and 150 are used as lower and upper bound thresholds.In the first instance, both the indices are considered togetherWhat do we do if XR-PLI flags the product as an outlier, but the BHPPP-PLI did not?Usually the BHPPP-PLI is accepted. In the Quarants Tables the Var. Co is computed for PPP-PLI.

Data Validation

When the BHPPP-PLI of a country is flagged, it is necessary to look closely at the average price underlying the index.All prices considered suspect are sent back to countries for verification. Usually countries either change the price or withdraw it.Countries check if the price is the wrong price, in which case they will either correct it or simply withdraw it. If the price is correct then they would confirm its validity and send it back to the regional coordinator.

Potential Sources of Errors

The price may be correct but it may be on clearanceThe quality of the product priced is of higher or lower qualityThe country has priced the right product in the wrong outletThe product specification is misrepresentedProduct may be imported when the specification is for local brandThe product is unrepresentativeAs the result of this process sometimes both price and representativeness could be changed Countries may also supply more info about the product pricedThis can result in revising the product specification