integrating big data in the belgian cpi...supermarket scanner data current method experimental...

Integrating big data

in the Belgian CPI

Ken Van Loon, Dorien Roels

Geneva, May 2018

http://statbel.fgov.be

Supermarket scanner data

Current method

Experimental results: multilateral methods & splicing options

Web scraping

Footwear

Second-hand cars

Renting student rooms

Hotel reservations

Consumer electronics

Overview

2 http://statbel.fgov.be

Current methodology: “dynamic” method

In production since January 2015

Monthly chained Jevons index

Threshold: 𝑠𝑚+ 𝑠𝑚−1

2 >

1

𝑛 ∗ λ (λ = 1.25)

Imputations

Dumping filters

Outlier filters

SKUs instead of GTIN

Linking relaunches

Scanner data


Goal: switch to a multilateral method in 2020

Ongoing research first results

13 months window length when splicing

Scanner data


Multilateral methods Splicing & extension methods

GEKS-Törnqvist Movement Splice

Time Product Dummy Window Splice

Geary-Khamis Half Splice

Augmented Lehr Mean Splice

Fixed Base Monthly Expanding Window (FBEW)

Fixed Base Moving Window (FBMW)

Dataset for testing multilateral methods

One retailer: period of 37 months

COICOPs:

01.1 Food (excl. seasonal products)

01.2 Non-alcoholic beverages

02.1 Alcoholic beverages

12.1.3.2 Articles for personal hygiene & beauty products

Around 480 product groups in total

Incl. extra relaunch linking

Scanner data


Relaunches

Scanner data


Comparison of multilateral methods (full window)

Scanner data


Results with splicing & extension methods (rolling window = 13 months)

Scanner data


Dumping impact: TPD/GK minus GEKS

Scanner data


Since 2014

R (rvest, Rselenium)

CPI: international train travel, videogames,…

+/- 70 scripts

Daily or several times a week

Web scraping


Clothing Drugstores

Footwear Books

Hotel reservations Videogames

Airfares DVD & Blu-ray discs

International train travel Supermarkets

Second hand cars Student rooms

Consumer electronics …

Websites largest footwear retailers

Feature of this market:

Products leave market at significant lower price

downward drift

Web scraping – Footwear


Non-matched model with stratification

Web scraping – Footwear


Not yet covered

Daily scraping

Time dummy hedonic method (incl. characteristics/depreciation)

Web scraping – Second-hand cars


Not yet covered

Aggregator sites are scraped

Price

Room size

Specific type of room

Address → geocoding

Web scraping – Renting student rooms


Geocoding:



Resulting index:

Current research:

Price collection expanded to more cities

More characteristics



Year City 1 City 2

T 100 100

T+1 102.1 102.3

Domestic destinations: Cities, Seaside, Ardennes

Virtual reservations:

Web scraping – Hotel reservations


Manual Web scraping

Frequency 1x / month Daily

Reservation 4 weeks before arrival 4 & 8 weeks before arrival

Characteristics Fri – Sun, Double room Fri - Sun, incl. breakfast & free cancellation

Method Sample of hotels Stratification:

Destination

Area

Weeks booked before arrival date

Hotel classification

Price Per hotel Per stratum

Web scraping – Hotel reservations


Product characteristics

Time dummy hedonic method

Splicing methods compared with:

Monthly matching Jevons index (MM)

Monthly chaining & replenishment (MCR)

Simulating current official method

Web scraping – Consumer electronics


Web scraping – Consumer electronics


Scanner data

Dynamic method ~ Multilateral methods

No large differences between splicing/extension options

Lehr index deviates

Web scraping

Footwear/Hotels: no large differences classical vs. web scraping

Second-hand cars/Renting student rooms: index calculation possible

Consumer electronics: Use of hedonics

Conclusion



Thank you!

integrating big data in the belgian cpi...supermarket scanner data current method experimental...

Documents