measuring retail trade with administrative data...2016/06/10 · data” web scraping/ application...
TRANSCRIPT
1 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
U.S. Bureau of Labor Statistics
Measuring Retail Trade with Administrative Data:
David FriedmanAssociate Commissioner for Prices & Living
ConditionsFederal Economic Statistics Advisory Committee
June 10, 2016
2 — U.S. BUREAU OF LABOR STATISTICS • bls.gov2 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Data Sources
Administrative/Publicly available data
Purchased data sets
Company provided data – “corporate level data”
Web scraping/ application program interface (API)
3 — U.S. BUREAU OF LABOR STATISTICS • bls.gov3 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
CPI Data Uses
Create sample frames
Benchmark samples
Supplement collected data to support hedonic modeling (quality adjustment)
Replace/supplement current data collection methods
4 — U.S. BUREAU OF LABOR STATISTICS • bls.gov4 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Summary: Replacing Collection Initiatives
Almost complete
CorpY – company provided dataset
In progress
CorpX – company provided dataset
JD Power – purchased data
Nielsen – purchased data
5 — U.S. BUREAU OF LABOR STATISTICS • bls.gov5 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Corporate Level Data: CorpY
Great Opportunity
maintain respondent cooperation
reduce respondent burden
work with transaction level data
receive insurance prices
Challenges
Average prices for broader category and aggregated
Data received in format difficult to process
Status: 1st production use is May 2016 Index for monthly quotes
6 — U.S. BUREAU OF LABOR STATISTICS • bls.gov6 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Corporate Level Data: CorpX
Receive sales data monthly by 5th of following month
Great Opportunity
maintain respondent cooperation
reduce burden
work with sales data
7 — U.S. BUREAU OF LABOR STATISTICS • bls.gov7 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Corporate Level Data: CorpX
Challenges
mapping the CorpX item categories to the CPI structure
melding the sales level data into our methodology and current systemo in particular, accommodate seasonality & item substitution
including new methodology
o achieve constant-quality price change w/a big data set
lack of characteristic detail
having enough history to validate method
8 — U.S. BUREAU OF LABOR STATISTICS • bls.gov8 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
CorpX Current Status
I. Received data for all CPI Primary Sampling Units (PSU’s) beginning with October 2014
II. Testing various methodologies
III. Will develop necessary CPI system changes to be ready to use
9 — U.S. BUREAU OF LABOR STATISTICS • bls.gov9 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
JD Power Project
Purchase JD Power dataset as source for replacement in New Vehicles index
Prime example of benefits and challenges of “big data”
Breadth of information
Challenge of integration with current systems
Methodological issues
10 — U.S. BUREAU OF LABOR STATISTICS • bls.gov10 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
New Vehicle Observations
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
400,000
450,000
CPI JDPower
Num
ber
of
Obse
rvations
11 — U.S. BUREAU OF LABOR STATISTICS • bls.gov11 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Model Year Price Indexes
12 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Unit Prices Increase
40
50
60
70
80
90
100
110
120
130Ja
n-0
7
Apr-
07
Jul-07
Oct
-07
Jan-0
8
Apr-
08
Jul-08
Oct
-08
Jan-0
9
Apr-
09
Jul-09
Oct
-09
Jan-1
0
Apr-
10
Jul-10
Oct
-10
Jan-1
1
Apr-
11
Jul-11
Oct
-11
Jan-1
2
Apr-
12
Jul-12
Oct
-12
Jan-1
3
Apr-
13
Jul-13
Oct
-13
Jan-1
4
Apr-
14
Jul-14
Oct
-14
Jan-1
5
Index (
100=
1/2
007)
UnitPriceInx MatchedModelTorn
13 — U.S. BUREAU OF LABOR STATISTICS • bls.gov13 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Ways to Treat the Price Declines
Show the drop
Show price change across model years
Create “Changeover” price relatives
Use Year-Over-Year Index
14 — U.S. BUREAU OF LABOR STATISTICS • bls.gov14 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Price DynamicsAverage Prices (Source: Aizcorbe, Bridgman and Nalewaik(2010))
15 — U.S. BUREAU OF LABOR STATISTICS • bls.gov15 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Price DynamicsAverage Prices (Source: Aizcorbe, Bridgman and Nalewaik(2010))
16 — U.S. BUREAU OF LABOR STATISTICS • bls.gov 16
JDPower vs CPI
90
92
94
96
98
100
102
104
106
108
110
20
07
_1
20
07
_3
20
07
_5
20
07
_7
20
07
_9
20
07
_11
20
08
_1
20
08
_3
20
08
_5
20
08
_7
20
08
_9
20
08
_11
20
09
_1
20
09
_3
20
09
_5
20
09
_7
20
09
_9
20
09
_11
20
10
_1
20
10
_3
20
10
_5
20
10
_7
20
10
_9
20
10
_11
20
11
_1
20
11
_3
20
11
_5
20
11
_7
20
11
_9
20
11
_11
20
12
_1
20
12
_3
20
12
_5
20
12
_7
20
12
_9
20
12
_11
20
13
_1
20
13
_3
20
13
_5
20
13
_7
20
13
_9
20
13
_11
20
14
_1
20
14
_3
20
14
_5
20
14
_7
20
14
_9
20
14
_11
20
15
_1
20
15
_3
Ind
ex (
10
0=0
6/2
00
9)
Proposed JDPower Index
CPI: New Vehicles JDP: YOY + Cycle
17 — U.S. BUREAU OF LABOR STATISTICS • bls.gov17 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Research Nielsen Indexes
Data set for August 2005 – September 2010
2 million UPC codes
Scantrack coverage limitations
Grocery>$2 million; Drug Stores>$1 million; Mass Merchandisers
Excludes one major retailer and non-UPC items (some produce, deli, bakery, fresh meat, etc.)
18 — U.S. BUREAU OF LABOR STATISTICS • bls.gov18 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Nielsen Indexes
18
0
20
40
60
80
100
120
140
FJ011 - Milk
CPI Nielsen
19 — U.S. BUREAU OF LABOR STATISTICS • bls.gov19 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Nielsen Indexes
19
0
20
40
60
80
100
120
140
FR02 - Candy and chewing gum
CPI Nielsen
20 — U.S. BUREAU OF LABOR STATISTICS • bls.gov20 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Nielsen Indexes – Current focus
Refine Nielsen indexes to :
Limit research to items that are well represented in the Scantrack data
Account for product downsizing
Account for UPC “churn”
Calculate a geomeans index (in addition to a Tornqvist index)
21 — U.S. BUREAU OF LABOR STATISTICS • bls.gov21 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Nielsen indexes – Current focus
Preliminary results for 4 item strata
Work on additional 10-12 strata in FY16
95
100
105
110
115
200608
200609
200610
200611
200612
200701
200702
200703
200704
200705
200706
200707
200708
200709
200710
200711
200712
200801
200802
200803
200804
200805
200806
200807
200808
200809
200810
200811
200812
200901
200902
200903
200904
200905
200906
200907
200908
200909
200910
200911
200912
201001
201002
201003
201004
201005
201006
201007
201008
201009
CPI and Nielsen Indexes for FA02 – 0000 Cereal and Cereal Products
CPI TQ (price in t & (t-1); Churn & Dwnsz) Geo (No missing prices, Churn & Dwnsz)
22 — U.S. BUREAU OF LABOR STATISTICS • bls.gov22 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Nielsen Indexes – Current focus
90.00
95.00
100.00
105.00
110.00
115.00
120.00
125.00
130.00
200608
200609
200610
200611
200612
200701
200702
200703
200704
200705
200706
200707
200708
200709
200710
200711
200712
200801
200802
200803
200804
200805
200806
200807
200808
200809
200810
200811
200812
200901
200902
200903
200904
200905
200906
200907
200908
200909
200910
200911
200912
201001
201002
201003
201004
201005
201006
201007
201008
201009
CPI and Nielsen Indexes for FA01 – 0000 Flour & Prepared Flour Mixes
CPI TQ (price in t and (t-1); churn & dwnsz Geo (no missing prices; churn & dwnsz)
23 — U.S. BUREAU OF LABOR STATISTICS • bls.gov23 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Nielsen downsizing
Automate identification
Compare to CPI
$0
$1
$2
$3
$4
Mill
ions
Betty Crocker Fudge Brownie Mix1600019726 - 18.3OZ
1600044830 - 19.8OZ
24 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
Summary: Benefits vs. Challenges
Benefits:
Increasingly more available
Allows for evaluation & improvement
May reduce collection costs
Reduces respondent burden
Increased sample size
May increase data quality
Sometimes ability to get quantity data
Challenges:
Data quality issues –especially lack of descriptive info
Timeliness and reliability concerns – mitigation strategies
Cost and other considerations (new skill set, IT infrastructure, etc.)
25 — U.S. BUREAU OF LABOR STATISTICS • bls.gov25 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
What’s Next
Continue work on CorpX, JD Power, Nielsen
Project to modify CPI production to more readily accept future alternative data
Work with CE to investigate secondary sources for Rent Data
Explore new opportunities
Contact Information
26 — U.S. BUREAU OF LABOR STATISTICS • bls.gov
David FriedmanAssociate Commissioner for Prices & Living
Conditionswww.bls.gov/bls/inflation.htm