1
Computer Mediated Transactions
Hal VarianGoogle April 7
Outline -- what does CMT enable?
1. Data extraction and analysis2. Personalization and customization3. Experimentation and continuous improvement4. Contractual innovation
There is now a computer in the middle of most economic transactions. What does this enable?
Data extraction and analysis
Initial claims: good leading indicator for recessions
Grey bars indicate recessions
Google Correlate with initial claims data
Initial claims and [unemployment filing]
Nowcasting initial claims
Predict NSA initial claims (yt), using lagged values of initial claims and contemporaneous queries on [unemployment filing] (xt)
Base: yt = a0 + a1 yt-1 + a52 yt-52 + et
Trends: yt = a0 + a1 yt-1 + a52 yt-52 + b xt + et
Result: R2 goes from 80.8% to 87.6%
How can we make variable selection easier?Big data
Rows or columns?
How to choose best predictors? Simple correlation?Judgment?Stepwise regression?Lasso, LARS, Elastic Net?
Spike-and-slab regressionKalman filter for trend and seasonality
George-McCulloch [1997]) ;Madigan-Raftery [1994] for regression Prior probability variable is included (spike)
Prior probability distribution over coefficient value (slab) Sample from simulated posterior, average to get predictionSee Scott and Varian (2012, 2013) for detailsDownload R package from CRAN (BoomSpikeSlab, bsts)
New Home Sales in US
Raw correlation
Predictors chosen by model
model: yt = trendt + seasonalt + b1 x1t + b2 x2t
plot1: yt = trendt
plot2: yt = trendt + seasonalt
plot3: yt = trendt + seasonalt + b1 x1t
plot4: yt = trendt + seasonalt + b1 x1t + b2x2t
Incremental fit plots
Visualize how much each predictor contributes to model fit
Trend
Seasonal
[appreciation rate]
[irs 1031]
[century 21 realtors]
[real estate purchase]
[80-20 mortgage]
One month ahead forecast
Does 23% better than simple AR1 model
Geo-amplification
You can do the same thing for any geographically distributed variable
Find out queries or query categories that are predictive of that variable
Make predictions/extrapolations to other geographies
Many applications
Social science
Policy
Marketing
Politics
Example: New York Times index of “hard places” (June 26, 2014)
Where are the hardest places to live in the U.S.?
What queries are associated with “hard places”?
Based on state level data and Google Correlate
What queries are associated with “easy places”?
Based on state level data and Google Correlate
Customization and
personalization
Assembled in America
Predictors of survey response
Top and bottom cities' predicted score
Kershaw, SC: 83.2 %
Summersville, WV: 82.8 %
Grundy, VA: 82.8 %
Chesnee, SC: 82.7 %
Duffield, VA: 82.5 %
Norton, VA: 82.3 %
Jonesville, VA: 82.2 %
Walnut Cove, NC: 82.2 %
Weston, WV: 82.2 %
Ennice, NC: 82.1 %
Calipatria, CA: 40.2 %
Fremont, CA: 40.2 %
Mountain View, CA: 40.8 %
San Jose, CA: 41.4 %
Berkeley, CA: 41.4 %
Redmond, WA: 41.5 %
Glendale, CA: 41.5 %
Cupertino, CA: 41.6 %
Palo Alto, CA: 41.7 %
Daggett, CA: 41.9 %
Top Bottom
Assembled in America by DMA
Experimentation and continuous improvement
“To find out what happens when you change something, it is necessary to change it.”
George Box
Causal inference
Experiments: gold standard for causality
What goes wrong with observational data?
yt = xt b + et = observed + unobserved
Correlation: if you observe x what is a good prediction for y?Causality: what happens to y if you change x?
Confounder: something unobserved that affects both x and y
Advertising
Q: How do your know your advertising works?A: Every December I increase my ad spend...
Advertising
Q: How do your know your advertising works?A: Every December I increase my ad spend...and every December my sales go up!
Advertising
Q: How do your know your advertising works?A: Every December I increase my ad spend...and every December my sales go up!
“Christmas holidays” are a confounding variable. Here the solution is obvious, but what happens if you can’t observe the confounders?
Train, test, treat, compare
1. Train a model on historical data2. Test the model on a holdout3. Apply treatment at some time4. Compare observed outcome with the
treatment to the counterfactual prediction of model
Compare outcome to counterfactual
Actual and natural experiments
You want randomized experiments to reduce systematic effects. Sometimes you get randomization “for free”.
Impact of class size on performance● Why are classes larger in some schools than others?● In Israel maximum class size is 40. Classes with 41 are split in two.● Can identify causal effect of class size on performance
Impact of ad impressions on movie revenueSuper Bowl facts● Ads are bought long before teams are chosen● Home cities of participating teams see elevated viewership● Natural randomization
Experimentation capability should be coded in
static code:const threshold = 3.14if (x > threshold) do something
learning code:param threshold = {3.13, 3.14, 3.15)performance = (num_right, num_wrong)if (x > threshold) do something report performance
Research challenge: How to turn legacy code into learning code?
Nice example: Keith Winstein et al, An Experimental Study of the Learnability of Congestion Control
Contractual innovation
What is a contract?
“If you do this, I’ll do that.”
But how do you verify “this” and “that”?
Can only contract on things that can be observed and verified...
What is a contract?
“If you do this, I’ll do that.”
But how do you verify “this” and “that”?
Can only contract on things that can be observed and verified…
But with a computer in the middle of the transaction, lots more can be verified.
Examples of contracts
● “You take me to my hotel on the best route, I will pay you.”
● “You use the car and send me a monthly payment.”
● “You drive this rental car safely, I will give you a discount.”
● “You display an ad that brings someone to my store, I will pay you.”
Summary
1. Data extraction and analysisa. Can use searches to nowcast economic activity
2. Personalization and customizationa. Can customize ads to different geos
3. Experimentation and continuous improvementa. Can use ML to estimate causal impact via train-test-
treat-compare cycle4. Contractual innovation
a. As more things become observable, more contracts become viable
Appendix
Advertise a movie about surfing
Honolulu: $1 ad spend $10 ticket salesFargo: $0.10 ad spend $1 ticket sales
Ticket sales = 10 x ad spend
fits the data perfectly...
Advertise a movie about surfing
Honolulu: $1 ad spend $10 ticket salesFargo: $0.10 ad spend $1 ticket sales
Ticket sales = 10 x ad spend
fits the data perfectly...
But do you really believe that if you increased spend to $1 in Fargo, you would get 10 times the ticket sales?
Ads and confounders
“Interest in surfing” is a confounding variable
Happens all the time in economics since people choose x (observing things you don’t observe.)
Causal effect of college on education?Causal effect of fertilizer on yield?Causal effect of health care on income?
Superbowl as a natural ad experiment
1. Viewership in home cities of teams that are playing is about 10-15% higher than elsewhere.
2. Ads are purchased long before it is known who is playing
Advertiser buys ad slot, then 2-3 months later two “random” cites get 10-15% more ad exposure.
Regression discontinuity
Impact of class size on performance● Why are classes larger in some schools than others?● In Israel maximum class size is 40. Classes with 41 are split in two.● Can identify causal effect of class size on performance
What would happen to auto fatalities if you changed the minimum drinking age?● 20.5 year olds are a lot like 21.5 year olds● So looking at people on each side of the threshold can give estimate of
causal effect
Regression discontinuity