February 19, 2010

How to Catch a Tiger:

Understanding Putting Performance on the PGA TOUR

Jason AcimovicMIT Operations Research Center, acimovic@mit.edu

Douglas FearingMIT Operations Research Center, dfearing@mit.edu

Professor Stephen GravesMIT Sloan School of Management, sgraves@mit.edu

February 19, 2010

Agenda

2

• Introduction– Project Question

– Applications

– Approach and contribution

• Golf and data overview

• Putting model

• Off-green model

• Situational analysis

February 19, 2010

Project Question

• How well do people perform on tasks?– Tasks differ from each other

– Not everyone performs every task

– Even the same task can be different from person to person

3

February 19, 2010

Applications

• Evaluating employees in a distribution center– Pickers in a warehouse vary in skill (picks per hour)

– Pick zones vary in difficulty (books vs. electronics)

– Difficulty also varies by hour of day and day of week

– Pickers shift around, but not enough to ensure perfect mixing

– How do you compensate the best employees and identify underperformers?

• Golf putting– Different golfers play different tournaments

– Greens vary in their difficulty

– Different golfers start on the green from different distances

– How do we identify the best putters?

4

February 19, 2010

Project approach and contribution

• Develop statistical models to predict strokes-to-go

• Correct for player skill and course difficulty

• Evaluate incremental value of each shot taken relative to the expectation for the field– Compare predicted strokes-to-go before and after shot

• Aggregate shot value across players, shot types, etc. to better understand player performance

• Compare our model to current metrics, namely, Putting Average

• Paper: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1538300 (or email us)

5

February 19, 2010

Agenda

6

• Introduction

• Golf and data overview– Strokes-to-go example

– ShotLink data

• Putting model

• Off-green model

• Situational analysis

February 19, 2010

Quick golf primer

• The goal is to get from the tee to the pin in the fewest number of strokes

• 18 holes in a round of golf

• Typically 4 rounds in a tournament

• Lowest total score wins

7

TeeTee

GreenGreen

FairwayFairway

February 19, 2010

Strokes-to-go example

Shot Location Strokes-To-Go

1 4.4

3 1.8

2 3.0

Shot Gain

0.4

0.8

0.2

8

4.4 – 3.0 – 1 = 0.4

February 19, 2010

ShotLink Data

9

• Every tournament, 250 volunteers gather data on every shot– Lasers pinpoint the ball location to within an inch

– Field volunteers gather qualitative characteristics

• Data is used for both real time reporting as well as detailed analyses

• 5 Million shot data points

• 2 Million putt data points

February 19, 2010

Visual explanation of ShotLinkTM dataset

CourseYearRound NumberHole NumberTee LocationBall LocationPin LocationPlayerShot NumberLocation TypeBall LieHole ParStimp ReadingGreen Length

X Coordinate

Y Coordinate

Z Coordinate

16th Hole on Colonial

10

X Coordinate

Y Coordinate

Z Coordinate

February 19, 2010

Data for the 14th hole at Quail Hollow – 1 day

11

February 19, 2010

Agenda

12

• Introduction• Golf and data overview• Putting model

– Empirical data

– Two stage model• Holing out submodel

• Distance-to-go submodel

– Markov chain

– Correct for hole difficulty and player skill

– Putts-gained per round and results

• Off-green model• Situational analysis

February 19, 2010

Empirical mean and std. dev. of putts-to-go

Mean Std. Dev.

13

February 19, 2010

Two-stage model to predict putts-to-go

• First stage sub-model– From anywhere on the green, the first model predicts the

probability of sinking the putt

14

Probability of 0.1 of making it in on this putt

February 19, 2010

• Second stage sub-model– If the golfer misses the putt, the second model calculates

the distribution of the distance-to-go for the green

If I miss, I have a 0.0021 probability of being in this blue area. (calculate this for entire green)

Second stage finds conditional distance-to-go

15

February 19, 2010

• We can calculate the putts-to-go distribution from anywhere on the green

Combine and …

16

Consider only distance in our

model

Consider only distance in our

model

February 19, 2010

Empirical probabilities of holing out

17

Empirical probability of holing out vs. distance

February 19, 2010

Normal regression is inappropriate

• With Ordinary Least Squares regression, “one” might predict the probability of making a putt based on starting distance….

• But…– We want to predict a probability with a range between 0 and 1

– Errors are not normal

18

0 1Y d

February 19, 2010

One-putt logistic regression model

• Y – putts-to-go

• d – initial distance to the pin

• Fitted model parameters:

• Probability:

19

41 5

1

0 4

[ 1| ]

1 e logxp ( )+

P Y d

d dd

L

0 5, ,

February 19, 2010

Model holing out as a logistic regression

20

Model probability of holing out vs. distance

February 19, 2010

2nd-stage problem, determining distance-to-go

• What happens if we miss the first putt?

21

z

February 19, 2010

Empirical mean and std. dev. of distance-to-go

Mean Std. Dev.

22

February 19, 2010

Empirical distributions of distance-to-go

From 10 ft. From 30 ft.

23

February 19, 2010

Distance-to-go gamma regression model

• d – initial distance to the pin

• z – distance-to-go (assuming a miss)

• Fitted model parameters:

• Mean:

• Density:

24

0 3,Shape ( ) ,,k 2

2 30 1exp{ log }d d d d

( | ) ( ; , )df z d gamma z k

1 dzk

k

d

ekzk

/

( )

February 19, 2010

Distance-to-go model: mean and std. dev.

Mean Std. Dev.

25April 21, 2023

February 19, 2010

Distance-to-go model distributions

From 10 ft. From 30 ft.

26

February 19, 2010

Putts-to-go as Markov chain

27

distanceH

p = 1

p = [ 1 + exp(…) ]-1 g (z|d) = (1 - [ 1 + exp(…) ]-1) x f(z|d)

z

Whereg(z|d): probability density of ending up at z conditioned on starting at d

f(z|d) probability density of ending up at z conditioned on missing and starting at d(from the distance-to-go gamma regression model)

d

Probability of holing out in n putts is probability of reaching absorbing state in n transitions

Probability of holing out in n putts is probability of reaching absorbing state in n transitions

February 19, 2010

Making it within n putts (model prediction)

• Over 90% of golfers 2-putt or better within 35 ft.

• Only a 1.6% chance of 4-putting or worse at 100 ft.

28

Two-Stage Model Within N Putts

February 19, 2010

Two-stage model mean and std. dev.

Mean Std. Dev.

29

February 19, 2010

Comparing putt quality

• Greens vary in difficulty– Fast vs. slow greens

– Type and length of grass

• Good putts on a hard green should be valued more than the same on an easy green

• Adjust parameters for each hole to the logistic and gamma regression models

30

February 19, 2010

Revised logistic and gamma regressions

• Every player p and hole h have their own dummy variables and specific holing-out probabilities*

– Ip is the indicatory variable, and is equal to 1 if observation i contains player p and is zero otherwise.

– Instead of a regression with 6 parameters, we now have thousands of parameters• E.g., there is a β0h parameter for every hole

31

1

0

54

0 1 4

1 0

.{ log

( 1)

..

1 exp}p h

p

ip p h

h

d

I d I

d d

P Y

*The actual analysis accounts for the number of observations per player and per hole, so that the model is more complex for players about whom we know more.

The gamma regression is adjusted similarly

The gamma regression is adjusted similarly

February 19, 2010

Visualizing player skill level differences

32

• Comparison of above average (Brent Geiberger), below average (John Huston), and field average putter for an average green

February 19, 2010

Visualizing green difficulty differences

• Comparison of an easy green (Bay Hill #9), difficult green (Sawgrass #1), and average green based on a field average golfer

33

February 19, 2010

Calculating putts gained per round

• Calculate the gain associated with each putt– Relative to the putts-to-go for each specific hole

– Example: Golfer starts at 12 ft. and takes 2 putts to sink ball• Expected putts-to-go: 1.71

• Actual number of putts: 2

• Relative gain: (- 0.29)

• Sum the relative gains for each player

• Divide by the number of rounds played

34

12 feet1.71 putts to go

February 19, 2010

Top 10 putts gained per round

35

Rank GolferPutts Gained /

RoundNumber of

RoundsPutts Gained / Round Stdev

1 Tiger Woods 0.69 230 0.12

2 David Frost 0.67 113 0.16

3 Fredrik Jacobson 0.56 248 0.11

4 Nathan Green 0.55 197 0.12

5 Aaron Baddeley 0.53 303 0.10

6 Jesper Parnevik 0.50 315 0.10

7 Stewart Cink 0.49 375 0.09

8 Darren Clarke 0.45 107 0.17

9 Ben Crane 0.44 273 0.11

10 Willie Wood 0.42 72 0.20

February 19, 2010

Putting average is the most popular metric today

• Putting Average– Average number of putts per green*

• When a golfer reaches a green– Count the putts it takes to get it in the hole

– Average this among all his green appearances

– Regardless of how close he starts on the green

36

*Actually, a green in regulation, which means the green was reached in no more than (par – 2) strokes

February 19, 2010

Comparing with putting average

37

GolferPutts Gained /

RoundPG/RRank Putting Average

PARank

Tiger Woods 0.69 1 1.71 1

David Frost 0.67 2 1.77 60

Fredrik Jacobson 0.56 3 1.74 4

Nathan Green 0.55 4 1.74 5

Aaron Baddeley 0.53 5 1.74 3

Jesper Parnevik 0.50 6 1.76 47

Stewart Cink 0.49 7 1.75 12

Darren Clarke 0.45 8 1.75 19

Ben Crane 0.44 9 1.75 17

Willie Wood 0.42 10 1.77 92

February 19, 2010

Understanding the discrepancies

• Insert first-putt distance histograms for most severe outlier.

38

PG/R Percentile Golfer

Putts Gained / Round Putting Average

PA Percentile

9th Stephen Leaney 0.26 1.79 59th

88th Ernie Els -0.63 1.75 5th

•54% for All Players•51% for Stephen Leaney•60% for Ernie Els

Percentage of 1st putts 20 ft. or closer

On average he starts closer to the hole, so his putting average is

inflated by his excellent approach

shots

On average he starts closer to the hole, so his putting average is

inflated by his excellent approach

shots

February 19, 2010

Agenda

39

• Introduction

• Golf and data overview

• Putting model

• Off-green model

• Situational analysis

February 19, 2010

Evaluating off-green performance

• For each hole, calculate “field par”– Empirical average number of strokes corrected for player

skill and hole difficulty

• Calculate total strokes gained per round for each player

• Calculate off-green strokes gained per round

40

(Off-green strokes gained = Total strokes gained – putts gained)

February 19, 2010

Top 10 golfers (on and off green performance)

41

Rank GolferPutts Gained /

RoundOff-Green Gain /

Round Total

1 Tiger Woods 0.69 2.53 3.22

2 Vijay Singh -0.36 2.65 2.29

3 Jim Furyk 0.00 2.03 2.03

4 Phil Mickelson 0.19 1.74 1.94

5 Ernie Els -0.63 2.48 1.85

6 Adam Scott 0.08 1.69 1.77

7 Sergio Garcia -0.67 2.20 1.52

8 David Toms 0.16 1.27 1.43

9 Retief Goosen -0.44 1.84 1.40

10 Stewart Cink 0.49 0.89 1.39

February 19, 2010

Agenda

42

• Introduction

• Golf and data overview

• Putting model

• Off-green model

• Situational analysis– Player specific putts

– Fourth round pressure

– Tiger woods’ fourth round performance

February 19, 2010

Situational putting performance

• Above, we used the general putting model to evaluate putting relative to the field of professionals

• We also have the capability to evaluate a golfer’s putting relative to his own expected performance

• For instance, even if Tiger Woods usually putts better than the field, we can also determine when he putts worse than himself– Does he putt better or worse after the cut?

– Does he putt better or worse for birdie vs. for par?

43

February 19, 2010

Player-specific putts gained – example

• On the 10th green at Quail Hollow, 9 feet from the pin:– Tiger Woods’ personal expected putts-to-go is 1.54

– Vijay Singh’s personal expected putt-to-go is 1.59

– If they each sink it, Tiger gains only 0.54 strokes whereas Vijay gains 0.59 strokes

44

Tiger: E[putts] = 1.54Tiger: E[putts] = 1.54Vijay: E[putts] = 1.59Vijay: E[putts] = 1.59

9ft9ft

February 19, 2010

Advantages of player-specific putts gained

• Easy to test various hypotheses– After calculating the shot value for every putt, we need

only to filter and aggregate the results

• Describes the magnitude in terms of score impact

• Suggests areas for further investigation– Standard deviation of putts gained provides the relative

significance of the effect

45

February 19, 2010

Fourth round pressure

46

• Putting does not seem to be affected by the pressures of being in the fourth round

Putt CountPutts Gained Per

PuttPutts Gained Per

Putt Deviation

3rd Round 359,079 0.00237 0.00027

4th Round 353,979 0.00246 0.00027

Difference 0.00009 0.00038

February 19, 2010

Tiger Woods’ fourth round performance

• A common perception is that Tiger has the ability to kick it up a notch during the final round

• Looking at his putts-gained suggests otherwise

47

Putt CountPutts Gained Per

PuttPutts Gained Per

Putt Deviation

1st Round 1,614 0.00036 0.00386

2nd Round 1,589 0.00847 0.00395

3rd Round 1,654 -0.00293 0.00375

4th Round 1,671 -0.00022 0.00380

February 19, 2010

Conclusion

• Developed a model for putting– Corrected for player skill and hole difficulty

– Intuitive model that describes how putts occur

• Demonstrated the differences between our metric and current putting statistics

• Developed a “field par” which corrects for hole difficulty and quality of field

• Compared on- and off-green performance

• Examined situational putting performance

48