ME 535SVD and PCA with the Specific Example of Analysis to Rank NFL Players in Comparison

to Fantasy Football

Andrew ZacharyStudent ID: 1540671

Abstract

Singular Value Decomposition (SVD) and its use for Principal Component Analysis (PCA) are highly powerful techniques for data analysis. Specifically when applied on data without significant prior knowledge of physical or theoretical governing equations. PCA of a dataset will define the ideal basis for the data, describing it with no redundancy. Useful applications for SVD and PCA are numerous, including but not limited to image compression, signal analysis, and turbulent fluid flows. This paper will show the specific, and fun, example of ranking National Football League (NFL) players based on season stats

1. Introduction and Overview

PCA is a method which describes a given dataset based solely on its principal values. However what are principal values. In the current language, principal values are the set of values, or components, which describe a basis for a dataset in which all redundancies have been removed. SVD is a method used to determine what the principal values are. Once the principal values have been found, they can be used to determine the most important points of information, and subsequently can be used to describe the system with a smaller set of data, or can be used to determine rank based on input correlation. These examples with be described further when PCA is applied to football statistics to rank the players of the NFL.

2. Theoretical Background

PCA is effectively a term used to describe the basis of a dataset which contains no redundancies. To understand what this means, the covariance matrix must be addressed. The covariance matrix of the data is,

CX=1

n−1X XT ,

Where X is the dataset and n is the number of data points. CX is an m by m matrix where m is the number of measuring probes (i.e. number of stats for the football analysis). The important features of CX are that the diagonal terms are the variances of the dataset and the off-diagonal terms are the covariances. The variances represent the dynamics of the data with larger variances being those of greater interest. The covariances represent all possible pairs of measurements. In the football analysis this each stat being compared to each player. Large covariances represents a high degree of redundancy between two measurements. In order to find the basis of a set of data with no redundancies, the covariance matrix must be found where all the off-diagonal terms are 0. This action is called diagonalizing the matrix.

There are several methods to diagonalize the covariance matrix. The method used for this analysis is SVD. SVD is a factorization of a matrix which breaks out the rotating and stretching

of a matrix into individual parts. The individual parts of a matrix, A, described by the SVD are three new matrixes. If A is an mxn matrix than the matrixes created by SVD are:

U, an mxm unitary matrix making a set of orthonormal basis vectors

V, an nxn unitary matrix making a set of orthonormal basis vectors

Σ, or S, an mxn diagonal covariance matrix

Where,A=UΣV ¿

A reduces to the diagonal matrix Σ where U and V represent the range and domain respectively. This diagonalization allows for a ranking of the significance of each value of Σ. The diagonal entries of Σ are as follows,

σ 1>σ2>…>σn

To compute U and V consider the following matrix products:

A AT=(U ΣV ¿ ) (U Σ V ¿)T

¿ (U ΣV ¿ )V Σ U ¿

¿U Σ2UA AT U =U Σ2

andAT A=(U ΣV ¿ )T (U Σ V ¿ )

¿V Σ U ¿U Σ V ¿

¿V Σ2V ¿

→ AT AV =V Σ2

This creates two eigenvalue problems where the singular values (the values of Σ) are the square root of the eigenvalues and U and V are the respective eigenvectors.

When SVD is applied to a covariance matrix, then the principal components are the resulting singular values.

3. Ranking NFL Players

A simple to understand example of PCA is to apply it to sports data. The technique can simply and quickly assess the weight of certain stats compared to others as well as be used to rank players or teams based on all available stats. This analysis follows a similar analysis in [Ref. 3] which ranks the players of the 2012 Indian Premier League for cricket.

This project uses Principal Component Analysis to create a ranking algorithm for players in the NFL in various positions. The end result will be a comparison of the ranking based on PCA with

the ranking from a 2017 season fantasy football league. Only positions used by most fantasy football leagues are considered. These positions are quarterbacks, running backs, tight ends, wide receivers, and defense. Kickers are also usually used in fantasy football however there is a one to one correlation between statistics kept for kickers and points scored on fantasy football so the position is omitted due to having trivial results.

The main dataset of player stats is obtained from the NFL website [Ref 1]. The stats provided are shown and described in Appendix B.

The following results are shown for Quarterbacks alone but an extended study with all positions is contained in Appendix B.

The first step is to create a covariance matrix. Following [Ref 3] a correlation matrix will be used instead of a covariance matrix. As is shown below the correlation matrix is simply the covariance matrix divided by the standard deviation of each stat. The example below is given for two inputs but can be expanded for m inputs. This is done using a Pearson correlation coefficient which is a measure of the linear correlation between two variables. It is a value of between -1 and +1 with 0 being no correlation, 1 being total positive correlation, and -1 being total negative linear correlation. It uses the covariance of the two variables and the respective standard deviation for each variable.

ρ ( A , B )= 1N−1∑i=1

N

( A i−μA

σ A)(Bi−μB

σ B)

Where μA and σ A are the mean and stand deviations of variable A respectively and the same for variable B. Another method to define the correlation coefficient is using the covariance of variables A and B (cov(A,B)) and then dividing it by the product of the standard deviations.

ρ ( A ,B )= cov ( A , B )σ A σB

The correlation coefficient of each variable combination is then calculated to create the correlation matrix as shown below. Where R is the correlation matrix for two variables A and B.

R=(ρ( A , A) ρ( A ,B)ρ(B , A ) ρ(B , B))

The diagonal values of the R matrix should be 1 for total positive correlation. R can be expanded for any number of variables. The R matrix for quarterbacks is shown below.

SVD is applied to the R matrix of the NFL stats. The resulting diagonal matrix, Σ, contains the principal components.

Table 1 Correlation Matrix for Quarterbacks

SVD is applied to the R matrix of the NFL stats. The resulting diagonal matrix, Σ, contains the principal components. This plot shows that most of the variance is contained in the first and second principal component. For this analysis just the first principal component was used to rank the players. This is done by the following principal component projection,

Y=V ¿ (: ,1 ) A

Using only the first column of V. Following the singular value plot are plots correlating the ranking of the NFL statistics page with the rankings found here and the ranking from Yahoo Sports 2017 fantasy football with the rankings found here.

Figure 1 Plot showing the singular values for the Quarterbacks.

Figure 2 Plot showing the ranking for Quarterbacks (R-square = 0.9815)

Figure 3 Plot showing the fantasy score versus PCA score for the Quarterbacks

4. Summary and ConclusionsPCA based ranking correlates very well with ranking based on fantasy points. Most R2 values

are greater than 0.83. This is somewhat remarkable considering how many stats were included which had no bearing on fantasy points. It shows the particular fantasy league used for comparison actually did do a good job of ranking players for skill. In fact, PCA based ranking correlates better in most cases with the fantasy rankings than the rankings provided by NFL. The NFL website ranks players based solely on a single statistic. It seems that it is not realistic to predict a players skill based on a single stat. The more intense analysis applied by fantasy leagues or using PCA to rank based on several stats does a better job finding consistency between ranking players. The one exception to this is that PCA applied to defense did not correlate well with fantasy rankings. This is likely due to the fact that many key defensive stats for fantasy were not captured in our set of NFL data. Namely defensive touchdowns which is a huge portion of fantasy points. All in all, it is likely that an advantage can be gained in your fantasy league by hours of analysis as long as you find the right aspects to assess. More importantly, it is likely that an advantage can be gained in most areas of data analysis by in-depth understanding of PCA.

5. References1) NFL website, nfl.com, 2017 Season Stats2) “Data-Driven Modeling and Scientific Computation, Methods for Complex Systems &

Big Data”, J. Nathan Kutz, 20133) “An Introductory application of Principal Components to Cricket Data”, Ananda B. W.

Manage et. al., 2013

Appendix A

This appendix includes the MATLAB code used to create all of the plots and figures.

clear all, close all; filename = 'NFL_Stats.xlsx'; QuarterBacks = xlsread(filename, 'Quarterback', 'E2:T50');RunningBacks = xlsread(filename, 'Running Backs', 'E2:P50');TightEnds = xlsread(filename, 'Tight Ends', 'E2:O50');WideReceivers = xlsread(filename, 'Wide Receivers', 'E2:O50');Defenses = xlsread(filename, 'Defense', 'C2:R33'); QBs = corrcoef(QuarterBacks);RBs = corrcoef(RunningBacks);TEs = corrcoef(TightEnds);WRs = corrcoef(WideReceivers);Ds = corrcoef(Defenses); [UQ,SQ,VQ] = svd(QBs);[UR,SR,VR] = svd(RBs);[UT,ST,VT] = svd(TEs);[UW,SW,VW] = svd(WRs);[UD,SD,VD] = svd(Ds); %% Plotting and evaluating - figure(1)plot(diag(SQ)/sum(diag(SQ)),'b*');title('Singular Values - Quarterbacks')xlabel('Singular values')ylabel('S/sum(S)')figure(2)plot(diag(SR)/sum(diag(SR)),'b*');title('Singular Values - Running Backs')xlabel('Singular values')ylabel('S/sum(S)')figure(3)plot(diag(ST)/sum(diag(ST)),'b*');title('Singular Values - Tight Ends')xlabel('Singular values')ylabel('S/sum(S)')figure(4)plot(diag(SW)/sum(diag(SW)),'b*');title('Singular Values - Wide Receivers')xlabel('Singular values')ylabel('S/sum(S)')figure(5)plot(diag(SD)/sum(diag(SD)),'b*');title('Singular Values - Defenses')xlabel('Singular values')ylabel('S/sum(S)')

%% RanksQBRanks = VQ(:,1)'*QuarterBacks';RBRanks = VR(:,1)'*RunningBacks';TERanks = VT(:,1)'*TightEnds';WRRanks = VW(:,1)'*WideReceivers';DRanks = VD(:,1)'*Defenses'; figure(6)plot(QBRanks,'k*')hold onx = 1:length(QBRanks);p = polyfit(x,QBRanks,1);f = polyval(p,x);[r1 rmse] = rsquare(QBRanks,f);plot(f)title('Quarterback Ranks')xlabel('NFL rank')ylabel('PCA score')r1 figure(7)plot(RBRanks,'k*')hold onx = 1:length(RBRanks);p = polyfit(x,RBRanks,1);f = polyval(p,x);[r1 rmse] = rsquare(RBRanks,f);plot(f)title('Running Backs Ranks')xlabel('NFL rank')ylabel('PCA score')r1 figure(8)plot(TERanks,'k*')hold onx = 1:length(TERanks);p = polyfit(x,TERanks,1);f = polyval(p,x);[r1 rmse] = rsquare(TERanks,f);plot(f)title('Tight Ends Ranks')xlabel('NFL rank')ylabel('PCA score')r1 figure(9)plot(WRRanks,'k*')hold onx = 1:length(WRRanks);p = polyfit(x,WRRanks,1);

f = polyval(p,x);[r1 rmse] = rsquare(WRRanks,f);plot(f)title('Wide Receivers Ranks')xlabel('NFL rank')ylabel('PCA score')r1 figure(10)plot(DRanks,'k*')hold onx = 1:length(DRanks);p = polyfit(x,DRanks,1);f = polyval(p,x);[r1 rmse] = rsquare(DRanks,f);plot(f)title('Defenses Ranks')xlabel('NFL rank')ylabel('PCA score')r1%%QuarterBacks2 = xlsread(filename, 'Quarterback2', 'E2:H51');Defenses2 = xlsread(filename, 'Defense2', 'C2:F33');QBs2 = corrcoef(QuarterBacks2);Ds2 = corrcoef(Defenses2); [UQ2,SQ2,VQ2] = svd(QBs2);[UD2,SD2,VD2] = svd(Ds2); QBRanks2 = VQ2(:,1)'*QuarterBacks2';DRanks2 = VD2(:,1)'*Defenses2'; figure(11)plot(QBRanks2,'k*')hold onx = 1:length(QBRanks2);p = polyfit(x,QBRanks2,1);f = polyval(p,x);[r1 rmse] = rsquare(QBRanks2,f);plot(f)title('Quarterback Ranks with less stats')xlabel('NFL rank')ylabel('PCA score')r1 figure(12)plot(DRanks2,'k*')hold onx = 1:length(DRanks2);p = polyfit(x,DRanks2,1);f = polyval(p,x);[r1 rmse] = rsquare(DRanks2,f);plot(f)

title('Defenses Ranks with less stats')xlabel('NFL rank')ylabel('PCA score')function [r2 rmse] = rsquare(y,f,varargin)% Compute coefficient of determination of data fit model and RMSE%% [r2 rmse] = rsquare(y,f)% [r2 rmse] = rsquare(y,f,c)%% RSQUARE computes the coefficient of determination (R-square) value from% actual data Y and model data F. The code uses a general version of % R-square, based on comparing the variability of the estimation errors % with the variability of the original values. RSQUARE also outputs the% root mean squared error (RMSE) for the user's convenience.%% Note: RSQUARE ignores comparisons involving NaN values.% % INPUTS% Y : Actual data% F : Model fit%% OPTION% C : Constant term in model% R-square may be a questionable measure of fit when no% constant term is included in the model.% [DEFAULT] TRUE : Use traditional R-square computation% FALSE : Uses alternate R-square computation for model% without constant term [R2 = 1 - NORM(Y-F)/NORM(Y)]%% OUTPUT % R2 : Coefficient of determination% RMSE : Root mean squared error%% EXAMPLE% x = 0:0.1:10;% y = 2.*x + 1 + randn(size(x));% p = polyfit(x,y,1);% f = polyval(p,x);% [r2 rmse] = rsquare(y,f);% figure; plot(x,y,'b-');% hold on; plot(x,f,'r-');% title(strcat(['R2 = ' num2str(r2) '; RMSE = ' num2str(rmse)]))% % Jered R Wells% 11/17/11% jered [dot] wells [at] duke [dot] edu%% v1.2 (02/14/2012)%% Thanks to John D'Errico for useful comments and insight which has helped% to improve this code. His code POLYFITN was consulted in the inclusion of% the C-option (REF. File ID: #34765). if isempty(varargin); c = true; elseif length(varargin)>1; error 'Too many input arguments';

elseif ~islogical(varargin{1}); error 'C must be logical (TRUE||FALSE)'else c = varargin{1}; end % Compare inputsif ~all(size(y)==size(f)); error 'Y and F must be the same size'; end % Check for NaNtmp = ~or(isnan(y),isnan(f));y = y(tmp);f = f(tmp); if c; r2 = max(0,1 - sum((y(:)-f(:)).^2)/sum((y(:)-mean(y(:))).^2));else r2 = 1 - sum((y(:)-f(:)).^2)/sum((y(:)).^2); if r2<0 % http://web.maths.unsw.edu.au/~adelle/Garvan/Assays/GoodnessOfFit.html warning('Consider adding a constant term to your model') %#ok<WNTAG> r2 = 0; endend rmse = sqrt(mean((y(:) - f(:)).^2));

Appendix B NFL PCA Analysis

The following tables describe the stats used for analysis and their bearing on fantasy football scores. The dataset is obtained from NFL’s website for the players’ stats [Ref 1].

Table C.2 Stat DescriptionsStat DescriptionComp: Number of pass completions on the season.Att: Number of attempts (passing, receiving, or rushing) on the season.Pct: Percentage of passes completed.Att/G Attempts (passing, receiving, or rushing) per game played.Yds: Number of yards (passing, receiving, or rushing) on the season.Avg: Average yards (passing, receiving, or rushing) per pass.Yds/G: Yards (passing, receiving, rushing, or allowed) per game.TD: Touchdowns on the season.Int: Interceptions thrown on the season.1st: Number of 1st downs on the season1st%: The percent of set of downs when 1st downs were achieved20+: Passes, receptions, or rushes of over 20 yards.40+: Passes, receptions, or rushes of over 40 yards.Sck: Number of sacks on the season.Rate: Quarterback rating on the season.Lng: Longest pass, reception, or rush on the season.FUM: Number of fumbles on the season.Rec: Number of receptions on the season.Pts/G: Points allowed per game.TotPts: Points allowed on the season.Scrm Plays:

Number of plays were the defense was on the field.

Yds/P: Average yards allowed per play.1st/G: Average number of first downs allowed per game.3rd Md: Number of third down conversions given up.3rd Att: Number of third down plays.3rd Pct: Percentage of third downs converted.4rd Md: Number of fourth down conversions given up.4rd Att: Number of fourth down plays.4rd Pct: Percentage of fourth downs converted.Pen: Penalties committed on the season.Pen Yds: Yards given up on penalties committed on the season.ToP/G: Time of possession for the opposing offense per game.Lost: Number of fumbles not recovered by the defense on the season.

The stats that were used for each position are shown below. Stats highlighted green are stats which give positive fantasy points. Stats highlighted red are stats which give negative fantasy points. Non highlighted stats do not affect fantasy points.

Quarterbacks:Comp, Att, Pct, Att/G, Avg, Yds/G, TD, Int, 1st, 1st%, Lng, 20+, 40+, Sck, Rate

Running Backs:Att, Att/G, Yds, Avg, Yds/G, TD, Lng, 1st, 1st%, 20+, 40+, FUM

Tight Ends:Rec, Yds, Avg, Yds/G, Lng, TD, 20+, 40+, 1st, 1st%, FUM

Wide Receivers:Rec, Yds, Avg, Yds/G, Lng, TD, 20+, 40+, 1st, 1st%, FUM

Defense:Pts/G, TotPts, Scrm Plys, Yds/G, 1st/G, 3rd Md, 3rd Att, 3rd Pct, 4th Md, 4th Att, 4th Pct, Pen, Pen Yds, ToP/G, FUM, Lost

When the rankings are compared to the Yahoo Sports Fantasy League the following rules are applied for score keeping:

Point for offensive positions:1 point for every 25 yards passing4 points for every touchdown thrown6 points for every rushing touchdown1 point for every 10 yards rushed-1 point for every interception6 points for every touchdown reception1 point for every 10 receiving yards2 points for every 2 point conversion-2 points for every lost fumble3 points for every field goal 0 to 39 yards4 points for every field goal 40 to 49 yards5 points for every field goal 50+ yards1 point for every point after attempt made

Defensive and special teams points:1 point for every sack caused2 for every interception2 points for every fumble recovery6 points for every touchdown2 points for every safety2 points for every blocked kick6 points for every kick-off or punt return touchdown

2 points for extra point returned

Defense starts with 10 points and keeps the following points based on score allowed10 points for 0 points allowed7 points for 1-6 points allowed4 points for 7-13 points allowed1 point for 14-20 points allowed0 points for 21-27 points allowed-1 points for 28-34 points allowed-4 points for 35+ points allowed

It is important here to note that there is not a perfect correlation between the stats provided by nfl.com and that stats that score points for our Yahoo fantasy league. For example, any player can score fantasy points by making a touchdown pass but this statistic is only kept for quarterbacks. This is quite rare that a non-quarter back will throw a touchdown pass but it does happen and it did happen in the last season. More common is that running back can score points by receiving passes. However, this statistic is also not provided by nfl.com. While there are many more cases like this, the one that most likely affects the correlation between PCA on nfl.com stats and fantasy points is that defenses score a significant amount of points from returning fumbles or interceptions for touchdowns. The number of touchdowns scored is not a stat kept for defenses on nfl.com. In fact, it isn’t even recorded how many interceptions they get. This means that by nfl.com standards a team is bad if they give up a significant amount of yardage on the season. However, they could give up 60 yards per drive on average and end each drive with an interception or interception returned for a touchdown.

The correlation matrix was computed for each position to standardize our dataset. Then Principal Component analysis was performed on the correlation matrix for each position. By using SVD on this correlation matrix we are able to plot the eigenvalues (S values) and view which modes carried the most energy and needed to be included in the ranking. The rank of the singular value matrix was reduced and the principal components were projected onto the original data ranking the players.

On the NFL website, players are ranked by one default stat per position. Results from the analysis here are shown compared to NFL stats and Yahoo fantasy league stats.

Table 3 through Table 7 show the correlation matrices for each position.

Table 3 Correlation Matrix for Quarterbacks

Table 4 Correlation Matrix for Running Backs

Table 5 Correlation Matrix for Tight Ends

Table 6 Correlation Matrix for Wide Receivers

Table 7 Correlation Matrix for Defenses

Figure 4 through Figure 8 show the singular values plots for each position using all of the stats listed in the correlation matrices.

Figure 4 Plot showing the singular values for the Quarterbacks.

Figure 5 Plot showing the singular values for the Running Backs.

Figure 6 Plot showing the singular values for the Tight Ends.

Figure 7 Plot showing the singular values for the Wide Receivers.

Figure 8 Plot showing the singular values for the Defenses.

Figure 9 through Figure 13 show the ranking of the places using this PCA mode versus the NFL ranking.

Figure 9 Plot showing the ranking for Quarterbacks (R-square = 0.9815)

Figure 10 Plot showing the ranking for Running Backs (R-square = 0.4842)

Figure 11 Plot showing the ranking for Tight Ends (R-square = 0.7756)

Figure 12 Plot showing the ranking the Wide Receivers (R-square = 0.5544)

Figure 13 Plot showing the ranking for the Defenses (R-square = 0.6304)

Figure 9 14 through Figure 13re 18 show the ranking of the places using this PCA mode versus the NFL ranking.

Figure 14 Plot showing the fantasy score versus PCA score for the Quarterbacks

Figure 15 Plot showing the fantasy score versus PCA score for the Running Backs

Figure 16 Plot showing the fantasy score versus PCA score for the Tight Ends

Figure 17 Plot showing the fantasy score versus PCA score for the Wide Receivers

Figure 18 Plot showing the fantasy score versus PCA score for the Defenses