The COTOR ChallengeThe COTOR Challenge
Committee on the Theory of RiskCommittee on the Theory of Risk
November 2004 Annual MeetingNovember 2004 Annual Meeting
History of the ChallengeHistory of the Challenge
Last spring a COTOR member challenged Last spring a COTOR member challenged actuarial geeks to estimate 500k xs 500k actuarial geeks to estimate 500k xs 500k layer based on list of 250 claimslayer based on list of 250 claims
Emails flew back and forth furiouslyEmails flew back and forth furiously A number of different approaches were usedA number of different approaches were used Literature about heavy tailed distributions Literature about heavy tailed distributions
was recommendedwas recommended Winner was Phil Heckman using mixture of 2 Winner was Phil Heckman using mixture of 2
lognormalslognormals
History cont.History cont.
Criticism existed around the sample since Criticism existed around the sample since some sample statistics were too far from some sample statistics were too far from the real distributionthe real distribution
COTOR feels that the solution of this COTOR feels that the solution of this problem is of interest ot the actuarial problem is of interest ot the actuarial communitycommunity• Our data is almost never normal/lognormalOur data is almost never normal/lognormal• Our data is typically heavy tailedOur data is typically heavy tailed• It is likely that in many real situations, a It is likely that in many real situations, a
sample of 250 claims would not represent a sample of 250 claims would not represent a random draw from any distributionrandom draw from any distribution
History cont.History cont.
Another challenge was issued under Another challenge was issued under well defined conditionswell defined conditions
Stuart Klugman picked the sampleStuart Klugman picked the sample 250 claims randomly generated from 250 claims randomly generated from
an inverse transformed gammaan inverse transformed gamma Challenge was to estimate severity in Challenge was to estimate severity in
the $5M xs $5M layer (mean and the $5M xs $5M layer (mean and 95% confidence intervals)95% confidence intervals)
The SampleThe Sample
Claim SizeClaim Size CountCount
Greater than 5,000,000Greater than 5,000,000 11
500,000 to 1,000,000500,000 to 1,000,000 22
100,000 to 500,000100,000 to 500,000 77
50,000 to 100,00050,000 to 100,000 1010
25,000 to 50,00025,000 to 50,000 88
10,000 to 25,00010,000 to 25,000 2626
5,000 to 10,0005,000 to 10,000 3030
2,500 to 5,0002,500 to 5,000 5656
1,000 to 2,5001,000 to 2,500 7474
500 to 1,000500 to 1,000 3232
250 to 500250 to 500 44
Under 250Under 250 00
250 claims randomly selected from an inverse transformed gamma250 claims randomly selected from an inverse transformed gamma
Purpose of SessionPurpose of Session
Raise awareness of audience of how Raise awareness of audience of how frequently extreme values need to frequently extreme values need to be dealt withbe dealt with
Present relatively easy to use Present relatively easy to use approachesapproaches
Make audience aware of how difficult Make audience aware of how difficult this problem is to solvethis problem is to solve
Normal Distribution AssumptionNormal Distribution Assumption
The normal or lognormal assumption is The normal or lognormal assumption is common in finance applicationcommon in finance application• Option pricing theoryOption pricing theory
• Value at riskValue at risk
• CAPMCAPM
Evidence that asset return data does not Evidence that asset return data does not follow the normal distribution is widely follow the normal distribution is widely availableavailable• 1968 Fama paper in Journal of the American 1968 Fama paper in Journal of the American
Statistical AssociationStatistical Association
Test of Normal Distribution Test of Normal Distribution AssumptionAssumption
Normal Q-Q Plot of Monthly Return on S&P
0.8 0.9 1.0 1.1 1.2 1.3
Observed Value
0.85
0.90
0.95
1.00
1.05
1.10
1.15
Theoretical
Value
Test of Normal Distribution Test of Normal Distribution AssumptionAssumption
Descriptive Statistics
251 .9931 .04585 1.410 .154 6.081 .306
251
Monthly Return on S&P
Valid N (listwise)
Statistic Statistic Statistic Statistic Std. Error Statistic Std. Error
N Mean Std.Deviation
Skewness Kurtosis
Consequences of Assuming Consequences of Assuming NormalityNormality
The frequency of extreme events is The frequency of extreme events is underestimated – often by a lotunderestimated – often by a lot
Example: Long Term CapitalExample: Long Term Capital• ““Theoretically, the odds against a loss such as Theoretically, the odds against a loss such as
August’s had been prohibitive, such a debacle August’s had been prohibitive, such a debacle was, according to mathematicians, an event so was, according to mathematicians, an event so freakish as to be unlikely to occur even once freakish as to be unlikely to occur even once over the entire life of the universe and even over the entire life of the universe and even over numerous repetitions of the universe”over numerous repetitions of the universe” When Genius FailedWhen Genius Failed by Roger Lowenstein, p. 159 by Roger Lowenstein, p. 159
Criteria for JudgingCriteria for Judging
New and creative way to solve the New and creative way to solve the problemproblem
Methodology that practicing Methodology that practicing actuaries can useactuaries can use
Clarity of expositionClarity of exposition Accuracy of known answerAccuracy of known answer Estimates of confidence intervalEstimates of confidence interval
Table of ResultsTable of Results
RespondResponderer
MeanMean Lower Lower CLCL
Upper Upper CLCL
MethodMethod
AA 9,500.009,500.00 450.00450.00 17,500.017,500.000
Inverse Logistic SmootherInverse Logistic Smoother
BB 6,000.006,000.00 0.000.00 26,000.026,000.000
Kernel Smoothing/BootstrappingKernel Smoothing/Bootstrapping
CC 12,533.012,533.000
2,976.002,976.00 53,049.053,049.000
Log Regression of Density Function on Log Regression of Density Function on large claimslarge claims
DD 2,400.002,400.00 ?? ?? Generalized ParetoGeneralized Pareto
EE 6,430.006,430.00 1,760.001,760.00 14,710.014,710.000
Fit distributions to triple logged data. Fit distributions to triple logged data. Used Bayesian approach for mean Used Bayesian approach for mean and CIand CI
F1F1 10,282.010,282.000
2,089.002,089.00 24,877.024,877.000
Scaled ParetoScaled Pareto
F2F2 30,601.030,601.000
6,217.006,217.00 74,038.074,038.000
ParetoPareto
GG 4,332.654,332.65 297.34297.34 7,645.867,645.86 Empirical Semi SmoothingEmpirical Semi Smoothing
H1H1 2,700.002,700.00 0.000.00 17,955.017,955.000
Single Parameter Pareto/Simulation Single Parameter Pareto/Simulation for Confidence Intervalsfor Confidence Intervals
H2H2 8,772.008,772.00 0.000.00 54,474.054,474.000
Generalized Pareto/Bayesian Generalized Pareto/Bayesian SimulationSimulation
True True MeanMean
6810.006810.00
Observations Regarding Observations Regarding ResultsResults
These estimations are not easyThese estimations are not easy Nearly 13 to 1 spread between lowest and Nearly 13 to 1 spread between lowest and
highest meanhighest mean Only 10% of answers came within 10% of Only 10% of answers came within 10% of
right resultright result All responders recognized tremendous All responders recognized tremendous
uncertainty in results (range from upper to uncertainty in results (range from upper to lower CL went from 8 to infinity)lower CL went from 8 to infinity)
Our statistical expert could not understand Our statistical expert could not understand the description of the method of 30% of the description of the method of 30% of the respondentsthe respondents
ObservationsObservations All but 2 of the methods relied on approaches commonly All but 2 of the methods relied on approaches commonly
found in the literature on heavy tailed distributions and found in the literature on heavy tailed distributions and extreme valuesextreme values
It is clear that it is very difficult to get accurate estimates It is clear that it is very difficult to get accurate estimates from a small samplefrom a small sample
The real world is even more challenging than thisThe real world is even more challenging than this• 250 claims probably don’t follow any known distribution250 claims probably don’t follow any known distribution• TrendTrend• DevelopmentDevelopment• Unforeseen changes in environmentUnforeseen changes in environment• Consulting with claims adjusters and underwriters should Consulting with claims adjusters and underwriters should
provide valuable additional insightsprovide valuable additional insights
ObservationsObservations The closest answer was 5% below the true The closest answer was 5% below the true
meanmean Half of the responses below the true mean, Half of the responses below the true mean,
Half were aboveHalf were above Average response was 40% higher than the Average response was 40% higher than the
meanmean Average response (ex outlyer) was within 2% Average response (ex outlyer) was within 2%
of the meanof the mean Read: Read:
““The Wisdom of Crowds: Why the Many are Smarter The Wisdom of Crowds: Why the Many are Smarter than the than the Few and How Collective Wisdom Shapes Few and How Collective Wisdom Shapes Business, Economics, Business, Economics, Societies and Nations”Societies and Nations”
by: James Surowieckiby: James Surowiecki
Implications for Insurance Companies?Implications for Insurance Companies?
SpeakersSpeakers
MeyersMeyers EvansEvans FlynnFlynn WoolstenhulmeWoolstenhulme VenterVenter HeckmanHeckman
Announcement of WinnersAnnouncement of Winners
Louise Francis – COTOR ChairLouise Francis – COTOR Chair
Possible Next StepsPossible Next Steps
Make the results of the challenge Make the results of the challenge available to the membershipavailable to the membership
COTOR subcommittee to evaluate COTOR subcommittee to evaluate how to make techniques readily how to make techniques readily availableavailable
Another round making the challenge Another round making the challenge more real worldmore real world
Include trend and development Include trend and development Give multiple random samplesGive multiple random samples