The COTOR ChallengeThe COTOR Challenge

Committee on the Theory of RiskCommittee on the Theory of Risk

November 2004 Annual MeetingNovember 2004 Annual Meeting

History of the ChallengeHistory of the Challenge

Last spring a COTOR member challenged Last spring a COTOR member challenged actuarial geeks to estimate 500k xs 500k actuarial geeks to estimate 500k xs 500k layer based on list of 250 claimslayer based on list of 250 claims

Emails flew back and forth furiouslyEmails flew back and forth furiously A number of different approaches were usedA number of different approaches were used Literature about heavy tailed distributions Literature about heavy tailed distributions

was recommendedwas recommended Winner was Phil Heckman using mixture of 2 Winner was Phil Heckman using mixture of 2

lognormalslognormals

History cont.History cont.

Criticism existed around the sample since Criticism existed around the sample since some sample statistics were too far from some sample statistics were too far from the real distributionthe real distribution

COTOR feels that the solution of this COTOR feels that the solution of this problem is of interest ot the actuarial problem is of interest ot the actuarial communitycommunity• Our data is almost never normal/lognormalOur data is almost never normal/lognormal• Our data is typically heavy tailedOur data is typically heavy tailed• It is likely that in many real situations, a It is likely that in many real situations, a

sample of 250 claims would not represent a sample of 250 claims would not represent a random draw from any distributionrandom draw from any distribution

History cont.History cont.

Another challenge was issued under Another challenge was issued under well defined conditionswell defined conditions

Stuart Klugman picked the sampleStuart Klugman picked the sample 250 claims randomly generated from 250 claims randomly generated from

an inverse transformed gammaan inverse transformed gamma Challenge was to estimate severity in Challenge was to estimate severity in

the $5M xs $5M layer (mean and the $5M xs $5M layer (mean and 95% confidence intervals)95% confidence intervals)

The SampleThe Sample

Claim SizeClaim Size CountCount

Greater than 5,000,000Greater than 5,000,000 11

500,000 to 1,000,000500,000 to 1,000,000 22

100,000 to 500,000100,000 to 500,000 77

50,000 to 100,00050,000 to 100,000 1010

25,000 to 50,00025,000 to 50,000 88

10,000 to 25,00010,000 to 25,000 2626

5,000 to 10,0005,000 to 10,000 3030

2,500 to 5,0002,500 to 5,000 5656

1,000 to 2,5001,000 to 2,500 7474

500 to 1,000500 to 1,000 3232

250 to 500250 to 500 44

Under 250Under 250 00

250 claims randomly selected from an inverse transformed gamma250 claims randomly selected from an inverse transformed gamma

Purpose of SessionPurpose of Session

Raise awareness of audience of how Raise awareness of audience of how frequently extreme values need to frequently extreme values need to be dealt withbe dealt with

Present relatively easy to use Present relatively easy to use approachesapproaches

Make audience aware of how difficult Make audience aware of how difficult this problem is to solvethis problem is to solve

Normal Distribution AssumptionNormal Distribution Assumption

The normal or lognormal assumption is The normal or lognormal assumption is common in finance applicationcommon in finance application• Option pricing theoryOption pricing theory

• Value at riskValue at risk

• CAPMCAPM

Evidence that asset return data does not Evidence that asset return data does not follow the normal distribution is widely follow the normal distribution is widely availableavailable• 1968 Fama paper in Journal of the American 1968 Fama paper in Journal of the American

Statistical AssociationStatistical Association

Test of Normal Distribution Test of Normal Distribution AssumptionAssumption

Normal Q-Q Plot of Monthly Return on S&P

0.8 0.9 1.0 1.1 1.2 1.3

Observed Value

0.85

0.90

0.95

1.00

1.05

1.10

1.15

Theoretical

Value

Test of Normal Distribution Test of Normal Distribution AssumptionAssumption

Descriptive Statistics

251 .9931 .04585 1.410 .154 6.081 .306

251

Monthly Return on S&P

Valid N (listwise)

Statistic Statistic Statistic Statistic Std. Error Statistic Std. Error

N Mean Std.Deviation

Skewness Kurtosis

Consequences of Assuming Consequences of Assuming NormalityNormality

The frequency of extreme events is The frequency of extreme events is underestimated – often by a lotunderestimated – often by a lot

Example: Long Term CapitalExample: Long Term Capital• ““Theoretically, the odds against a loss such as Theoretically, the odds against a loss such as

August’s had been prohibitive, such a debacle August’s had been prohibitive, such a debacle was, according to mathematicians, an event so was, according to mathematicians, an event so freakish as to be unlikely to occur even once freakish as to be unlikely to occur even once over the entire life of the universe and even over the entire life of the universe and even over numerous repetitions of the universe”over numerous repetitions of the universe” When Genius FailedWhen Genius Failed by Roger Lowenstein, p. 159 by Roger Lowenstein, p. 159

Criteria for JudgingCriteria for Judging

New and creative way to solve the New and creative way to solve the problemproblem

Methodology that practicing Methodology that practicing actuaries can useactuaries can use

Clarity of expositionClarity of exposition Accuracy of known answerAccuracy of known answer Estimates of confidence intervalEstimates of confidence interval

Table of ResultsTable of Results

RespondResponderer

MeanMean Lower Lower CLCL

Upper Upper CLCL

MethodMethod

AA 9,500.009,500.00 450.00450.00 17,500.017,500.000

Inverse Logistic SmootherInverse Logistic Smoother

BB 6,000.006,000.00 0.000.00 26,000.026,000.000

Kernel Smoothing/BootstrappingKernel Smoothing/Bootstrapping

CC 12,533.012,533.000

2,976.002,976.00 53,049.053,049.000

Log Regression of Density Function on Log Regression of Density Function on large claimslarge claims

DD 2,400.002,400.00 ?? ?? Generalized ParetoGeneralized Pareto

EE 6,430.006,430.00 1,760.001,760.00 14,710.014,710.000

Fit distributions to triple logged data. Fit distributions to triple logged data. Used Bayesian approach for mean Used Bayesian approach for mean and CIand CI

F1F1 10,282.010,282.000

2,089.002,089.00 24,877.024,877.000

Scaled ParetoScaled Pareto

F2F2 30,601.030,601.000

6,217.006,217.00 74,038.074,038.000

ParetoPareto

GG 4,332.654,332.65 297.34297.34 7,645.867,645.86 Empirical Semi SmoothingEmpirical Semi Smoothing

H1H1 2,700.002,700.00 0.000.00 17,955.017,955.000

Single Parameter Pareto/Simulation Single Parameter Pareto/Simulation for Confidence Intervalsfor Confidence Intervals

H2H2 8,772.008,772.00 0.000.00 54,474.054,474.000

Generalized Pareto/Bayesian Generalized Pareto/Bayesian SimulationSimulation

True True MeanMean

6810.006810.00

Observations Regarding Observations Regarding ResultsResults

These estimations are not easyThese estimations are not easy Nearly 13 to 1 spread between lowest and Nearly 13 to 1 spread between lowest and

highest meanhighest mean Only 10% of answers came within 10% of Only 10% of answers came within 10% of

right resultright result All responders recognized tremendous All responders recognized tremendous

uncertainty in results (range from upper to uncertainty in results (range from upper to lower CL went from 8 to infinity)lower CL went from 8 to infinity)

Our statistical expert could not understand Our statistical expert could not understand the description of the method of 30% of the description of the method of 30% of the respondentsthe respondents

ObservationsObservations All but 2 of the methods relied on approaches commonly All but 2 of the methods relied on approaches commonly

found in the literature on heavy tailed distributions and found in the literature on heavy tailed distributions and extreme valuesextreme values

It is clear that it is very difficult to get accurate estimates It is clear that it is very difficult to get accurate estimates from a small samplefrom a small sample

The real world is even more challenging than thisThe real world is even more challenging than this• 250 claims probably don’t follow any known distribution250 claims probably don’t follow any known distribution• TrendTrend• DevelopmentDevelopment• Unforeseen changes in environmentUnforeseen changes in environment• Consulting with claims adjusters and underwriters should Consulting with claims adjusters and underwriters should

provide valuable additional insightsprovide valuable additional insights

ObservationsObservations The closest answer was 5% below the true The closest answer was 5% below the true

meanmean Half of the responses below the true mean, Half of the responses below the true mean,

Half were aboveHalf were above Average response was 40% higher than the Average response was 40% higher than the

meanmean Average response (ex outlyer) was within 2% Average response (ex outlyer) was within 2%

of the meanof the mean Read: Read:

““The Wisdom of Crowds: Why the Many are Smarter The Wisdom of Crowds: Why the Many are Smarter than the than the Few and How Collective Wisdom Shapes Few and How Collective Wisdom Shapes Business, Economics, Business, Economics, Societies and Nations”Societies and Nations”

by: James Surowieckiby: James Surowiecki

Implications for Insurance Companies?Implications for Insurance Companies?

SpeakersSpeakers

MeyersMeyers EvansEvans FlynnFlynn WoolstenhulmeWoolstenhulme VenterVenter HeckmanHeckman

Announcement of WinnersAnnouncement of Winners

Louise Francis – COTOR ChairLouise Francis – COTOR Chair

Possible Next StepsPossible Next Steps

Make the results of the challenge Make the results of the challenge available to the membershipavailable to the membership

COTOR subcommittee to evaluate COTOR subcommittee to evaluate how to make techniques readily how to make techniques readily availableavailable

Another round making the challenge Another round making the challenge more real worldmore real world

Include trend and development Include trend and development Give multiple random samplesGive multiple random samples