2008 cas spring meeting project management for predictive models john baldan, iso
TRANSCRIPT
![Page 1: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/1.jpg)
2008 CAS SPRING MEETING
PROJECT MANAGEMENT FOR PREDICTIVE MODELS
JOHN BALDAN, ISO
![Page 2: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/2.jpg)
2
Where Does Modeling Fit In?
ISO
ISO Innovative Analytics (IIA)
Insurance Lines of Business
Modeling Division
![Page 3: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/3.jpg)
3
Modeling Division – Outward-Facing Goals
•Work with IIA to develop predictive models for the major lines of insurance, using insurer data.
– Personal Auto
– Homeowners
– Commercial Lines
![Page 4: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/4.jpg)
4
Modeling Division – Inward Facing Goals
•Make greater use of predictive modeling techniques within areas of traditional ISO ratemaking:
– Loss costs
– Classifications
•Disseminate modeling knowledge and expertise to pricing actuaries via training, modeling projects.
![Page 5: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/5.jpg)
5
Developing Resources
• Staff Knowledge– Good sources re: GLMs:
• “A Practitioner’s Guide to Generalized Linear Models” – Anderson, Feldblum, et. al.
• “Generalized Linear Models” – McCullagh and Nelder
• Generalized Linear Models for Insurance Data – de Jong and Heller
• “A Systematic Relationship Between Minimum Bias and Generalized Linear Models” – Mildenhall
![Page 6: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/6.jpg)
6
Developing Resources
• Software– PC SAS
• Interactive aspect: advantage AND disadvantage• Graphics (no more +-----)• Avoid divisional chargebacks!
– R (used for MARS, for example)• Incredibly flexible, since object oriented• Special purpose modules available on Web • Widely adopted in statistical, academic world• Free!
– Other software packages, as needed
![Page 7: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/7.jpg)
7
Avoid Scope Creep!
![Page 8: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/8.jpg)
8
Avoid Scope Creep!
•Firm Project Management
•Consolidated development platform
![Page 9: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/9.jpg)
9
Building Predictive Models
•Know your data
• Get data in common format, at level of individual risk.
• Interface with insurer to:– Understand their database structure
– Examine univariate distributions to clarify the meanings of data elements, unclear codes and missing values.
![Page 10: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/10.jpg)
10
Predictive Models
•Know your model input– Not all potential model variables will be
available at the time of deployment. Determine which ones will be. Considerations:
• Simplicity of input
• Rating variables specified as offsets
• Lookup time
• Nature of model (marketing models)
![Page 11: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/11.jpg)
11
Predictive Models
•Know your model output– What are you producing?
– Frequency vs. severity vs. pure premium vs. loss ratio
– Do you want a relativity to a specified base risk?
![Page 12: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/12.jpg)
12
Measuring Model Effectiveness
• Measures of Lift– Decile plot– Gini index
What these share is an ordering of risks, against which experience is evaluated.
• Lift is measured against the rating system currently in place. Define sort order by:
– Modeled loss cost to current loss cost; or– Modeled loss cost to average modeled loss cost
for risks currently rated identically (for example, in a location model, risks in same territory)
![Page 13: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/13.jpg)
13
Measuring Lift – Decile Plot
![Page 14: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/14.jpg)
14
Measuring Lift – Gini Index
![Page 15: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/15.jpg)
15
Model Diagnostics
BI Model Diagnostics : Partial Residuals
NewVars Frequency Component
Pa
rtia
l L
og
it R
esid
ua
l
0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
![Page 16: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/16.jpg)
16
Models and Regulation
• Establish dialogue between modelers and legal/regulatory experts
• Modeling ground rules– Restricted modeling variables
– Model variable creation techniques
– Interpretability of final model form
• Model Smoothing
• Reason codes
• Diagnostics
![Page 17: 2008 CAS SPRING MEETING PROJECT MANAGEMENT FOR PREDICTIVE MODELS JOHN BALDAN, ISO](https://reader036.vdocuments.us/reader036/viewer/2022062322/5697bf741a28abf838c7f6eb/html5/thumbnails/17.jpg)
17
Updating the Model
•Know your product life cycle
• Input updates– Insurance data
– Third-party data
•Output/Model updates– Recalculation
– Re-estimation
– Rebuilding