2007 cas predictive modeling seminar project management for predictive models beth fitzgerald, iso
TRANSCRIPT
![Page 1: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/1.jpg)
2007 CAS PREDICTIVE MODELING SEMINAR
PROJECT MANAGEMENT FOR PREDICTIVE MODELS
BETH FITZGERALD, ISO
![Page 2: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/2.jpg)
2
Accomplishing Business Goals
• Project Management
• Implementation
• Future
![Page 3: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/3.jpg)
3
Project Management
• Determine business processes that support strategic goals
– Underwriting decisions– Pricing decisions
• Develop project plan aligned with strategic goals
– Model Building– Technology Development– Implementation Phases
• Determine project needs
• Monitor actual vs. planned costs/milestones
![Page 4: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/4.jpg)
4
Project Needs
• Team Skills
– Data management – Analytical/statistical– Technology– Business Knowledge
• Data
• Statistical Tools
• Computer Capacity
![Page 5: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/5.jpg)
5
Prior to Modeling
•Formulate the Problem
•Evaluate Possible Data Sources
•Prepare the Data
•Explore the Data with Simple Modeling Techniques
![Page 6: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/6.jpg)
6
25% 50% 75% 85%
What percent of a model building project is the data preparation and data
management?
![Page 7: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/7.jpg)
7
Prepare the Data
• Do quality checks in level of detail needed for project
• Understand how to prepare individual variables for use in models
• Need to be practical about number of classification categories models can handle
• Need to decide on truncation and bucketing of variables that are continuous
• Create new variables
![Page 8: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/8.jpg)
8
Data Management Issues
• Matching additional internal policy information to premium/loss data
– Different points in time– Tracking & balancing audited exposures
• Different summarization keys – handling of mid-term endorsements
• Address scrubbing
• Matching to external data for correct point in time
• Significance of missing values within variable
![Page 9: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/9.jpg)
9
Modeling Procedures and Diagnostics
•Basic modeling training – GLM, Data Mining
•Decide on appropriate diagnostics
•Evaluate diagnostics
![Page 10: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/10.jpg)
10
Modeling Process
BusinessKnowledge
Data Linking
Data Cleansing
Analyze Variables
Determine Predictive Variables
Evaluation
Data Gathering
Modeling
![Page 11: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/11.jpg)
11
Business Questions
•What goals are you trying to achieve?
•What results do you expect to see?
•How will you know if the results are reasonable?
•How do you ensure sufficient knowledge transfer to business staff?
![Page 12: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/12.jpg)
12
Model Performance
Relative Loss Ratio LiftOptimal Model
0.7
0.8
0.9
1
1.1
1.2
1.3
1 2 3 4 5 6 7 8 9 10
Decile of Worst to Best Risk
Loss
Rat
io R
elat
ivity
LR Relativity by Decile
![Page 13: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/13.jpg)
13
Model Input/Output
• Model input considerations
– Access to data
– Robustness/quality of data
– Timeliness of refreshed data
• Design Model output for users
– Definition of output – expected loss ratio, pure
premium, loss ratio relativity?
– Provide support for output – reason codes
![Page 14: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/14.jpg)
14
Business Implementation of Model
• Model usage determined by strategic goals
– Underwriting risk decision
– Pricing of risks
– Support of market growth
• Integration of Model into business workflow
decisions
– Consistency in underwriting/pricing decisions
– Compliance with regulations based on
implementation decisions
![Page 15: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/15.jpg)
15
Implementation of Model
Workflows:
• Underwriting– New Business– Renewal business
• Rating– Pricing– Coverage Adjustment
![Page 16: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/16.jpg)
16
Implementation of Model
• New Business decision options
– Write risk– Request additional info on risk– Decline risk– Adjust price/coverage
• Consider model output alone or along with other information available from application
• Model output needed within seconds for quick decision
![Page 17: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/17.jpg)
17
Implementation of Model
• Renewal decision options
– Automatic renewal– Flag for non-renewal– Adjust coverage level for risk– Adjust pricing for risk
• Initial Year
– review all in-force policies on weekly or monthly basis
• Subsequent years
– establish schedule for reevaluation based on specific underwriting guidelines
![Page 18: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/18.jpg)
18
Implementation of Model
Rating
•Model O/P represents relative loss ratio factor
•Determine rating selections
•Determine rating process– Modify application of IRPM plan– Implement new rating factors based on
Model– Tier risks into different insurers within
insurer group
![Page 19: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/19.jpg)
19
Technology Development
• Incorporate business implementation decisions
• Decide on how Model will be accessible
electronically
– Web-based interface
– Integrated into existing workflow
– Batch processing
• Develop/Modify Systems
– Phase-in technology
• Model uses information from a third-party vendor
• Determine I/P and O/P criteria
![Page 20: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/20.jpg)
20
Rollout Implementation of Model
•Prepare Announcement/Training
Material for Internal & External
Customers
•Coordinate Implementation Phases
•Monitor Feedback/Adjust
Implementation
•Monitor Results against Strategic
Goals
![Page 21: 2007 CAS PREDICTIVE MODELING SEMINAR PROJECT MANAGEMENT FOR PREDICTIVE MODELS BETH FITZGERALD, ISO](https://reader035.vdocuments.us/reader035/viewer/2022062517/56649ee85503460f94bf9f07/html5/thumbnails/21.jpg)
21
Future of Predictive Modeling
• More refined rating plans– Industry-sourced or internally developed
– Combination of internally-developed & industry-sourced risk component variables
• Ongoing updating and maintenance of Models – Refresh data
– New data sources/variables
– New tools/techniques
– React to new market environments