big data in actuarial space how big is enough?acsw.us/fall17/yu.pdf · agenda i. what is big data?...
TRANSCRIPT
![Page 1: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/1.jpg)
Big Data in Actuarial Space –How Big is Enough?
By Haofeng Yu
Actuaries’ Club of the Southwest
Fall Meeting 2017
![Page 2: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/2.jpg)
Agenda
I. What is Big Data?
II. Insurers’ perspective
III. Big data less hype for actuaries
IV. Process of Big Data or Predictive Analytics
V. Applications with a mini lab on Jupyter
2
![Page 3: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/3.jpg)
Prelude…
![Page 4: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/4.jpg)
"Toto, I've a feeling we're not in Kansas anymore.”
Dorothy, The Wizard of Oz
4
I Our Actuarial SpacePast paradigm shifts:
• Refined UW with Preferred classes• Rise of Term Life• Rise of Variable Products• Rise of DC plan• Obama Care• ERISA/RBC(C3Px)/ERM/PBA• Living To 100• Business Intelligence• Big Data
![Page 5: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/5.jpg)
Best advice going to SOA Annual Meetings
5
I
“Go early, sit close.”
SOA AME
Opening General Session Speakers
KeywordsPresidential
Lunch SpeakersKeywords
2013 Dan RoamThe Back of the Napkin: Solving Problems and Selling Ideas with Pictures
Zanny Minton-Beddoes
economic editor for The Economist
2014 Adam Steltzner
Lead Landing Engineer of NASA's Mars Science Laboratory Curiosity Rover Project
Madeleine Albright
Former U.S. Secretary of State
2015 Salim Ismailtechnological developments and its implications on society.
Shawn AchorHappiness, Success, Positive Psychology
2016 Sal KhanFounder Khan Academy, Education innovator
Nick BiltonTechnology Journalist and Author, hardware hacker
2017 Kenneth Cukier
coauthor of Big Data: A Revolution That Transforms How We Live, Work, and Think
Scott PageProfessor, teaches popular course, Model Thinking, on Coursera
![Page 6: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/6.jpg)
Is this a hype?
6
I
Is this a hype?SOA AME
Opening General Session Speakers
KeywordsPresidential
Lunch SpeakersKeywords
2013 Dan RoamThe Back of the Napkin: Solving Problems and Selling Ideas with Pictures
Zanny Minton-Beddoes
economic editor for The Economist
2014 Adam Steltzner
Lead Landing Engineer of NASA's Mars Science Laboratory Curiosity Rover Project
Madeleine Albright
Former U.S. Secretary of State
2015 Salim Ismailtechnologicaldevelopments and its implications on society.
Shawn AchorHappiness, Success, Positive Psychology
2016 Sal KhanFounder Khan Academy, Education innovator
Nick BiltonTechnology Journalist and Author, hardware hacker
2017 Kenneth Cukier
coauthor of Big Data: A Revolution That Transforms How We Live, Work, and Think
Scott PageProfessor, teaches popular course, Model Thinking, on Coursera
![Page 7: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/7.jpg)
I. What is Big Data?
II. Insurers’ perspective
III. Big data less hype for actuaries
IV. Process of Big Data or Predictive Analytics
V. Applications with a mini lab on Jupyter
7
![Page 8: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/8.jpg)
What is an Elephant or Big Data?
8
I
The parable of blind men feeling elephant tells us:• you could be locally right,• yet globally wrong,• depending on your perspectives.
Big Data often associates with:data warehouse, data lake, date ocean, cloud
computing, Cognitive computing deep learning, supervised learning, unsupervised learning, Hadoop, tensor flow, IoT, wearables, and/or crowd-sourcing, statically learning, machine learning, AI, Blockchain,
FinTech, InsurTech, business intelligence, etc. predictive analytics
We are drowning in information and starving for knowledge — Rutherford D. Roger, 1985
![Page 9: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/9.jpg)
I. What is Big Data?
II. Insurers’ perspective
III. Big data less hype for actuaries
IV. Process of Big Data or Predictive Analytics
V. Applications with a mini lab on Jupyter
9
![Page 10: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/10.jpg)
Insurers’ perspective
10
II
Big data
![Page 11: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/11.jpg)
FinTech/InsurTech disruptors
FinTechNow describes a broad variety of technological interventions into personal and commercial finance
Keywords:
Blockchain, Crowd-sourcing, peer-to-peer
Disruptors:
LendingClub, WealthSimple, Finn.ai, OutsideIQ
InsurTechRefers to the use of technology innovations designed to squeeze out savings and efficiency from the current insurance industry model
Keywords:
on-demand, peer-to-peer, wearables, micro events
virtual distribution
Disruptors:
Ladder, Haven, Besure, Tuque, Energlm Sonnet, Trov, Zensurance, Lemonade
FinTech/InsurTec definitions by Investopedia 11
II-c
Enabling solutionsBig data techniques, cognitive computing (e.g., AI), cyber/mobile technology, etc.
![Page 12: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/12.jpg)
Insurers’ perspective
12
II-e
![Page 13: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/13.jpg)
I. What is Big Data?
II. Insurers’ perspective
III. Big data less hype for actuaries
IV. Process of Big Data or Predictive Analytics
V. Applications with a mini lab on Jupyter
13
![Page 14: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/14.jpg)
Big data less hype for actuaries
14
III
a. Data
b. Techniques
c. Big data for insurance value chain
![Page 15: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/15.jpg)
DataSet, group, base, warehouse, lake, ocean, cloud, and universe
Happy Thanksgiving!
III-a
![Page 16: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/16.jpg)
All data?
Google Flu Trend
16
III-a
Events and choices in a life cycle
Graph source: Agent Based Modeling and Its Actuarial Applications, Lombardi and Rao, PWC SOA Webcast, 2015.
![Page 17: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/17.jpg)
Zero data?
AlphaGo Lee was trained using 30 millions of human moves; AlphaGo Zero none!
Go is a game with perfect information in a closed system,
but Insurance industry is an open system, not well defined, and full of Imperfect information and behavior
The Game of the 20th Century: Go Seigen (Black) vs. Honinbo (White), 1933 17
III-a
![Page 18: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/18.jpg)
Data - the art of the possible
Source: McKinsey & Company, presented in Transforming Life Insurance with Design Thinking, by way of RGA
III-c
Apply big data, incrementally, while be aware of regulatory/legal boundaries
![Page 19: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/19.jpg)
Techniques
Graph by RGA, Peter Banthrope, SOA PA Symposium 2017 19
III-b
Supervised learning Regression
Classification
Unsupervised learning Clustering
Others: Dimension Reduction
Genetic Algorithm
Reinforcement learning
Deep Learning
Be practical! Start with
![Page 20: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/20.jpg)
Big data for Insurance value chain
20
III-d
![Page 21: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/21.jpg)
I. What is Big Data?
II. Insurers’ perspective
III. Big data less hype for actuaries
IV. Process of Big Data or Predictive Analytics
V. Applications with a mini lab on Jupyter
21
![Page 22: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/22.jpg)
Process of Predictive Analytics
22
IV
1 Define the Problem"Far better an approximate answer to the right question ... than the exact answer to the wrong question..." - John Tukey
2 Data Collection What could be used? What should be used?
3 Data Preparation Data structure and scrub; variable transformation and grouping
4 Build Model Select, fit and diagnose, etc.
5 Model Validation Parameterize model on the test dataset, k-fold cross-validations
6 Model Evaluation out-of-time test, Gini
7 ImplementationEasy to solve a problem, hard to prove a theory?Easy to solve theoretically, hard to operationalize a solution!
Ewald and Wang, Predictive Modeling: A Modeler’s Introspection, 2015, SOA
![Page 23: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/23.jpg)
I. What is Big Data?
II. Insurers’ perspective
III. Big data less hype for actuaries
IV. Process of Big Data or Predictive Analytics
V. Applications with a mini lab on Jupyter
23
![Page 24: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/24.jpg)
Set up a mini PA lab
24
V
a. Excel
b. SQL type of Environment
c. R + RStudio
d. Python + Anaconda
Welcome aboard! Let us go to Jupyter!
![Page 25: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/25.jpg)
25
V Examples on JupyterWith potential applications for Actuaries
Platform Examples Applications for Actuaries
Supervised learning
R Regression GLM for Assumptions setting
R Classification Logit Regression for UW
Unsupervised learning
R Clustering Risk segmentation
Python Agent based model Policyholder behavior simulation
Others
Reinforced learning UW
Deep learning Distribution model
Dimension Reduction
Model Ensemble
…
R, Python,
GPU,
Hadoop,
cloud,
TensorFlow,
etc.
Jup
yte
r
![Page 26: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/26.jpg)
Final Thoughts
26
a. Embrace big data, incrementally
b. Take action - set up your Big Data lab and explore
c. Merely an actuary? Be a PAFer? https://www.soa.org/predictive-analytics-and-futurism
Be a Kaggler? https://www.kaggle.com/50119-Kaggler
![Page 27: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/27.jpg)
Additional References
1. Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani, An Introduction to Statistical Learning with Applications in R, 2013 http://www-bcf.usc.edu/~gareth/ISL/book.html,
2. Peter Harrington, Machine Learning in Action, 2012 (using Python)
3. Alan Mills, “Complexity Science - An introduction (and invitation) for actuaries”, SOA Special Report, 2010
4. Piet de Jong and Gillian Z. Heller, Generalized Linear Models for Insurance Data, 2008
27
![Page 28: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/28.jpg)
Questions?
Limitations - The views expressed in this presentation are those of the presenter, and not those of AIG. Nothing in this presentation is intended to represent a professional opinion.
![Page 29: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/29.jpg)
29
Appendix - Agent Based Modeling
Traditional actuarial models largely project historical aggregate patterns and “actuarial judgment” into the future - top-down.
ABM simulates agents’ (e.g., policyholders, agents and brokers) interactions with their environment and other agents in order to understand the emergent behavior of complex systems – bottom-up
![Page 30: Big Data in Actuarial Space How Big is Enough?acsw.us/fall17/yu.pdf · Agenda I. What is Big Data? II. Insurers’ perspective III. Big data less hype for actuaries IV. Process of](https://reader035.vdocuments.us/reader035/viewer/2022070805/5f03d9df7e708231d40b1360/html5/thumbnails/30.jpg)
ABM helps understand
• Strategy & Growth
• Marketing & Sale
• Product Development & UW
• Process & Operation
• Inforce Management
• Capital, Risk & Finance
30Graph source: “Agent Based Modeling and Its Actuarial Applications”, Lombardi and Rao, PWC, SOA Webcast, 2015.
Appendix - Agent Based Behavioral Simulation