accelerating the value of big data analytics for p&c insurers with hortonworks and clarity...

© Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC

The Modern Data Architecture for the Insurance Industry


Today’s Speakers

Tripp Smith, CTO Clarity Solution Group

Cindy Maike, GM-Insurance Hortonworks


The Insurance Industry Data Equation …the current situation


Trends require a shift from product to prevention

HDP and Hadoop allow insurers to shift interactions from…

Reactive Post-Transaction

Proactive Pre-Decision

…to prevention services customized for needs From traditional coverage

…to proactive advisors From siloed information

…to 1x1 targeting & engagement From “mass-market”

A shift in Customer Engagement

A shift in Products

A shift in Agent/Broker and Call Center Support

…to ‘valid and pay’, anomaly detection and severity From “a claim is a claim”

A shift in Claims Management


Consensus of Analysts estimate enterprise data growth of 50x year over year through 2020

Data is growing exponentially at unprecedented rates

0 5 10 15 20 25 30 35 40

2020

2018

2015

2013

The “Digital Universe” expressed in Zettabytes* 85% of growth from new types of data with machine-generated data increasing 15x *Multiples of Bytes

Kilobyte Megabyte Gigabyte Terabyte Petabyte Exabyte Zettabyte Yottabyte


Clickstream Capture and analyze website visitors’ data trails and optimize your website

Sensors Discover patterns in data streaming automatically from remote sensors and machines

Server Logs Research logs to diagnose process failures and prevent security breaches

New types of data

Sentiment Understand how your customers feel about your brand and products – right now

Geographic Analyze location-based data to manage operations where they occur

Unstructured Understand patterns in files across millions of web pages, emails, and documents

Data is growing exponentially – causing IT delays


Enabling a Mature Enterprise Strength Hadoop

Opportunities with Using Hadoop

Information Agility Increased Opportunity

•  Single point of access to high-priority enterprise assets, transactional assets and Dark Data

•  Centralization of data combined with decentralization of analytic capabilities

Processing Horsepower Increased Capacity

•  Near-linear hardware capacity scalability •  Portfolio of components that scale to data or

computational complexity

New or Expanded Analytics Expanded Capability

•  Increased depth of conventional analysis •  Application of analytics to real-time needs •  Deep machine learning and discovery analytics

Cost Containment Reduced Expense

•  Cheaper than enterprise SAN or proprietary RDBMS •  Scalable with inexpensive hardware vs. expensive

optimization or recoding

"Out of the Box" Challenges

Manageability – Risk – TCO Spiral

•  Platform security and repeatable processes for securing data

•  Common vocabulary and business data definitions

•  Consistently applied data integration and transformation processes

•  Transparent data quality and data lineage

•  Ability to manage complex mixed workloads and a variety of access patterns to support disparate user groups and use cases

•  Time which instead is spent on data forensics rather than analysis

Operational and Interactive Platforms


MESH achieves agility with integrity

The Mature Enterprise Strength Hadoop (MESH) framework addresses scaling a Hadoop ecosystem that meets enterprise needs

Matrix of architecture, governance and enablement capabilities

Integrated ecosystem addressing the full breadth of enterprise analytics, users and use cases

Enterprise-strength security and achievable data governance

Automation and acceleration across implementation, governance and enablement vectors

Operational roadmap and tool kit to activate business value through analytic agility

Managed Raw Materials

Structured Integration and

Discovery

Governed End-User

Consumption

Organic, Process-Driven

Information Refinement


3 Dimensions of Success: Architecture, Governance, and Enablement

Security and Role Based Access Controls

Acquisition and Ingestion

Archival Data Management

Event Processing

Data Transformation

Master Data Integration

Information Delivery

Discovery Analytics

Machine Learning

Common Vocabulary and Data Definitions

Tool Rationalization

Process Automation

Testing and Quality Assurance

Resource and Workload Management Processes

Data Quality and Stewardship


The components beyond the warehouse

Iterative workspace for deep analytics and data discovery

Enablement and alignment

Streamlined service-driven data integration

Incremental enrichment of analytic data service

Governance and operational clarity


Big Data & Hadoop market drivers and opportunities

Business Drivers

•  From reactive analytics to proactive customer interaction

•  Insights that drive competitive advantage and optimal returns

Financial Drivers

•  Cost of data systems, as % of IT spend, continues to grow

•  Cost advantages of commodity hardware and open source software

$

Technical Drivers

•  Data is growing exponentially and existing systems are overwhelmed

•  Predominantly driven by NEW types of data that can inform analytics


The Modern Data Architecture …use case examples of Insurers using Hortonworks Data Platform


Customers using HDP to meet Insurance challenges

Top 3 Challenges Relevant Insurance Use Cases

Change in Customer Engagement Model

Rising Claims Costs (frequency and severity)

Data Explosion (Complexity of Risk/

Underwriting Information)

Personalization / Next Best Action

720º Degree Customer Visibility

Risk/Underwriting Profile Analysis

Sensor-based Telematics (Prevention Services, UBI)

ETL/EDW Optimization

Claim anomaly and fraud detection


Use Case Analysis


Enhanced Insurance cross-sell and catching fraudulent claims

Problem: ETL challenges with multiple data streams hampers analysis on ways to improve customer service

•  Traditional and newer types of data were difficult to combine in the EDW, because of

“schema on write” architecture some data was discarded

•  Company missed data-driven ways to serve customers better

•  Poor data visibility hampered analysis separating legitimate from fraudulent claims

Solution: Data lake to improve up-sell and identify fraud

•  “Schema on read” architecture ingests more data sources for predictive analytics

•  Agents use new insights to provide higher service levels to valued customers

•  Claims analysts and underwriters process streaming data to quickly flag fraud risks and

fast-track legitimate claims

Insurance – Health

Large US medical insurer

IH2

Why Hadoop?

Data Systems Optimization

Claim Anomalies & IT Optimization


Improved risk analysis and margins for usage-based car insurance

Problem: Slow ETL processing hampered speedy underwriting of usage-based car insurance

•  Usage-based car insurance requires rapid ingest and analysis of sensor data

•  Volume, velocity and variety of incoming data taxed existing systems and the high cost

of storage eroded margins

•  ETL process only captured 25% of the dataset and took 5-7 days to complete

Solution: Faster time-to-insight, improved ETL & predictive analytics

•  Built Azure POC cluster to justify the big data project before launching HDP on site

•  Improved performance and predictive analytics with Apache Hive

•  Faster ETL in Hadoop now processes 100% of the data in three days or less

Insurance – Property & Casualty

Personal auto & other property-casualty insurance

IP1

Why Hadoop?

Predictive Analytics

Telematics/UBI New Analytic Applications

Sensor Data and ETL Offload


Improved visibility and accuracy for P&C Insurance claim analysis

Problem: Analysis of unstructured data did not keep pace with mature systems for analyzing structured data

•  Large P&C insurance provider had systems for analyzing structured data at scale

•  Unstructured data from claims notes and social media data could add valuable

information to claims analysis, but is was unable to analyze this data at scale

•  Impartial data visibility hampered underwriting and claims, driving up costs, eroding

margins and blocking efforts to reduce fraudulent claims

Solution: Join structured and unstructured data for accuracy in claims processing, reducing risk, processing costs and fraud

•  “Schema on read” architecture captures more data sources (text and social data)

•  Larger data sets fed to front-end business tools provided by Hortonworks partners: SAS,

Tableau and QlikView

Insurance – Property & Casualty

Major provider of property casualty, life and mortgage insurance

IP2

Why Hadoop?

Data Discovery

Claims: New Analytic Applications Structured, Social & Unstructured Data


Case Study: 12-month Hadoop evolution at TrueCar

Dat

a Pl

atfo

rm C

apab

ilitie

s

12 months execution plan

June 2013 Begin Hadoop Execution

July 2013 Hortonworks Partnership

May ‘14 IPO

Aug 2013 Training & Dev Begins

Nov 2013 Production Cluster 60 Nodes 2 PB

Jan 2014 40% Dev Staff Perficient

Dec 2013 Three Production Apps (3 total)

Feb 2014 Three More Production Apps (6 total)

12 Month Results at TrueCAR •  Six Production Hadoop Applications •  Sixty nodes/2PB data •  Storage Costs/Compute Costs

from $19/GB to $0.23/GB

“We addressed our data platform capabilities strategically as a pre-cursor to IPO.”

Leverage commodity hardware for efficient data storage


Optimized marketing impact with advanced customer analytics

Problem: Difficulty understand marketing effectiveness and customer behavior

•  25M customers and prospects conducting over 10M weekly corporate interactions

•  Lack of visibility into effectiveness of marketing spend and impacts on consumer

behavior

•  Difficulty modelling behavior across disparate data sources from internal enterprise

master data and external vendors

Solution: Advanced analytics and experiment design

•  Closed-loop marketing analytics across key enterprise business units

•  Informed tactical decisions that ensure efficient marketing spend

•  Quantification of marketing effectiveness and audience behavior

•  Strategic insight for enterprise marketing efforts to evolve cross-sell to customers and

drive revenue generation

Insurance

Large US-based financial services and insurance company

IH1

Why Clarity?

Advanced Customer Engagement

Personalization / Next Best Action


Consumer 720 – Enterprise data with new data for personalized customer engagement

720º Degree Customer Visibility

Enterprise Master Data Evolution

•  Companies continue to evolve the single version of key Customer data for use throughout the enterprise

Social and Interactive Data Challenges

•  Additional data from Social Media sources provide additional consumer insight, but data integration challenges prevent this insight from turning into action

Limited Options for Enablement

•  There is no solution in the marketplace today that allows companies to seamlessly integrate core customer data needed to manage a business along with the vast amount of social data that these customers use to express affinity

Inside the Enterprise

Outside the Enterprise In-store

Activity

Service & Support

Data Enrichment

Online Purchases Enterprise Social

360 360

Consumer Engagement Roadblocks

•  Inability to accurately identify customers across the enterprise

•  Key marketing systems (campaign management, analytics, CRM, etc.) unable to leverage holistic customer data

•  Time and effort wasted with validating, integrating and managing consumer data across the enterprise

Co

nsum

er D

ata

Cha

lleng

es

Enterprise Customer Profiles

Internal Sources

3rd-parties

Consumer720

Enterprise Customer Attributes used for identity

Digital Interactions

Personalized Customer Experience


Insurance business value matrix using HDP

New

Typ

es

of A

naly

tics

New Types of Data

New Types of Data New Analytics Apps

•  Sentiment

•  Click-stream

•  Sensor

•  Geographic

•  Server Logs

•  Unstructured

Existing Data

Exis

ting

Ana

lytic

s

RDBMS

MPP

EDW

•  EDW & ETL data & load balancing

•  Cost & flexibility •  Building new skill sets •  Scale out using

commodity hardware

•  Single-View of Customer showing full 360-degree profile and history

•  Clickstream analysis for Next Best Action with Customers

•  Analyzing submission and claims models against larger historical data sets

HDP

HDP

New Historical View

IT Optimization New Data Influencers

•  Collecting Sensor/Telematics for Usage Based Insurance

•  Sentiment •  Enhanced Loss Control /

Prevention Services •  Needs based coverage vs.

traditional coverage

HDP

New Analytics Applications •  Text Analytics and Link

Analysis for Claim Anomaly and Fraud Analysis/Detection

•  Enhance Risk Analysis with Related Party Network Link Analysis

•  Enhanced Claim Severity and Frequency Models using “new” predictive data

HDP


Evaluation Biz Value

Awareness & Interest

Evaluation Technical

Enterprise Deployment

Enterprise Production

Point Deployment

Point Production

* Timeline varies by company size. Often smaller or focused online businesses achieve milestones at the shorter end of the range.

1 – 2 months 2-6 months

9-15 months 18-36 months

Start small and grow over time…

1 2 3 4 Potential Operational Strategic Data-Driven

Data Lake

Modern Data Architecture

Industry Leadership


Clarity Solution Group and Hortonworks Background and Focus …how we can help the Insurance Industry


Elegant solutions for insurance business needs

Clarity helps top insurers tackle:

•  Telematics & UBI for pricing and risk

management

•  360-Degree Customer Views to improve cross-selling

•  Multi-Channel Optimization to measure marketing effectiveness and improve customer experience

•  Distribution Channel Analysis to reduce costs, improve retention and drive profitability

•  Underwriting Optimization to reduce loss and improve pricing

•  Product Development to increase customer satisfaction and target new markets

The largest independent US services firm exclusively focused on data and analytics


Hadoop for the Enterprise: Implement a Modern Data Architecture with HDP

Customer Momentum

•  230+ customers (as of Q3 2014)

Hortonworks Data Platform •  Completely open multi-tenant platform for any app & any data. •  A centralized architecture of consistent enterprise services for

resource management, security, operations, and governance.

Partner for Customer Success •  Open source community leadership focus on enterprise needs •  Unrivaled world class support

•  Founded in 2011 •  Original 24 architects, developers,

operators of Hadoop from Yahoo! •  600+ Employees •  800+ Ecosystem Partners


Only HDP delivers a Centralized Architecture for the modern data needs HDP is uniquely built around YARN serving as a data operating system that provides multi-tenant Resource Management, consistent Governance & Security and efficient Operations services across Hadoop applications.

Hortonworks Data Platform

YARN Data Operating System •  A centralized architecture of

consistent enterprise services for resource management, security, operations, and governance.

•  The versatility to support multiple applications and diverse workloads from batch to interactive to real-time, open source and commercial.

Key Benefits

•  Multiple applications on a shared data set with consistent levels of service: a multitenant data platform.

•  Provides a shared platform to enable new analytic applications.

•  Delivers maximum cost efficiency for cluster resource management. Fewer servers fewer nodes.

Storage

YARN: Data Operating System

Governance Security

Operations

Resource Management

Existing Applications

New Analytics

Partner Applications

Data Access: Batch, Interactive & Real-time


Customer partnerships matter Driving our innovation through Apache Software Foundation Projects

Apache Project Committers PMC Members

Hadoop 27 21

Pig 5 5

Hive 18 6

Tez 16 15

HBase 6 4

Phoenix 4 4

Accumulo 2 2

Storm 3 2

Slider 11 11

Falcon 5 3

Flume 1 1

Sqoop 1 1

Ambari 34 27

Oozie 3 2

Zookeeper 2 1

Knox 13 3

Ranger 10 n/a

TOTAL 161 108 Source: Apache Software Foundation. As of 11/7/2014.

Hortonworkers are the architects and engineers that lead development of open source Apache Hadoop at the ASF

•  Expertise Uniquely capable to solve the most complex issues & ensure success with latest features

•  Connection Provide customers & partners direct input into the community roadmap

•  Partnership We partner with customers with subscription offering. Our success is predicated on yours.

27

Cloudera: 11

Facebook: 5

LinkedIn: 2

IBM: 2

Others: 23

Yahoo 10


Q&A: Please use the Q/A panel to ask your questions! More Information: Clarity Solution Group - clarity-us.com Hortonworks - hortonworks.com Industry email Updates A Modern Data Architecture Whitepaper The Rise of the Data First Enterprise

Speaker Contact Information: Tripp Smith, CTO Clarity Solution Group: [email protected] Cindy Maike, GM-Insurance Hortonworks: [email protected]


Our Missions: Clarity Solution Group: We help businesses decrease time to market and reduce costs by providing data and analytics solutions to discover hidden trends and find value in the data they already have.

Hortonworks: To enable Apache Hadoop to be the enterprise data platform that powers the modern data architecture and process half the worlds data

accelerating the value of big data analytics for p&c insurers with hortonworks and clarity...

Technology