appsphere 15 - mining the world’s largest healthcare data warehouse while ensuring great consumer...
TRANSCRIPT
Mining the world’s largest healthcare data warehouse while ensuring great consumer experience Siva Balu, Managing Director, Blue Cross Blue Shield Association
BlueCross BlueShield • Federation of 36 separate United States health insurance organizations and
companies
• Providing health insurance to 105+ million Americans
• Trusted for 80+ years
• Presence in all 50 states
• Accepted by over 90% of doctors and specialists
Copyright © 2015 AppDynamics. All rights reserved. 5
Data as A Strategic Asset • Board aligned that the Blue System should have a broader, aggregated data
asset as a foundation to drive value and enable future competitiveness
• The System has many of the key data assets to enable immediate Plan business needs and Plans prioritized potential uses of collected data
• Focus on Consumer
Copyright © 2015 AppDynamics. All rights reserved. 9
What is Web Service Platform?
• Web-Services Platform is an enabler to provide Access to the said Data in a real-time, secure, responsive and reliable manner
• Different software systems often need to exchange Data with each other, and a Web-Service is a method of communication that allows two disparate software systems to exchange this Data over the network
Copyright © 2015 AppDynamics. All rights reserved. 11
Why is there a need for Web Services?
• Once data is collected, cleansed and organized…
• … a reliable and secure method is necessary to access this data
• … and mask all the data storage and retrieval details
• … and provide fast, real-time access to meaningful and actionable information
• A Web-Service (using Application Programming Interface – API) is an industry standard to accomplish this
Copyright © 2015 AppDynamics. All rights reserved. 12
What types of Data is available via Web Services? • Consumer Cost • Patient Review of Providers • Provider Data • Provider Faceted Search • Member out of Pocket • Eligibility • Claims Status • Subscriber information • And more …
Copyright © 2015 AppDynamics. All rights reserved. 13
Web Services Architecture
Copyright © 2015 AppDynamics. All rights reserved. 15
Web Service Request
Private Internet
Load Balancer (External)
Gateway
Web Server
AP
I Key
M
anag
emen
t
Database Layer
Load Balancer (Internal)
Applica<on Layer
Web Service Response End User
Web Services Technology Stack
Copyright © 2015 AppDynamics. All rights reserved. 16
Web Service Request
BluesNet Internet
Load Balancer (F5)
Intel Enterprise Services Gateway (ESG)
HTTPS Layer -‐ IBM HTTP Server (IHS)
Mas
hery
AP
I
Database Layer (Oracle & SOLR)
Load Balancer (F5)
Applica<on Layer (WebSphere)
Web Service Response End User
How do users get what they want?
17
Internet
Loca l BCBS
Internet / BluesNet
Core Da ta Warehouse
BCBSA
A Brief History of Web Service Volume…
~50k ~900k ~7.5M ~18M
~80M
~250M
0 50000000
100000000 150000000 200000000 250000000 300000000
2010 2011 2012 2013 2014 2015
Volume
Copyright © 2015 AppDynamics. All rights reserved. 18
So, what’s the problem?
Capacity: Unexpected exponential volume increase Performance: System Performance issues Response Time: Need for sub-second response for 100% of requests Increased Research Time: Excess time taken to research root cause Customer Satisfaction: User experience impacts Copyright © 2015 AppDynamics. All rights reserved. 19
Where does come in?
Copyright © 2015 AppDynamics. All rights reserved. 20
Existing monitoring tools had limitations Need for a APM tool that will quickly point us to the problem Fairly new to using AppDynamics Still in the infancy stages of adoption Started with web services, expanding to Tier-1 Business Applications Working on a long-term strategy to integrate AppDynamics into SDLC and Operations
Web Services - AppDynamics
Copyright © 2015 AppDynamics. All rights reserved. 21
Web Service Request
BluesNet Internet
Load Balancer (F5)
Intel Enterprise Services Gateway (ESG)
HTTPS Layer -‐ IBM HTTP Server (IHS)
Mas
hery
AP
I
Database Layer (Oracle & SOLR)
Load Balancer (F5)
Applica<on Layer (WebSphere)
Web Service Response End User
What we learned… couple of examples:
Application: Code inefficiencies
Example: Zip code search Database: Query
Example: Queries mining large data volumes for small inquiries
Copyright © 2015 AppDynamics. All rights reserved. 22
What we did…
Added Intelligence into the code Changed the way code returned results to users Significant query tuning Expanded the environment to highly clustered Separated web-service calls into smaller service methods Moved from Core Warehouse to DataMart model Copyright © 2015 AppDynamics. All rights reserved. 23
Business Application - Rapidly growing volume
Copyright © 2015 AppDynamics. All rights reserved. 29
0
500,000
1,000,000
1,500,000
2,000,000
2,500,000
3,000,000
3,500,000
4,000,000 N
ov-0
9
Jan-
10
Mar
-10
May
-10
Jul-1
0
Sep
-10
Nov
-10
Jan-
11
Mar
-11
May
-11
Jul-1
1
Sep
-11
Nov
-11
Jan-
12
Mar
-12
May
-12
Jul-1
2
Sep
-12
Nov
-12
Jan-
13
Mar
-13
May
-13
Jul-1
3
Sep
-13
Nov
-13
Jan-
14
Mar
-14
May
-14
Jul-1
4
Sep
-14
Nov
-14
Jan-
15
Mar
-15
May
-15
Jul-1
5
Sep
-15
6000% claim volume growth
Tier-1 Business Application - Highly clustered
Copyright © 2015 AppDynamics. All rights reserved. 30
What we did so far…
Copyright © 2015 AppDynamics. All rights reserved. 31
Dec Nov Oct Sep Aug Jul Jun May Apr Mar Feb Jan
2015
Proof of Concept
Performance Engineering
Production Implementation
Controller Upgrade
Production Implementation
Beyond Web Services
Additional Apps
Proof of Concept (POC)
New Web Services
New Web Services
Windows to Linux
All Web Services
Applications
More Apps
Takeaways… Define Scope: Identify what you want to monitor and how to use the product POC: Do a Proof-of-Concept Decide the implementation model: On-premise vs. Cloud Strategy: Define and Execute a long-term Strategy for APM People: Educate your team, peers and customers Process: Devise processes for feedback to Dev, provisioning mechanism, etc. Technology: Look at other AppDynamics offerings and extensions Copyright © 2015 AppDynamics. All rights reserved. 32