369 diane bartoli
TRANSCRIPT
Analytics to Drive Effective Product Development
Diane Bartoli
Vice President, Product Development
SSP Boston, June 1, 2011
About Me
Vice President of Product Management
Elsevier Health Sciences
Clinical Reference Group
Analytics EXPERT
NOT*
*(Just a product manager desperate for data to inform good product development...)
2
With seemingly endless data, is the world not your...?
3
A few of the tools at your disposal...
4
WHICH TO USE???
The trick is not capturing the data, it’s asking the right questions...
Case Study: Elsevier Evergreen
A new Online Medical Reference product in development intended for Doctors in
the Hospital setting
6
Where is the need? Understanding the user
• Ethnographic research brings end-users to life• Highlights pain points• Understanding motivations and ambitions
What is needed and WHEN? Mapping workflow…
1Diagnosis
2
Create Care Plan
Medical treatment Surgical treatment
Procedures
After treatment care plan
Patient Education & Compliance
Sharing information
•Patient specific information•Case presentation (within dept)•Presentation to other bodies (e.g. Treatment committees, drug committees etc)•External presentations (Grand rounds, association presentations etc).
2a 2b
3
4
Non-surgical Surgical
Keeping current
•Conferences / society meetings•Reading current literature
Treatment
5
•Patient care management workflows take over 45% of the average physician's time
Patient care management workflows Collateral workflows
•Collateral workflows take 24% of the average physician’s time
Source: Qualitative research (76 in person respondents)
99
Key Findings
Quantifying qualitative findings....
Daily activity
Weekly activity
Weekly/monthly
Monthly activity
Annual activity
% of time medical reference products are requested
To
tal
tim
e s
pen
t (h
rs)
• Validation of critical information needs for physicians: trustworthy, current content
• Validation of critical pain points: lack of comprehensive access to information, clinical search relevance not optimal today
• Quantification of physician workflow: Understanding what they do, how often they do it, how important those activities are, when they need information and what information they need
• Business justification: Sizing market opportunity and potential growth
No current offering delivers on all three components doctors need in a medical information product
Zeroing in on the goal…
Speed to AnswerSpeed to Answer
TrustedTrustedComprehensiveComprehensive
Wikis
Aggregators
Clinical Reviews
Online Journals
ElsevierEvergreen
Online Books
The Physician’s Dilemma
• Residents report significant use of Google (64%), Wikipedia (27%) and WebMD (36%) given need for fast and easy reference
• Regardless of where they start their online search, 40-53% of doctors use more than one source
• Despite generally preferring E over P, physicians find PRINT resources more trustworthy (75% vs 59%) than current E offerings
Building products with analytics at the core: Alpha, Beta....Zeta
11
Volunteered comments :
Capturing the voice of the user
Measuring the Right things: Composite metrics intended to get to “workflow”
Quantifying results: Quantitative
Survey at close of Alpha Period
Elsevier Evergreen
Understanding Motivation: Why are you here? Are
you satisfied?
Lots of Data.... What to do with it?
System Data:• What are they looking for? Search logs help tune
search relevance and recall
Usage Data:• What was used? What was not used?• Are users coming back? • Are we inspiring loyalty and engaging our users?
Satisfaction:• Why are they coming to our site?• Did they really find what they were looking for?• Touchy-feely: visual design? Cultural differences?
Quantifying results:• Prioritization of development• Directing go-to-market planning• Return on investment?
System Data:• What are they looking for? Search logs help tune
search relevance and recall
Usage Data:• What was used? What was not used?• Are users coming back? • Are we inspiring loyalty and engaging our users?
Satisfaction:• Why are they coming to our site?• Did they really find what they were looking for?• Touchy-feely: visual design? Cultural differences?
Quantifying results:• Prioritization of development• Directing go-to-market planning• Return on investment?
Some lessons
• Setting appropriate expectations• Imperfect data yields imperfect results
• Apples, oranges, and some pears too: matching mis-matched data
• The Holy Grail: marrying customer demographics to usage and conversion
13
More questions?
Diane Bartoli,
VP, Product Development
215-239-3602
14