analytics and xapi: measuring your way to success€¦ · analytics and xapi: measure your way to...
Post on 17-Jul-2020
10 Views
Preview:
TRANSCRIPT
Session 809Analytics and xAPI:
Measure Your Way to Success
Steve Foreman, InfoMedia Designs
Andy Johnson & Craig Wiggins, ADL
Las Vegas, NV | September 30 – October 2, 2015
Sponsored by the Office of the Under Secretary of Defense for Personnel and Readiness (OUSD
P&R)
Analytics and xAPI: Measuring Your Way To Success
Peter Berking, Andy Johnson, Craig Wiggins, Steve Foreman
Devlearn 20152 October 2015Las Vegas, NV
The authors
3
Steve ForemanPresident
InfoMedia Designs
Andy JohnsonResearch Analyst
ADL
Craig WigginsCommunity Manager
ADL
Peter BerkingSenior Instructional
DesignerADL
4
The article
Step 1: Analyze need
Step 2: Design the solution
Step 3: Design the xAPI model
Step 4: Develop and implement
Step 5: Measure, monitor, improve
Agenda
The situation
The situation
Step 1: Analyze need
Step 2: Design the solution
Step 3: Design the xAPI model
Step 4: Develop and implement
Step 5: Measure, monitor, improve
Agenda
9
1. Identify business problem
2. Identify goals indicating success in addressing
problem
3. Identify individual metrics to track with xAPI*:
• Refining understanding of problem
• Indicating progress towards addressing problem to:
• Stakeholders
• End users
4.Identify factors that contribute to business problem
1. Directly
2. Indirectly
Needs Analysis Process
*Remember - your metrics are unconstrained by the limitations of current standards!
10
Direct Success Contributors
Business
Problems
Business
Metric to
Track
Revenue below targets* Which product management
experts are sales people
contacting?
Number of contacts per product
manager
Lower numbers of prospects
and contacts in sales funnel*Which product management
experts are sales people getting
most value from?
Sales rep ratings of product
manager interactions
Higher percentage of
customers dropping after
prospect and contact stages*
Which sales managers are
conducting meetings?
Whether weekly meetings
occurred per sales manager
Time is allocated less
efficiently for most sales
teams compared to one high
performing sales team
Are sales reps attending the
meetings?
Number of sales reps in
attendance
*for 3-6 months after launch of updated product line
11
Direct Success Contributors
Business
Problems
Business
Metric to
Track
Lower numbers of prospects
and contacts in sales funnel*Which product management
experts are sales people getting
most value from?
Sales rep ratings of product
manager interactions
*for 3-6 months after launch of updated product line
12
Direct Success Contributors
Business Problems
Lower numbers of prospects and contacts in sales funnel
13
Direct Success Contributors
Business
Problems
Business
Metric to
Track
Lower numbers of
prospects and contacts in
sales funnel*
Which product management
experts are sales people getting
most value from?
Sales rep ratings of product
manager interactions
*for 3-6 months after launch of updated product line
14
Direct Success Contributors
Which product management experts are sales people getting
most value from?
15
Direct Success Contributors
Business
Problems
Business
Metric to
Track
Lower numbers of prospects
and contacts in sales funnel*Which product management
experts are sales people getting
most value from?
Sales rep ratings of product
manager interactions
*for 3-6 months after launch of updated product line
16
Direct Success Contributors
Sales rep ratings of product manager interactions
Business Metric to Track
17
Indirect Success Contributors
Learning &
Performance
Solution
Component
Question
to Answer
Activity to Track
Searchable Product
Knowledgebase
What types of information are
salespeople seeking and either
finding or not finding?
Sales rep searches where
results were:
• Selected and accessed
• Not selected and accessed
What types of information are
salespeople using?
Sales rep consumption of
information by:
• Product (i.e., each product in
the product line
• Type (i.e., market research,
competitor comparisons,
product specifications, sales
collateral
What types of information are
being populated?
Product manager and
marketing expert contribution
by:
• Product (see above)
• Type (see above)
18
Indirect Success Contributors
Learning &
Performance
Solution
Component
Question
to Answer
Activity to Track
Searchable Product
Knowledgebase
What types of information are
salespeople seeking and either
finding or not finding?
Sales rep searches where
results were:
• Selected and accessed
• Not selected and accessed
What types of information are
salespeople using?
Sales rep consumption of
information by:
• Product (i.e., each product in
the product line
• Type (i.e., market research,
competitor comparisons,
product specifications, sales
collateral
What types of information are
being populated?
Product manager and
marketing expert contribution
by:
• Product (see above)
• Type (see above)
19
Indirect Success Contributors
Learning &
Performance
Solution
Component
Question
to Answer
Activity to Track
Searchable Product
Knowledgebase
What types of information are
salespeople seeking and either
finding or not finding?
Sales rep searches where
results were:
• Selected and accessed
• Not selected and accessed
What types of information are
salespeople using?
Sales rep consumption of
information by:
• Product (i.e., each product in
the product line
• Type (i.e., market research,
competitor comparisons,
product specifications, sales
collateral
What types of information are
being populated?
Product manager and
marketing expert contribution
by:
• Product (see above)
• Type (see above)
20
Indirect Success Contributors
Learning & Performance Solution Component
Searchable Knowledge Base
21
Indirect Success Contributors
Learning &
Performance
Solution
Component
Question
to Answer
Activity to Track
Searchable Product
Knowledgebase
What types of information are
salespeople seeking and either
finding or not finding?
Sales rep searches where
results were:
• Selected and accessed
• Not selected and accessed
What types of information are
salespeople using?
Sales rep consumption of
information by:
• Product (i.e., each product in
the product line
• Type (i.e., market research,
competitor comparisons,
product specifications, sales
collateral
What types of information are
being populated?
Product manager and
marketing expert contribution
by:
• Product (see above)
• Type (see above)
Indirect Success Contributors
Question to Answer
What types of information are salespeople
seeking and either finding or not finding?
23
Indirect Success Contributors
Learning &
Performance
Solution
Component
Question
to Answer
Activity to Track
Searchable Product
Knowledgebase
What types of information are
salespeople seeking and either
finding or not finding?
Sales rep searches where
results were:
• Selected and accessed
• Not selected and accessed
What types of information are
salespeople using?
Sales rep consumption of
information by:
• Product (i.e., each product in
the product line
• Type (i.e., market research,
competitor comparisons,
product specifications, sales
collateral
What types of information are
being populated?
Product manager and
marketing expert contribution
by:
• Product (see above)
• Type (see above)
24
Indirect Success Contributors
Activity to Track
Sales rep searches where results were:
• Selected and accessed
• Not selected and accessed
Indirect Success Contributors
Online Access to
Product Management
Experts
Which product management
experts are sales people
contacting?
Number of contacts per product
manager
Which product management
experts are sales people getting
most value from?
Sales rep ratings of product
manager interactions
Weekly Sales Team
Meetings
Which sales managers are
conducting meetings?
Whether weekly meetings
occurred per sales manager
Are sales reps attending the
meetings?
Number of sales reps in
attendance
Learning &
Performance
Solution
Component
Question
to Answer
Activity to Track
Indirect Success Contributors
Online Access to
Product Management
Experts
Which product management
experts are sales people
contacting?
Number of contacts per product
manager
Which product management
experts are sales people getting
most value from?
Sales rep ratings of product
manager interactions
Weekly Sales Team
Meetings
Which sales managers are
conducting meetings?
Whether weekly meetings
occurred per sales manager
Are sales reps attending the
meetings?
Number of sales reps in
attendance
Learning &
Performance
Solution
Component
Question
to Answer
Activity to Track
Indirect Success Contributors
Online Access to
Product Management
Experts
Which product management
experts are sales people
contacting?
Number of contacts per product
manager
Which product management
experts are sales people getting
most value from?
Sales rep ratings of product
manager interactions
Weekly Sales Team
Meetings
Which sales managers are
conducting meetings?
Whether weekly meetings
occurred per sales manager
Are sales reps attending the
meetings?
Number of sales reps in
attendance
Learning &
Performance
Solution
Component
Question
to Answer
Activity to Track
Indirect Success Contributors
Online Access to Product Management Experts
Learning & Performance Solution Component
Indirect Success Contributors
Online Access to
Product Management
Experts
Which product management
experts are sales people
contacting?
Number of contacts per product
manager
Which product management
experts are sales people getting
most value from?
Sales rep ratings of product
manager interactions
Weekly Sales Team
Meetings
Which sales managers are
conducting meetings?
Whether weekly meetings
occurred per sales manager
Are sales reps attending the
meetings?
Number of sales reps in
attendance
Learning &
Performance
Solution
Component
Question
to Answer
Activity to Track
30
Indirect Success Contributors
Question to Answer
Which product management experts are sales
people contacting?
Indirect Success Contributors
Online Access to
Product Management
Experts
Which product management
experts are sales people
contacting?
Number of contacts per product
manager
Which product management
experts are sales people getting
most value from?
Sales rep ratings of product
manager interactions
Weekly Sales Team
Meetings
Which sales managers are
conducting meetings?
Whether weekly meetings
occurred per sales manager
Are sales reps attending the
meetings?
Number of sales reps in
attendance
Learning &
Performance
Solution
Component
Question
to Answer
Activity to Track
32
Indirect Success Contributors
Activity to Track
Number of contacts per product manager
Step 2: Design the solution
Step 3: Design the xAPI model
Step 4: Develop and implement
Step 5: Measure, monitor, improve
Step 1: Analyze need
Agenda
Learning & Performance Solution
Learning & Performance Solution
36
Knowledgebase
Sales reps use a searchable
knowledgebase to find
information from product
management and marketing.
● Product information
● Market research
● Competitive analysis
● Customer intelligence
37
Access to Experts
Sales reps can
connect with
product
management
experts.
38
Social Networking & Collaboration
Sales information sharing
meetings are held with
structure & guidance that
emulates high
performance sales team.
Step 2: Design the solution
Step 3: Design the xAPI model
Step 4: Develop and implement
Step 5: Measure, monitor, improve
Step 1: Analyze need
Agenda
xAPI Model Design Process
40
• Identify actions to track within current processes
• Carefully create verbs with one-to-one correlation
• Determine what is being measured (if anything) with each action
• Create “natural” Statements
• Identify characteristics/profile information
• Put it all together to determine which metrics come out
Identify Actions
41
Prospected
Contacted
Negotiated
Made an Offer
Closed/Sold
Dropped
Searched
Accessed
Published
Updated
Rated
Contacted
Asked
Answered
Rated
Held
Attended
Verbs One-to-One
42
Tying in Measurables
43
44
Constructing Statements
• <Actor> <Verb> <Object> <Result>
• “Person” “Action” “within Activity” with “Measured Data”
• Andy sold (advanced the state within) in “General Sales”
500 of Widget 505 for $20000.00 USD
• Andy searched KnowledgeBase for “Widget505”
• Andy contacted Craig (in Ask the Experts)
• Andy asked Craig (in Ask the Experts) “Can you tell me the
top selling point of Widget505?”
• Andy attended “September 2015 Team Alpha Sales
Meeting”
Background and Other Info
45
Meet Bob.
Salesman for 30 years(with Integrated Widgets for 25 years )
Certified, qualified and educated
Balanced task/time allocation
46
Putting It All Together
Step 2: Design the solution
Step 3: Design the xAPI model
Step 4: Develop and implement
Step 5: Measure, monitor, improve
Step 1: Analyze need
Agenda
48
xAPI Development Process
• Establish continuous dialog between Designer and Developer to establish what is tracked, for which roles, and in what context
• Look carefully across communities of practice for established best practices
• Translate the natural language Statements into code
• Create visualizations, probably using tools
49
• Ask questions like “Does it make sense for this type of person to do this activity?”
• Or, “What did you want to see tracked out of this, a 1-5 rating or a like/dislike?”
• Easy to use when sides speak the same language but understand their different roles
Designer and Developer
50
Take Verbs from CoPs
51
Translate into Code
52
Code - Part 2
53
Visualization Tools
Placeholder for xAPI Dashboard image
Step 2: Design the solution
Step 3: Design the xAPI model
Step 4: Develop and implement
Step 5: Measure, monitor, improve
Step 1: Analyze need
Agenda
56
• Real data science and empirical
investigation
• Personalized “did you know?” information
• Persistent needs analysis
–Evidence of impact
–Change in behavior
–Long-term effects and trends
Conclusion – What is Possible?
Questions?
Check Out the Entire Series
58
Ten Steps to Plan & Communicate Your xAPI Designs to a Web Developer
Five Things a Web Developer Needs to Know About xAPI
Establish an xAPI Infrastructure: Guide to Gain Sponsorship & IT Support
xAPI and Analytics: Measuring Your Way to Success
Are You an ISD, Business Process Engineer, or Both?
eLearning Authoring Tools: Taking the Next Step with xAPI
59
Contact UsSteve ForemanPresident, InfoMedia DesignsEmail: steve@infomediadesigns.comTwitter: @steveforemanimdLinkedIn: www.linkedin.com/in/steveforemanimd
Peter BerkingSenior Instructional Design, ADLEmail: peter.berking.ctr@adlnet.govTwitter: @pberkingLinkedIn: https://www.linkedin.com/pub/peter-berking/3/693/590
Craig WigginsCommunity Manager, ADLEmail: craig.wiggins.ctr@adlnet.govTwitter: @oxala75LinkedIn: www.linkedin/in/craigwiggins
Andy JohnsonTech Team Software Engineer, ADLEmail: andy.johnson.ctr@adlnet.govTwitter: @andyjohnsonadlLinkedIn: www.linkedin.com/pub/andy-johnson/36/768/50
top related