analytics and xapi: measuring your way to success€¦ · analytics and xapi: measure your way to...

Session 809Analytics and xAPI:

Measure Your Way to Success

Steve Foreman, InfoMedia Designs

Andy Johnson & Craig Wiggins, ADL

Las Vegas, NV | September 30 – October 2, 2015

Sponsored by the Office of the Under Secretary of Defense for Personnel and Readiness (OUSD

P&R)

Analytics and xAPI: Measuring Your Way To Success

Peter Berking, Andy Johnson, Craig Wiggins, Steve Foreman

Devlearn 20152 October 2015Las Vegas, NV

The authors

3

Steve ForemanPresident

InfoMedia Designs

Andy JohnsonResearch Analyst

ADL

Craig WigginsCommunity Manager

ADL

Peter BerkingSenior Instructional

DesignerADL

4

The article

Step 1: Analyze need

Step 2: Design the solution

Step 3: Design the xAPI model

Step 4: Develop and implement

Step 5: Measure, monitor, improve

Agenda

The situation






Agenda

9

1. Identify business problem

2. Identify goals indicating success in addressing

problem

3. Identify individual metrics to track with xAPI*:

• Refining understanding of problem

• Indicating progress towards addressing problem to:

• Stakeholders

• End users

4.Identify factors that contribute to business problem

1. Directly

2. Indirectly

Needs Analysis Process

*Remember - your metrics are unconstrained by the limitations of current standards!

10

Direct Success Contributors

Business

Problems

Business

Metric to

Track

Revenue below targets* Which product management

experts are sales people

contacting?

Number of contacts per product

manager

Lower numbers of prospects

and contacts in sales funnel*Which product management

experts are sales people getting

most value from?

Sales rep ratings of product

manager interactions

Higher percentage of

customers dropping after

prospect and contact stages*

Which sales managers are

conducting meetings?

Whether weekly meetings

occurred per sales manager

Time is allocated less

efficiently for most sales

teams compared to one high

performing sales team

Are sales reps attending the

meetings?

Number of sales reps in

attendance

*for 3-6 months after launch of updated product line

11


Business

Problems

Business

Metric to

Track




most value from?




12


Business Problems

Lower numbers of prospects and contacts in sales funnel

13


Business

Problems

Business

Metric to

Track

Lower numbers of

prospects and contacts in

sales funnel*

Which product management


most value from?




14


Which product management experts are sales people getting

most value from?

15


Business

Problems

Business

Metric to

Track




most value from?




16


Sales rep ratings of product manager interactions

Business Metric to Track

17

Indirect Success Contributors

Learning &

Performance

Solution

Component

Question

to Answer

Activity to Track

Searchable Product

Knowledgebase

What types of information are

salespeople seeking and either

finding or not finding?

Sales rep searches where

results were:

• Selected and accessed

• Not selected and accessed


salespeople using?

Sales rep consumption of

information by:

• Product (i.e., each product in

the product line

• Type (i.e., market research,

competitor comparisons,

product specifications, sales

collateral


being populated?

Product manager and

marketing expert contribution

by:

• Product (see above)

• Type (see above)

18


Learning &

Performance

Solution

Component

Question

to Answer

Activity to Track

Searchable Product

Knowledgebase





results were:




salespeople using?


information by:


the product line




collateral


being populated?

Product manager and


by:



19


Learning &

Performance

Solution

Component

Question

to Answer

Activity to Track

Searchable Product

Knowledgebase





results were:




salespeople using?


information by:


the product line




collateral


being populated?

Product manager and


by:



20


Learning & Performance Solution Component

Searchable Knowledge Base

21


Learning &

Performance

Solution

Component

Question

to Answer

Activity to Track

Searchable Product

Knowledgebase





results were:




salespeople using?


information by:


the product line




collateral


being populated?

Product manager and


by:




Question to Answer

What types of information are salespeople

seeking and either finding or not finding?

23


Learning &

Performance

Solution

Component

Question

to Answer

Activity to Track

Searchable Product

Knowledgebase





results were:




salespeople using?


information by:


the product line




collateral


being populated?

Product manager and


by:



24


Activity to Track

Sales rep searches where results were:




Online Access to

Product Management

Experts



contacting?


manager



most value from?



Weekly Sales Team

Meetings






meetings?


attendance

Learning &

Performance

Solution

Component

Question

to Answer

Activity to Track


Online Access to Product Management Experts

Learning & Performance Solution Component


Online Access to

Product Management

Experts



contacting?


manager



most value from?



Weekly Sales Team

Meetings






meetings?


attendance

Learning &

Performance

Solution

Component

Question

to Answer

Activity to Track

30


Question to Answer

Which product management experts are sales

people contacting?


Online Access to

Product Management

Experts



contacting?


manager



most value from?



Weekly Sales Team

Meetings






meetings?


attendance

Learning &

Performance

Solution

Component

Question

to Answer

Activity to Track

32


Activity to Track

Number of contacts per product manager






Agenda

Learning & Performance Solution

36

Knowledgebase

Sales reps use a searchable

knowledgebase to find

information from product

management and marketing.

● Product information

● Market research

● Competitive analysis

● Customer intelligence

37

Access to Experts

Sales reps can

connect with

product

management

experts.

38

Social Networking & Collaboration

Sales information sharing

meetings are held with

structure & guidance that

emulates high

performance sales team.






Agenda

xAPI Model Design Process

40

• Identify actions to track within current processes

• Carefully create verbs with one-to-one correlation

• Determine what is being measured (if anything) with each action

• Create “natural” Statements

• Identify characteristics/profile information

• Put it all together to determine which metrics come out

Identify Actions

41

Prospected

Contacted

Negotiated

Made an Offer

Closed/Sold

Dropped

Searched

Accessed

Published

Updated

Rated

Contacted

Asked

Answered

Rated

Held

Attended

Verbs One-to-One

42

Tying in Measurables

43

44

Constructing Statements

• <Actor> <Verb> <Object> <Result>

• “Person” “Action” “within Activity” with “Measured Data”

• Andy sold (advanced the state within) in “General Sales”

500 of Widget 505 for $20000.00 USD

• Andy searched KnowledgeBase for “Widget505”

• Andy contacted Craig (in Ask the Experts)

• Andy asked Craig (in Ask the Experts) “Can you tell me the

top selling point of Widget505?”

• Andy attended “September 2015 Team Alpha Sales

Meeting”

Background and Other Info

45

Meet Bob.

Salesman for 30 years(with Integrated Widgets for 25 years )

Certified, qualified and educated

Balanced task/time allocation

46

Putting It All Together






Agenda

48

xAPI Development Process

• Establish continuous dialog between Designer and Developer to establish what is tracked, for which roles, and in what context

• Look carefully across communities of practice for established best practices

• Translate the natural language Statements into code

• Create visualizations, probably using tools

49

• Ask questions like “Does it make sense for this type of person to do this activity?”

• Or, “What did you want to see tracked out of this, a 1-5 rating or a like/dislike?”

• Easy to use when sides speak the same language but understand their different roles

Designer and Developer

50

Take Verbs from CoPs

51

Translate into Code

52

Code - Part 2

53

Visualization Tools

Placeholder for xAPI Dashboard image






Agenda

56

• Real data science and empirical

investigation

• Personalized “did you know?” information

• Persistent needs analysis

–Evidence of impact

–Change in behavior

–Long-term effects and trends

Conclusion – What is Possible?

Questions?

Check Out the Entire Series

58

Ten Steps to Plan & Communicate Your xAPI Designs to a Web Developer

Five Things a Web Developer Needs to Know About xAPI

Establish an xAPI Infrastructure: Guide to Gain Sponsorship & IT Support

xAPI and Analytics: Measuring Your Way to Success

Are You an ISD, Business Process Engineer, or Both?

eLearning Authoring Tools: Taking the Next Step with xAPI

59

Contact UsSteve ForemanPresident, InfoMedia DesignsEmail: [email protected]: @steveforemanimdLinkedIn: www.linkedin.com/in/steveforemanimd

Peter BerkingSenior Instructional Design, ADLEmail: [email protected]: @pberkingLinkedIn: https://www.linkedin.com/pub/peter-berking/3/693/590

Craig WigginsCommunity Manager, ADLEmail: [email protected]: @oxala75LinkedIn: www.linkedin/in/craigwiggins

Andy JohnsonTech Team Software Engineer, ADLEmail: [email protected]: @andyjohnsonadlLinkedIn: www.linkedin.com/pub/andy-johnson/36/768/50

mailto:[email protected]

http://www.linkedin.com/in/steveforemanimd


https://www.linkedin.com/pub/peter-berking/3/693/590


http://www.linkedin/in/craigwiggins


http://www.linkedin.com/pub/andy-johnson/36/768/50

analytics and xapi: measuring your way to success€¦ · analytics and xapi: measure your way to...

Documents