Measuring What Matters for Maturity

Thomas Vander Wal, Independent Consultant 9 November 2017 :: KM World

Designing, developing, and / or managing social platforms since

1996

Model of Attraction (2002)

Come to Me Web (2004)

Folksonomy (2004)

70+ Social Lenses (2008)

Connected Company (2010)

Understanding Need

Understanding Need• Measurement has multiple purposes:

- Measuring the finding and reuse of successful knowledge related to questions and searches

- Assessing practices and patterns that build to knowledge capture and reuse

- Seeing patterns around growth, maturity, and role types

- Identifying change and adaption

Understanding What Matters• Depth of use / engagement

• Most systems and platforms focus on pure numbers, as in clicks

• Identify different social patterns and roles

• Finding answers that fit is important

- Identify patterns that lead to successful answers

- Identify and build healthy patterns that lead to finding answers, and measure success of the patterns

• Identifying when change in what is known is needed and how the change is made

Keep In Mind…• Not all organizations are the same

• Departments, location, and other contexts create differences

• Not all people are the same

- Personality traits

- Mental models

- Cultures

Improving Search and Knowledge Success

How to Improve Search and Knowledge Success• Pair search with the ability to ask a question to a group or community

• A focus on finding and refinding information and knowledge

- A community with groups that can help point others to existing knowledge builds value (look at time to response and percent answered)

- A community with groups can also answer unanswered questions or point to previously unshared resources

• Providing answers and pointing to previously unshared resources can have strong lasting value

• Valuable knowledge formalized into learning modules - measure to identify and then results

https://communityroundtable.com/best-practices/thecrs-work-out-loud-framework/

How to Improve Search and Knowledge Success• Working out Loud

- Questions asked in the open

- Potential answers shared

- Identifying answers that work

- People sharing answer that works by voting it up and / or linking to it

- Pointing to an answer that works build patterns that most search engines use to increase that answer’s search relevance

- Use depth of use ladders, with measurement of progress and plateaus

Roles and Support

Focus on Roles and Support

• Ladders are helpful for building maturity, but understanding roles and related activities is a different focus to also track and measure

• Select a few roles that have value and identify their activity patterns that are valuable for those roles

- Build analytics and measurement models to see roles in action, so to better support them

• Sharer • Lurker • Writer / Creator • Editor • Curator • Connector • Synthesizer • Theorizer • Mitigator • Negotiator

• Contextualizer • Interloper • Infovore • Learner • Monitor • Councilor • Gossip • Critic • Expert • Broadcaster /

Rebroadcaster

Social Roles

Open Measurement as a Game

Gamification Realities• Gamification can move the needle, but it only goes so far

• Understanding the three high level segments helps frame how it can work or not

- Those whom are receptive

- Those not moved by the gamification and it doesn’t matter

- Those who are turned off by it

Gamification User Impact Type Ratios

Perc

enta

ge

0

33

67

100

Balanced Not Moved Don’t Like Moved By

Moved User %Not Moved %Dislike User %

Gamification Realities• Gamification will positively move the needle up to the point of saturation

of those who are receptive to it

• The 2 categories of people not receptive to gamification look to the patterns of use by those who are moved forward by gamification

- What is rewarded is important as it sets patterns for others to follow

• It is difficult to measuring receptivity to gamification to understand the size of each category prior to rolling it out

- If testing it, keep in mind those likely to opt into a trial of a service are likely going to be receptive to gamification

Use Flow Models to Identify Points of Measurement

Team

Subject Matter Group 1 Subject Matter Group 2 Subject Matter Group 2Subject Matter Group 2

Community in Organization

Question Answer QuestionAnswer

Where work happens

Relevant Info Shared

Keys to Success

Key to Measurement• Access to the data

- This is often the most difficult part

- Platforms with open data APIs enable getting to that data

• Measure to see trends and patterns, that lead to success and frame success

• Measure to see gaps and pain points as well as to watch them get resolved

Thank you!

URL: www.vanderwal.net Blog: www.personalinfocloud.com E-mail: thomas@vanderwal.net Twitter: @infocloud