User Feedback and its Role in Making Great Mobile Software

November 10, 2009Stephen King, CEOMob4Hire.com

Why is user feedback important?

① 69% of mobile users discover apps based on top store rankings, user reviews and recommendations from friends.

② Making great software is your best marketing strategy

③ You can’t make great software without including users

The Big Picture

A bunch of stuff you probably already know about the mobile app industry and hopefully some stuff you don’t

… to get us all on the same page

• 6 out of every 10 people in the world have a mobile handset

• 4B subscriptions; 18% of people (700M) have two subscriptions.

3.3B mobile handset users

http://www.tomiahonen.com/

Mobile users compared to other industries

Smartphones experiencing rocketship growth

• North American adoption and usage is distinctly different than other continents

• European handsets and subscriptions are often separate purchases; SIMs interchangeable, much more operator competition

Where is everybody?

http://www.tomiahonen.com/

From “no phone” to “cell phone.” Of the 280M people with phones in Africa, 260M of them use mobile handsets.

• Many markets saturated (top 5: Italy:153%, Greece, Hong Kong, Portugal, Israel): new subs slowing

• U.S. at 84%, Canada at 61% (the only country < 80% that’s NOT a developing nation)

• Still lots of growth for new users in developing nations

Handset Market Saturation

http://www.tomiahonen.com/

App Industry growth 26% YOY from $8B to $25B

http://www.juniperresearch.com/shop/viewreport.php?id=180

It’s not about handset features, it’s about software

http://www.tomiahonen.com/

“… it’s all about the apps.” Wired

Networks, Handset Mftrs and O/S’s Want YOU

“No single company can come up with the massive portfolio of applications that will please consumers. That would be like asking Wal-Mart to make all the products it sells. And that’s why platforms are trying so hard to woo developers to make the programs that will run on their phones / platforms.” - Fortune Magazine, Aug 2009

I like these guys

“Don’t Crash. Don’t Suck. Get Sticky”

Problems with including user feedback in Mobile Development

Don’t Crash

The “Fragmented Mobile Ecosystem”

Mobile Ecosystem• Handset diversity ... Over 25,000 on 750+ operators• Software diversity

• O/S diversity (platforms, middleware, widgets)• Standards implementation diversity

(“According to standards, this should work …”, “That API isn’t documented but it kinda works like this …”)

• Mobile 2.0 browser diversity• Operator / Environmental diversity

Developer or Regional• Feature variations, such as freemium business models• User preference diversity (requirements, accessibility,

languages)

Don’t Suck

“The Curse of the One Star”

One star reviews will kill your revenue … no subsequent downloads will occur

Difficult to get user feedback; no direct relationship

Get Sticky

App persists on users handset over many months

App is used frequently

App is recommend to friends and given good reviews

Doesn’t Mobile 2.0 fix fragmentation and usability?

It’s not Web 2.0, but it is browser based, just like your PC internet surfing. Sort of.

And, it’s not going to fix anything.

Just change it.

Mobile 2.0 Usage … web on a mobile browser

http://www.tomiahonen.com/

More people access the internet through mobile handsets (1.02B) than PC’s (940M)

Within 2 years, 71% of U.S. mobile users will access the mobile internet vs. 28% in 2008*

* Nielson research http://www.slideshare.net/clisco/the-mobile-web-is-awesome

Mobile 2.0 isn’t easy, either

• Just like PC web, browsers have their “idiosyncrasies”; in mobile it’s worse• Mobile specific websites required. Test on 25+ different browsers.• Keep info minimal. Thin graphics. Low bandwidth. Collect only essential

data.

Mobile Functional and Usability testing

So … how can we make our software and mobile websites better?

• The only media generation that can replicate all the benefits of the other six • The first personal media channel, permanently carried and always on: “Intimate”• Built-in payment channel• Point of creativity “eyewitness,” enabling user-generated content• Near-perfect audience data• Social and location context of media consumption

Mobile changes everything; use it to your advantage

What is usability as it relates to Mobile Apps?

•Small and varied screens.• Fonts are too small• Contrast and color choices are poor• Too much stuffed into screen real estate

•Lack of proper keyboards means input is awkward (although BlackBerry is the best ). Make it easy.

•Use device feature characteristics; stylus, cross pad, track pad, track ball, touch screen, etc…

•5 minute tasks vs. 50 minute tasks • Install crashes, software loading, download delays•No mobile 2.0 optimization on websites

The Mobile Test “Cycle of Life”

2. White BoxPrototyping, alpha testing, developer stage testing, regression testing, code review. No emulators!

4. GoldUsability and user experience feedback. Customer loyalty and business model studies.Viral / killer feature feedback for marketing.What needs to go into next version?

3. Black BoxUsers testing software on their devices, in-market. Developers make the worst testers. How do users respond to your app? Make sure it’s a great experience BEFORE you port or post.

1. Blank CanvasWhat do we want to build? Who is our target user? Storyboarding. What’s our business model?

“Mobile Test Cycle of Life” © 2009 Mob4hire.com

User Feedback in the 10 Step Development Cycle

“User Feedback in the 10 Step Mobile Development Cycle ” © 2009 Mob4hire.com

A Customer Loyalty Primer

How does the customer feel about getting software they don’t like?

Not great. You can learn from that.

Turn it into revenue growth.

The effect of a bad user experience in the Mobile Ecosystem

• True Story! Happened to me the week Blackberry App World launched• My Carrier is Rogers Wireless in Canada. • My handset is a Blackberry Bold. The above picture is what it usually looks

like.• I downloaded a few apps including Google Talk• The free Google Talk downloaded ok. But when it rebooted my system, my

Bold started spewing error messages like “java exception.” Lots of them! See next slide.

Background is gone as are all my photos. Icons are moved (some programs are missing). The keyboard #’s wouldn’t work, only the QWERTY letters. The “airplane” mode icon is missing. SMS’s and Emails in both inbox and sent items are gone. Email config is gone. Kept my contacts, WiFi and weather settings intact, though.

Where’s the [Not OK] button?

After I downloaded/installed Google Talk, my Bold had errors & reset to default Rogers O/S footprint, without any of my saved information.

WARNING!!! PAINFUL!!! Who should I blame?

Company / Product Business PriceRecommend

?(Advocacy)

Buy from you

again?(Retentio

n)

Buy something else from

you?(Purchasing)

Why?

BlackBerry App World App Store Free Yes, but ... Yes Yes

Love it … But, now I don’t trust it quite as much after the crash and apps that are obviously baaaaad.

BlackBerry Handset $$ Evangelize Yes YesLove my Bold (Pearl prior).

Productivity (keyboard) & less data $$$ than iPhone.

Rogers Operator $$$$ Neutral Probably Probably

Not related to Bold crash, so I don’t blame them; consumers will,

though

Google* Talk (IM) Free NONONO Yes Yes OMG!!! ARGH!!!Reset my whole system.

What are your customers saying about you? Evangelize? Neutral? Criticize?

Your revenue growth depends on creating customer loyalty

for your software

* Actually written by RIM …

• Company success determined by different types of loyalty

Three Dimensions of Customer Loyalty

Based on research by Dr. Bob Hayes, Ph.D.www.businessoverbroadway.com

BusinessPrograms

MarketingSales

Service

CustomerDevelopment

(cross/up-sell)

FirmValue

CustomerLifetime

Value

CustomerAcquisition

CustomerRetention

BusinessPrograms

MarketingSales

Service

CustomerAcquisition

FirmValue

CustomerLifetime

Value

CustomerRetention

CustomerDevelopment

(cross/up-sell)

Product

Three ways to grow business

Outcome Behavior Measure

Revenue Growth

Increase length of customer

life

Churn/ Defection rate

Retention Loyalty

Increase size of customer base

Number of referrals

Advocacy Loyalty

Increase number of purchases

Purchase behavior

Purchasing Loyalty

• How likely are you to upgrade your MOBILE APP from Freemium to a Paid subscription?(Please rate us on a scale of 0-10 where 0 = Extremely Unlikely and 10 = Very Likely)

• Would you buy this app from us again?

• How likely would you be to recommend MOBILE APP to a friend?• What’s the main reason you’d give for recommending MOBILE

APP?

• How likely are you to purchase different types of APPS from us?• What’s the main reason you’d want to purchase different APPs

from us?

Customer Loyalty Indices … sample questionsR

eten

tion

Pur

chas

ing

Adv

ocac

y

Customer Loyalty Qualitative + Quantitative Response

0 1 2 3 4 5 6 7 8 9 10

EvangelistsFansAdvocatesAngelsMissionariesPromoters

DetractorsCritics

PassiveNeutral

Ambivalent

How likely would you be to recommend MOBILE APP to a friend?

What’s the one thing we could do to improve that rating for MOBILE APP?

(calculate an average %)

Plot verbatims according to biggest impact from recommending behavior

Lo

yalt

y

ProfitabilityLow High

Crit

ic

N

eutr

al

E

vang

elis

t

This is the stuff you want to do

Contact Dr. Bob Hayes, Ph.D. for more help on customer loyalty measurement

www.businessoverbroadway.com

Starting a Customer Loyalty Measurement Program

•Someone in the Executive Team must sponsor it• Don’t delegate leadership or the program will fail• Get outside help to set it up

•Measure constantly and consistently• standard survey tools• at regular events or times; when the user registers,

disconnects, upgrades, hits a high score

• Incorporate metrics into executive management meetings

•Watch for quantitative upward trending … if things are going up, that’s good!

Townhalls

You can’t make great software without the user

Townhalls can be live or virtual events

I learned this technique at Intuit, adopted from similar practices at Toyota

In Mobile, Standard Usability Research applies

• Lab-based (using a camera to record the session)• Diary-studies (asking people to keep a diary of how they have used their mobile phone and any problems they encounter)

• Storyboard paper prototypes (running usability testing on a paper-based version of the site, using mobile phone screen-sized pieces of paper)

• http://www.webcredible.co.uk/user-friendly-resources/web-usability/mobile-usability.shtml

• In mobile, accessibility to users and handsets is a problem

• Today, we’re going to learn a technique known as a “Townhall”

Who is at a Townhall?

• Team Members• 1 team member to be Facilitator• 1 coordinator to handle logistics• 1 note taker for “stream of consciousness”• Bring in others from the team to participate (support,

development, sales, executives mandatory) … virtual townhalls can have many

• Recruiting customers• 12 participants is great• Contact day before townhall to confirm / remind• Ensure customers have directions and parking instructions

(pay for parking if needed); if virtual, clear instructions on how to log in

• Townhalls take approximately 2 hours

Prepare

• Prepare two flipcharts before meeting• Put your company or software

logo XXXX on page 1 of both flipcharts

• On flipchart 1, page 2, write:• “Take a few minute to jot down:”

– 3 things you like about XXXX– 3 things you dislike about XXXX– 3 opportunities that we’re missing”

• On flipchart 2, page 2, write:• “This is what we’ll do:

– Record your ideas / issues (private time)– Vote to prioritize ideas / issues– Discuss top ideas / issues”

• Cover both flipcharts with page 1• On wall, clear three large areas with the titles “Like” “Dislike” and

“Opportunities

Townhall Opening Comments

• Welcome … XXXX is built on customer feedback! We want to hear from everyone … facilitator will guide conversation

• Describe process briefly• Flipchart #1: Customers to write down answers to 3 questions• Flipchart #2: Describe process

• Results will be summarized and communicated to the rest of the XXXX team We can’t promise anything today, but rest assured your thinking will be evaluated for production

• Pass out post it notes … make sure everyone uses the same colour for the same question. In a virtual situation, everyone in the group could email you with each question as the subject line.

Get Ideas Recorded

1. Ask participants to write down 3 answers to each of the following questions1: What do you like about XXXX, XXXX's

products, XXXX's services, the way we market/sell to customers, the way we support customers, training, etc.?

What do you dislike about XXXX? What are the opportunities we're

missing?

2. Give as much time as needed … 10 to 15 minutes?

3. XXXX team collect post-it notes and stick on wall … consolidate comments into categories

1Ensuring that participants actually write down their answers will help “commit” them to the discussion.

Prioritize by Voting

1. Once all post-it notes are on wall, pass out 10 little sticky dots to everyone

2. Everyone should stand up and review what’s on the wall. That includes XXXXstaff -- you can discuss, but don’t try to solve problem … “seek to understand.” Ask questions to dive deeper into problem. XXXX staff can get dots of another colour, optional.

3. Customers place their dots on the sticky notes that resonate the most with them. They can allocate their dots any way they want; 10 on one note if they like

4. Virtually, go through each issue with the team voting in a chat room

“Seek to understand”

On flipchart, list the top priorities from

each category based on the # of dots

Then … “Seek to understand” … dive into each priority … involve customers in trying to understand. Spend only a little time on “likes” and much more on “dislikes” and “opportunities”. Ask customers how to solve it.

“Low hanging fruit”Im

po

rtan

ce

Ease of implementation

Difficult Easy

Low

Hig

h This is the

stuff you want to do

Follow Up … Close the User Feedback Loop

• Immediately following townhall … team meets to discuss process and improvements for next time.

• The facilitator or coordinator prepares a list of customer comments, along with the number of votes and any additional suggestions or commentary.

• Include the “stream on consciousness” notes taken• Team meets to summarize and take action• “Close the Feedback Loop!”

• “This is what we heard”• “This is what we’ll be working on.”

Thanks!

November 10, 2009Stephen King, CEOstephen@mob4hire.com