how to build amazing products through customer feedback
TRANSCRIPT
How to build amazing Products through Customer Feedback
/Productschool @ProdSchool /ProductmanagementNY
Venue Sponsored by
Eli Holder
- Product Leader, Designer and
Entrepreneur
- Founder of Notch.me and Unblab
www.productschool.com
How to build amazing Products through Customer Feedback
Good Qs,Good P’s.31 Good Questions Product Managers
can ask to build Good Products.
by Eli Holder [email protected]
“That’s a great question.”
Work Work Work.Eli’s Background
unblab. “Spent about 10 minutes training it to recognize my important emails and it's already saved me hours and hours of sorting through acquired by
“The SMS like model of having in and outbound messages keeping a conversation timely is the killer feature.”
“I found it very refreshing and energizing to see creative products such as this one at
“Until I saw your graphics and your ‘result stories’, I really wasn’t that dissatisfied. Now, I want to view all of my data this
“Eli has been instrumental in bringing an organized process to studying users"
Nodd
Nodd helps team leaders get honest, constructive feedback from their teams at work.
"I loved the emojis and the survey format was awesome. I was engaged the whole time. Excellent platform.”
Nodd.co
Dumb.2009
Dumb. Slightly less dumb.2009 2016
The power of questions.
Goal of this talk:
Hooray Questions!
Why Questions?“TheUnknown Unknowns”i.e. the epistemology of Donald Rumsfeld
UnknownUnknowns
KnownUnknowns
When you learn a new question:
Goals of the talk:
1.The power of 2. Asking the right
3.Product Questions
Audience Q:Warmup: “How are you?”
1.Who are we? Why are we here? (strategy & existential crises)
2.Who’s our audience? What do they want? (user research)
3.What problem should we solve? (opportunity assessment)
4.How should we solve the problem? (design sprinting)
5.Are we approaching Product /Market Fit? (build, measure, learn loop)
Big Product Q’s(for each Product Lifecycle
Stage)
Related Product Q’s.Who are we? What’s our mission? What’s our aligning metric? What are our organizational objectives for the quarter/year (OKRs)? Who are
the stakeholders? What are their goals? Why invest more in the product (e.g. instead of sales?) Who’s going to be building it? What are their
personal goals and interests? What are their strengths and weaknesses? How much of their time do we have (e.g. are they cross functional? do
they have other projects? are there churn risks?) What are the team’s strengths and weaknesses? What are people’s expected communication
styles? What’s the organization’s culture (e.g. Risk tolerance)? What’s the organization’s perceived brand? What are the expectations for a
product manager? What’s the (target) business model? What stage is the company at? What’s the prevailing role within the company? Who are
the other companies in the space? How is your company different? Tech advantages / debt? Design advantages / debt? Data advantages? What
are the religious issues that everyone’s tired of discussing? External requirements? Whats your support team like? How do you (think) you make
money? Who’s our audience? What do our users think about on the subway? Who are our primary (and secondary) users? What does a
typical day look like for them? What problems are they currently experiencing? (What are the 5 whys for the problems? What are they doing now
to solve them? What do they like about the current solution? What do they dislike about the current solution? What are they feeling?) What are
the most important dimensions to describe them? (e.g. demographics? behavior? company role? values? physical attributes? ) Who are the
extreme users? What do our users value? What are their goals? What are their obligations? What are their frustrations? What are their
stresses? What gets them excited? Who are the people in their lives? How do they typically interact with them? What’s the users value to us?
What language do they use to describe their experiences? What other products / tools are they using? What are they reading? Who are they
following? How do they find / buy related products? How do they fit into purchasing decisions? What can they spend? When do they spend?
What’s their organization’s culture like? How do they fit in? What’s their organization’s business model? How do competitors perceive them?
What does academia think about them? What problem are we solving? (value prop) What problems are incidental (non-goals)? For
whom do we solve this problem? (target market) How big is the opportunity? (market size) How will we measure success? (metrics)
What alternatives are out there now? (competitors) Why us? What’s our differentiator? Why now? (market window) Why might this a
bad idea? How will we market this? Other considerations for success? How should we solve this? What are all the different ways we
could possibly solve this? How are our competitors solving this? How have academics solved this? How have people solved similar problems in
different industries? Which solution is the best for us to pursue? Which solution is most interesting to the team? Which solution has the most
learning potential? Is it technically feasible? (What’s the relative complexity?) Is it consistent with the behavior / data we’ve collected so far?
Who are we?Why are we here?Product Lifecycle Stage #1:
Q:
Strategy & Existential Crises
What’s the mission?Q: Who are the stakeholders? What do they want?
Q:
What are our strengths & weaknesses relative to competitors?
Q:
Strategy & Existential Crises
Why it’s importantFollow your dreams, idiot.
Team Plan: Follow my dreams.
“What’s our one metric that matters”?“What are our near-term OKRs”?“What are our product principles”?
Ask:
What’s the mission?Q:
Who are the stakeholders?What do they want?
Q:
This is not you.
This is you.
1.Conformity bias2.Positivity bias 3.They don’t know4.Don’t want to look dumb
challenges:
Who are the stakeholders?What do they want?
Q:
The Hopes / Fears Exercise:
Who are the stakeholders?What do they want?
Q:
“What are your 3 biggest hopes for this project? (e.g. what do you hope this project accomplishes for [company]?)
“What are your 3 biggest fears about this project? (e.g. what do you think will go wrong?)
“What are your other considerations for this project? (anything else on your mind?)
Ask:
Hopes:• That project X opens up a new market for us• I’m optimistic about the chance at a new market• That project X opens up a substantial new
revenue stream for us• That project X has interesting technical
challenges
Fears:• That project X is much more technically difficult
than we expected• That we get distracted and don’t give project X a
fair shot• That we’re more excited about the revenue
opportunity than solving a real customer problem.
Considerations:
Hopes / Fears Exercise Results:
Who are the stakeholders?What do they want?
Q:
Answers:
Hopes, Fears and Psychological Safety.
What’s our mission? What’s our aligning metric? What are our organizational objectives for the quarter/year (OKRs)? Who are the stakeholders? What are their goals? Why invest more in the product (e.g. instead of sales?)
Who’s going to be building it? What are their personal goals and interests? What are their strengths and weaknesses? How much of their time do we have (e.g. are they cross functional? do they have other projects? are there churn risks?) What are the team’s strengths and weaknesses? What are people’s expected communication styles?
What’s the organization’s culture (e.g. Risk tolerance)? What’s the organization’s perceived brand? What are the expectations for a product manager? What’s the (target) business model? What stage is the company at? What’s the prevailing role within the company?
Who are the other companies in the space? How is your company different? Tech advantages / debt? Design advantages / debt? Data advantages?
What are the religious issues that everyone’s tired of discussing? External
Strategy: Relevant Q’s.
Audience Q:Any questions so far?
Who’s our audience?What do they want?Product Lifecycle Stage #2:
Q:
User Research
“Set up at least 30 meetings, or you won’t have a good
Michael SippeyFormer VP Product
User Research: Why?
Allison Smith*27, ClevelandNew MomYahoo Mail User
*Not really Allison But thank you to this person for CC Flickr photo.
How does Allisonuse email?
What does a typical day look like?What’s important to them?What’s holding them back?What do they think about on the
Qs:
Who’s our Audience?Q:
What do users want?Q:
Audience Q:Why shouldn’t you directly ask users “What do you want?”
“If I had asked people what they wanted, they would have said: Faster Horses.”
Henry Ford*Model-T Inventor
*Not really
What do users want?Q:
1.They can’t say what they want2.They won’t know what’s possible
3. It’s not their job to know
challenges:
What do users want?Q: “How do you currently do [area you want to explore]?”
“What are the top 3 things you love about how you’re doing it now?”
“What are the 3 biggest frustrations with how you’re doing it now?”
Ask:
Happy Event
Frustrating Event
Emotion Timeline Exercise.
What do users want?Q: "If you were to draw a chart of your emotions throughout your experience [doing X], with peaks when you're most happy and valleys when you're most unhappy, what would that look like?"
Ask:
A designer describes the 360 performance review experience.
Choose my reviewers.
Find out it’s perf review time.
Find out I have to write 4 reviews
I write the reviews, feel relieved
A long time goes by and I forget
about it. Boss emails to say it’s time to have the talk.
Talk day.
Promotion!
Emotion Timeline Exercise Results:
What do users want?Q: Answer:
Who are your competitors customers? What do they value?
Q:
“[your industry]” (testimonials|our clients|our customers)
site:[competitor_site] filetype:pdf
site:twitter.com "apple maps" ("is a”|"sucks"|"can't"|"fuck"|"shit")
https://twitter.com/search?q=to%3A[COMPETITORS_TWITTER]%20%3A)
:
Who are your competitors customers? What do they value?
Q:
User Research: Relevant Q’s.What do our users think about on the subway?Who are our primary (and secondary) users? What does a typical day look like for them? What problems are they currently experiencing? (What are the 5 whys for the problems? What are they doing now to solve them? What do they like about the current solution? What do they dislike about the current solution? What are they feeling?)
What are the most important dimensions to describe them? (e.g. demographics? behavior? company role? values? physical attributes? ) Who are the extreme users? What do our users value? What are their goals? What are their obligations? What are their frustrations? What are their stresses? What gets them excited? Who are the people in their lives? How do they typically interact with them? What’s the users value to us?
What language do they use to describe their experiences? What other products / tools are they using? What are they reading? Who are they following? How do they find / buy related products? How do they fit into purchasing decisions? What can they spend? When do they spend? What’s their organization’s culture like? How do they fit in? What’s their organization’s business model? How do competitors
What problem should we solve?Product Lifecycle Stage #3:
Q:
Opportunity Assessment
Audience Q:What is the #1 Most Important Question about a Product?
The Most Important Question:
What problem are we Q: *Independently of how we might solve it.
*Independently of how we might solve it.
Advil.
Features are for suckers..
What problem are we solving?
Q:
310M Americans experience mild headaches and body pain due to inflammation at least once per year. We’ll make that pain go away.
This:We’re manufacturing doses of ibuprofen. They’ll come in blue or brown pills. The bottle is child proof. We’re advertising the hell out of them.
Not This:
“Shiny feature X.”CEO: It needs to be social. Add share buttons everywhere!
PM: Sir, it’s a porn site. How many people share porn on Facebook?
CEO: The investors say we’re not growing fast enough. And some site called “Buzzfeed" is really taking off because of the sharing.
PM: It sounds like you’re saying the real problem is growth? Are there other ways we could solve that?
*Independently of how we might solve it.What problem are we solving?
Q:
Get to the root of the problem. Avoid building things that don’t add value.
What problem are we solving?*Q: *Independently of how we might solve it.
80M Americans have have a hard time finding things on the internet.
Finding information on the internet.Problem
Idea #1 Portals and directories (yahoo)
Idea #2 Search by phrase matching (altavista)
Idea #3 search by phrase matching and back links signal (google)
Problem framing leaves room for multiple solutions.
What problem are we solving?*Q: *Independently of how we might solve it.
PM:
Designer:*Quits. Goes to Facebook.*
Problem framing helps you avoid irritating your teams’ designers and engineers.
The signup button must be big and red so more people will see it and click on it.
What problem are we solving?Q: Problem Statements:
1. Focus on value, not features.2. Avoid building useless shit.3. Allow for alternate solutions. 4. Aren’t prescriptive.5. Force clearer thinking.
*Independently of how we might solve it.
From The PM Handbook (mostly).
Opportunity Assessment:The 11 most important questions for any product.
1. What problem are we solving? (value prop)2. What problems are incidental (non-goals)?3. For whom do we solve this problem? (target market)4. How big is the opportunity? (market size)5. How will we measure success? (metrics)6. What alternatives are out there now? (competitors)7. Why us? What’s our differentiator?8. Why now? (market window)9. Why might this a bad idea?10.How will we market this?11. Other considerations for success?
From The PM Handbook (mostly).
Big Product Q’s(for each Product Lifecycle
Stage)1.Who are we? Why are we here?
(strategy & existential crises)
2.Who’s our audience? What do they want? (user research)
3.What problem should we solve? (opportunity assessment)
4.How should we solve the problem? (design sprinting)
5.Are we approaching Product /Market Fit? (build, measure, learn loop)
“How can I improve?”
Q: How to get Honest Team Feedback at work.
1.Fear of Retribution2.Timing3.Management Middleman
challenges:
“How can I improve?”
Q: How to get Honest Team Feedback at work.
Problem:People at work- especially managers - have difficulty getting honest, constructive feedback from their teams at work, causing missed opportunities for learning and development. Root causes include: Reviewers’ fear of retribution, feedback infrequency, and concerns about how criticism can negatively impact coworkers’ careers.
Nodd helps team leaders ask the right questions to get honest, constructive feedback from their teams at work. It solves feedback’s honesty problem with 3 simple rules. Feedback through Nodd is:
Nodd keeps reviewers’ identities confidential and responses are only ever shown in aggregate from 3+ reviewers.
Nodd helps create a feedback ritual by prompting teams with simple, reviewer friendly micro-surveys every two weeks or at the end of each month.
Feedback is sent directly from person-to-person and never through management. So reviewers can feel safe knowing it won’t affect the person’s pay.
Anonymous. Frequent. Private.
Nodd.co
Summary:
1.The power of 2. Asking the right 3.Product Questions4.Nodd is in beta. Signup at: https://
http://bit.ly/good-product-questions
Upcoming Courses
www.productschool.com
www.productschool.com [email protected]
APPLY AT
Weekdays: June 7th
Weekends: June 4th
Three Rules Every Mobile Product Needs to Be Successful With The Author Of
Mobilized
UPCOMING WORKSHOP
www.productschool.com
RSVP ON EVENTBRITE
Appendix
“How could we solve this problem AND REALLY DO AN AWFUL JOB & EMBARRASS OURSELVES?“How would we solve this problem IN SPACE?!“How would we solve this problem IF OUR USERS WERE EXCLUSIVELY 19th CENTURY PIRATES?“How would we solve this problem WITH BENEVOLENT, AUTONOMOUS NANO-BOTS?“How would you solve this problem BY DOING THE OPPOSITE OF YOUR LEFT-NEIGHBOR’S IDEA?
What are all the different ways we could possibly solve this?Q:
Ask:
To assess team alignment, how many different ways do people answer:“What problem are we solving for our users over the next 2 months?”When someone shows you work:“What were your goals for this?”To assess team’s understanding of a story:
“On a scale of 1, 2, 3, 5, 8, 13, 21, how complex does everyone think this story is?” (planning poker: okay for planning, great for communication).
Are we on the same page?Q: Ask:
(probably not)
How can we improve our process?Q: Ask: Ask each team member to write:
“What are 3 things that we don’t currently do, that we should start doing?”“What are 3 things that we currently do, that we should stop doing?”“What are 3 things that we currently do, that we should continue doing?”