gathering content insights from user surveys confab2014

Gathering Content Insights from User

SurveysConfab Central 2014

t: @Kathy_CS_Inc #ContentSurvey #ConfabMN

Kathy Wagner:

Mom (and grandma!)

Tai chi master (yeah, right)

Not quite a crazy cat lady

(yet…)

Kathy Wagner:

Technical Communicator

User Experience

Researcher

Content Strategist

“We cannot solve our problems with the same kind of thinking we

used to create them.”~ Albert Einstein


Why use surveys for content projects?

Conduct research quickly, easily, and affordably

Gives your audience a voice

Aligns stakeholders and streamlines decision-making




It’s your responsibility too

Content Strategy

Customer Experience


Audience Attitudes + Content InsightsGoals Topics

Priorities Formats

Likes & dislikes Channels

Comparisons Terminology

t: @ Kathy_CS_Inc #ContentSurvey #ConfabMN

Surveys are GOOD for gathering insights to make better content decisions based on user need and interest.

1. Focus your research questions2. Choose your sample population3. Choose your incentive4. Design the survey5. Distribute the survey6. Analyse & communicate the results


6 Steps to Effective User Surveys

“Focusing is about saying ‘No’.”

~ Steve Jobs


Don’t begin your survey until you know:

1. Exactly what you want to find out.

2. From who.

3. How you’re going to use the information.

Focus your research questions




2. From who.



Do you want to find out:What terminology to use?What topics to cover?What formats to publish?Or…?




2. From who.



Do you want information about:All your audiences?Specific audiences?Sub-set of an audience?




2. From who.



How are you going to use the information?

Don’t gather any information that you don’t have a clear and immediate use for!



What kind of content do our web users want?

Example:

Which news article topics and formats do our alumni audience want to see on our homepage?


“‘Stalking’ is such a strong word. I prefer ‘intense research of an individual.’”

~ Sandrine, in Girl From Paris


Choose your survey sample


Choose your survey sample

Tips for choosing your survey sample:

Random sampling is better.

Don’t survey people more than 4 times per year.

Target 400 responses, or as many as possible.


“Nobody made a greater mistake than he who did

nothing because he could only do a little.”

~ Edmund Burke


Choose your incentive

Ideas for budget-conscious incentives:

$5 or $10 Starbucks gift cards.

Enter names into a draw for an iPad or ?

Free or discounted product or service.


“If I had an hour to solve a problem and my life depended on it, I would use the first 55 minutes determining the proper questions

to ask.”~ Albert Einstein


Design your survey: Structure

Tips for structuring surveys:

Include opening and exit text.

Divide the survey questions into categories.

Organize the questions strategically.







Opening text should include:

Name of your organization

Confidentiality of data

Estimated time to complete

Information about incentive

Why their feedback is important







Exit text:

Thank people for their time

Reiterate importance of feedback

Remind them about incentive







Example categories:

1. Audience tasks and goals

2. Topics of interest

3. Formats of interest








Broad & Easy

Specific & Sensitive

1. Screening questions

2. Easy questions

3. Difficult questions

4. Sensitive questions

Design your survey: Question types

Question types to consider:

Open-ended questions

Multiple choice

Ranking order

Rating scale





Multiple choice

Ranking order

Rating scale

Open-ended questions allow participants to answer freely.

Example:

What are two ways that we could improve our videos?





Multiple choice

Ranking order

Rating scale

Multiple choice questions force participants to choose one or more answers from a predefined list.

Example:

Are you a US Citizen? (Y/N)t: @Kathy_CS_Inc #ContentSurvey #ConfabMN




Multiple choice

Ranking order

Rating scale

Ranking order questions ask participants to rank answers in priority against each other.





Multiple choice

Ranking order

Rating scale

Example:

Please rank the following kinds of articles, from least important to most important:

Personal profiles

Brief news articles

Detailed research articles

1st 2nd 3rd





Multiple choice

Ranking order

Rating scale

Rating scale questions ask participants to rate the importance of various options.

To consider:

Numbered scales

Word scales

Using neutral scorest: @Kathy_CS_Inc #ContentSurvey #ConfabMN




Multiple choice

Ranking order

Rating scale

Example numbered scale:

On a scale of 1 – 7, with 1 being awful and 7 being amazing, how would you rate your overall experience with our knowledge base articles?





Multiple choice

Ranking order

Rating scale

Example word scale:

How satisfied are you with our knowledge base articles?o Very satisfied

o Satisfied

o Neither satisfied nor dissatisfied

o Dissatisfied

o Very dissatisfiedt: @Kathy_CS_Inc #ContentSurvey #ConfabMN




Multiple choice

Ranking order

Rating scale

Understanding neutral scores:

#1) I would rather watch a real-life video than an animated video.o Strongly agree (5)

o Agree (4)

o Neither agree nor disagree (3)

o Disagree (2)

o Strongly disagree (1)t: @Kathy_CS_Inc #ContentSurvey #ConfabMN




Multiple choice

Ranking order

Rating scale

Understanding neutral scores:

#2) I would rather watch a real-life video than an animated video.o Strongly agree (4)

o Agree (3)

o Disagree (2)

o Strongly disagree (1)

o No opinion (void)t: @Kathy_CS_Inc #ContentSurvey #ConfabMN


This question type

Is best for

Open-ended Attitudes and opinions or when options are unknown.

Multiple choice Demographic info, or all feedback with a limited set of options.

Ranking Comparing importance between up to five options.

Rating number scaleRate the importance of items within a limited set of options.

Global or ESL audiences when survey can’t be translated.

Rating word scale Most rating questions and audiences.

Rating w/o neutral score

Gathering insights to make decision between two options.


Design your survey: Phrasing questions

Do not:

Lead respondents to an answer

Use emotional language

Use double-barrelled questions

Use extreme absolutes

Provide vague or subjective options

Do:

Be brief

Be grammatically simple

Use simple, familiar terms



Do not:

Lead respondents to an answer Bad

Staying involved in your professional society is important. How often would you like to receive news updates?

Better

How often would you like to receive news updates?


Hey, You:What’s a better

question?

Do:


Do not:

Lead respondents to an answer

Do:

Bad

Staying involved in your professional society is important. How often would you like to receive news updates?

Better

How often would you like to receive news updates?



Do not:


Bad

How likely are you to share life-saving safety information with your loved ones?

Better

How likely are you to share safety information with your family members?


Do:

What’s a better question?


Do not:


Bad

How likely are you to share life-saving safety information with your loved ones?

Better

How likely are you to share safety information with your family members?


Do:


Do not:

Use double-barreled questionsBad

Would you like to be notified about new articles through Facebook and Twitter?

Better

Would you like to be notified about new articles? (Y/N)

Choose how you’d like to receive notifications:


Do:



Do not:


Bad

Do you always read articles from beginning to end?

Better

Do you read articles from beginning to end?


Do:



Do not:

Provide vague or subjective optionsBad rating scale

o Frequently

o Sometimes

o Rarely

Better rating scale

o Daily

o A few times per week

o Once a week or less


What’s a better rating scale?

Do:


Do not:

Provide vague or subjective optionsBad rating scale

o Frequently

o Sometimes

o Rarely

Better rating scale

o Daily

o A few times per week

o Once a week or less


Do:


Question 5 4 3 2 1

Satisfaction

How satisfied are you with…

Very satisfied SatisfiedNeither satisfied nor dissatisfied

DissatisfiedVery dissatisfied

Agreement

State your level of agreement with…

Strongly agree

AgreeNeither agree nor disagree

DisagreeStrongly disagree

Extent To what extent do you… To a large extent

To a moderate extent

To some extentTo little extent

Not at all

Helpfulness

How helpful is… Very helpfulSomewhat helpful

NeitherNot so helpful

Not at all helpful

Interest Indicate your degree of interest in…

Considerable interest

Moderate interest

Some interestLittle interest

No interest

Importance

How important to you is…

Very important

Somewhat important

Neither important nor unimportant

Somewhat unimportant

Very unimportant

Quality Please rate the quality of…

ExcellentAbove average

AverageBelow average

Poor

Tips for phrasing questions using a 5-point scale


Tips for minimizing survey bias:

Provide list options in random order.

Keep questions and answer options short and simple.

Allow respondents to answer anonymously.

Allow respondents to skip questions.

Design your survey: Minimize bias


Tips for maximizing response rate:

Pre-notify survey sample of an upcoming survey.

Pilot test your survey for clarity and functionality.

Keep the survey short. Aim for 5 – 10 minutes.

Offer incentives.

Schedule reminder messages.

Design your survey: Maximize response


“You’ve got to kiss a lot of toads before you find your prince.”

~ Proverb


Distribute the survey through:

Email lists (Monday mornings work best)

Your website

Company newsletters

Social network channels

Relevant industry, professional, or social networks or clubs

Printed announcements that staff can hand to customers

Distribute your survey


“They’ve done studies, you know.

60% of the time, it works every time.”

~ Brian Fantana, in Anchorman


Tips for analysing and communicating survey results:

Remove invalid responses

Choose the best chart type

Filter to compare responses between audience segments

Provide context

Analyse and communicate the results



This chart type Is best for

Vertical bar chartComparing percentages of 2 – 7 different groups.

Horizontal bar chart

Comparing percentages of 8 or more different groups

Pie chart Illustrating a sample breakdown.



Example: Vertical bar chart



Example: Horizontal bar chart



Example: Pie chart



Provide context. Example:

People want to find specific information

All target audiences identified finding information quickly as the most important priority for them on the website.

Having engaging and entertaining editorial content is a “nice to have”.


1. Focus your research questions2. Choose your sample population3. Choose your incentive4. Design the survey5. Distribute the survey6. Analyse & communicate the results


6 Steps to Effective User Surveys

Questions?

Get more content tips at ContentStrategyInc.com

Find me on Twitter: @Kathy_CS_IncLinkedin: ca.linkedin.com/in/kathywagnercsSlideshare: Content_Strategy_Inc

gathering content insights from user surveys confab2014

Technology

survey questions

confabmn surveys

research questions t

incentive t

paris t

decisionmaking t

important t

results t