ethnography in software design updated for big design 2015

Ethnography in Software Design An Anthropologist’s Point of View Big Design - Dallas, September 2015

: https://www.flickr.com/photos/101187156@N03/14366224997/

@Kel_Moran #bigd15

https://www.flickr.com/photos/101187156@N03/14366224997/

Kelly Moran Lead Design Researcher

@Kel_Moran #bigd15

MS Applied Anthropology University of North Texas

BA Anthropology California State U. Northridge

US Peace Corps Uzbekistan

Study Abroad Saudi Arabia

In

the

Wild

Worked with:

“Uncover user needs, Design great solutions,

and build out solutions to launch.”

Dist

ill M

eani

ng fr

om O

bser

vatio

n

Build

the

Back

log

CONTACT BUILDING & EVOLVINGFOCUSED INNOVATION

Only a subset of these activities will be appropriate for any given project.

Qualitative Research

Quantitative Research

Affinity DiagrammingConstruct themes from qualitative data.

Analysis & Synthesis Opportunities

Ideation & Iteration

Validation & Evaluation

Approach Planning Design Research Experience Strategy & Strategic Ideation PlanningDefinition

Iden

tify

Chal

leng

es in

Con

text

Set t

he S

tage

REVEALING REALITY

Foundational Analysis

Heuristic EvaluationIdentify inital breakdowns and opportunities

Digital Marketing SWOT AnalysisIdentify strengths, weaknesses, opportunities & theats.

Stakeholder InterviewsUnderstand staheholders’ business goals & strategy.

Technical Organization Capability AanalysisAnalyze existing skills and toolsets.

Scenario-based System WalkthroughsDemo of existing solution.

Metrics EvaluationEstablish quantitative baseline of engagement & conversion data.

Content InventoryCatalog the content of the site or application.

Competitive AnalysisEvaluate competitors and comparables on specific axes.

User Experience

Data

Technology

Enterprise Architecture Capability AssesmentReview existing enterprise technology infrastructure.

Solution(s) Architecture AssessmentReview existing application(s) architecture.

RFP RequestDeliver proposal to defined scope document and existing requirements.

Project ApproachAsses a possible project and plan high level approach.

Align & Assess WorkshopAssess readiness across core capabilities.

Backlog GroomingReprioritize backlog, add new stories.

Zero Feature ReleaseDemonstrate CI, automated testing, core solution setup.

High Level Technical ArchitectureDescribe high level architecture, including packaged components.

Development Infrastructure ConfigurationContinuous integration setup.

Technical Package Identification & EvaluationPerform product evaluations for package solution components.

Architecture Spikes & Proofs of ConceptProve candidate architectures via top bottom spikes.

Existing Research ReviewMarket research, website feedback, corporate strategy, etc.

Brin

g th

e So

lutio

n in

to F

ocus

Research PlanDesign activities to meet research goals.

Contextual InquiriesObserve & document user in context & environment.

User Journals & DiariesUser document their experiences over time.

Participatory DesignCo-creation explorations with users.

Card SortExplore users’ mental models for content and labeling.

SurveysSolicit structured feedback from users.

User Workflow ModelingVisually document workflows & work systems.

Ideation WorkshopsImmerse stakeholders in data and brainstorm opportunities.

Persona DevelopmentCreate customer types to document observed behaviors and values.

Consolidated Workflow DiagramAggregate individual user workflows into one diagram.

Current Journey MapVisualize the user’s perspective of the current experience.

Quantitative Data VisualizationPresent quantitative data visually.

Opportunities Generation & EvaluationOpportunities & prototype choice.

Opportunities MatrixPrioritize in three dimensions, including user experience impact.

Marketing OpportunitiesStrategic planning of owned, earned & paid online tactics.

Engagement PlanDevelop the strategic and tactical plan to achieve the client’s goal.

Experience-Driven RoadmapPlan how great UX can be achieved through the design.

Design PrincipalsArticulate design principals to guide the design and development process.

Requirements & User Stories DefinitionWrite user stories based on detailed user scenarios.

Application & Navigation FrameworkValidated navigation and framework.

Wireframed Key WorkflowsValidated wireframes of key workflows.

Visual Design LanguageVisual design language defined.

Information ArchitectureMap the product from the users’ point of view.

StoryboardsIllustrate graphical representations of scenarios.

Qualitative & Quantitative Data SynthesisAnalyze validation data..

User ScenariosWrite detailed narratives for user experience flows.

Future Journey MapVisualize the user’s future, improved experience.

Concept ValidationValidate design prototypes through user feedback.

Application & Navigation Framework ConceptsCreate models for the navigation & framework of the application or site.

Workflow ConceptsDraw high-level wireframes for key workflows.

Visual ExplorationExplore different visual treatments and styles of the application or site.

KANO Feature PrioritizationPrioritize features with users through KANO analysis.

we make software make sense.TM

“Design Research directs the strategy & design for

our software projects”

WHAT DOES ANTHROPOLOGY

LOOK LIKE?

http://michaelvhurley.com/2013/01/04/my-worthless-degree/

Anthropologist?

CSI Vegas?

“Anthropology demands the open-mindedness with which one

must look and listen, record in astonishment, and wonder that which one would not have been

able to guess" – Margaret Mead

@Kel_Moran

@Kel_Moran

WHAT IS ETHNOGRAPHY?

10 things to know

1

Ethnography is the descriptive study of people and cultures

@Kel_Moran

2

Ethnographic Research is…

An approach not a specific method

@Kel_Moran

3

Ethnographic Research…

Favors qualitative over quantitative - the ethnographer

seeks to provide “thick” description

@Kel_Moran

;-}

;-/

;-)

Thick Description: Clifford Geertz @Kel_Moran

4


Conducted in context, typically over an extended period of time, and holistic, seeking the wider

picture@Kel_Moran

To understand the pen, you must understand the paper.

@Kel_Moran

5


Systematically conducted but responsive to emerging

trends and themes.

@Kel_Moran

6


Utilizes key informants who can act as guides and help provide

access to the community

(Also offer “reality checks”)@Kel_Moran

7


Seeks out the insider (emic) perspective and layers in outsider

(etic) insights and interpretations

@Kel_Moran

8


Generative - done to discover new information, not to test

existing hypotheses

@Kel_Moran

“Never theorize before you have data. Invariably you end up

twisting facts to suit theories instead of theories to suit facts.”

- Sherlock Holmes (Arthur Conan Doyle)

@Kel_Moran

9


Seeks to tease out the implicit, not typically stated, features of a group

@Kel_Moran

“Making the familiar strange and the strange familiar”

@Kel_Moran http://www.gizmocrazed.com/2012/04/top-10-theories-on-how-the-world-will-end/

http://www.gizmocrazed.com/2012/04/top-10-theories-on-how-the-world-will-end/

The Nacirema

@Kel_Moran The Nacirema: Horace Miner

The NaciremaAmerican

@Kel_Moran The Nacirema: Horace Miner


Is inclusive, typically including a participatory component

10

@Kel_Moran

Texas BONUS! In 1919 James Edwin

Pearce, chairman of the Department of

Institutional History at the University of Texas,

changed its name to the Department of

Anthropology - making it one of the first such

departments of its type.

@Kel_Moran

WHAT IS DESIGN ETHNOGRAPHY?

3 things to know

@Kel_Moran

1Design Ethnography…

Is a way “…to increase the success probability of a new product or service or, more appropriately, to reduce the

probability of failure specifically due to a lack of understanding the basic

behaviors and frameworks of consumers.”

Salvator, Tony; Genevieve Bell; and Ken Anderson (1999) “Design Ethnography,” Design Management Journal (pp. 35-41). p.37

1Design Ethnography…

Is a way “…to increase the success probability of a new product or service or, more appropriately, to reduce the

probability of failure specifically due to a lack of understanding the basic

behaviors and frameworks of consumers.”

Salvator, Tony; Genevieve Bell; and Ken Anderson (1999) “Design Ethnography,” Design Management Journal (pp. 35-41). p.37 @Kel_Moran

75-95% of

new products fail

2

Avoids an over-reliance on self-reported data

(what they say is not always what they do)

@Kel_Moran

What does “green” mean?

3

Qualitative research, done in the context (environment) of the intended users, seeking to discover and understand their problems from their viewpoint,

with the designer’s viewpoint used to ideate potential solutions.

@Kel_Moran

INTEGRATINGIN-CONTEXT RESEARCH

Discovery Phase

• Designer pairs with a Researcher • Researcher leads with a background in the social

sciences • Designer assists

Researcher Designer

@Kel_Moran

Design Phase

• Roles switch, and the Researcher assists the Designer

ResearcherDesigner

@Kel_Moran

After Detailed Design • Designer typically is embedded into the Development

Phase • Researcher comes back on board for user testing

DesignerDeveloper (x N)

Researcher @Kel_Moran

Provides continuity and keeps the user’s voice present

User Focused Innovation

Discovery Design Build

@Kel_Moran

CASE STUDY

ENTERPRISE SOFTWARE ACCOUNT RECONCILIATIONS

The Client and the Product

Vendor of accounting software • Customer feedback of “too many clicks” and “hard to use” • Sales were lagging

Used in organizations with large, multi-functional accounting departments • Needed to be customizable • Should fit within a suite of other enterprise products

@Kel_Moran

Research Basics

Who • Nineteen users at six user sites, plus three users inside the client

company

Where • Seven locations across four states • 2-4 users at each location

@Kel_Moran

From 22 users: Around 800 unique

insights and observations were recorded

@Kel_Moran

IN-CONTEXT OBSERVATION

@Kel_Moran

“We have everything set up for you in a

conference room.”

@Kel_Moran

What do you see?

@Kel_Moran

Contextual Learnings• 5 out of 6 of the client-user groups observed used two monitors

• adding machines still in use

• typically in either a cube or an open workspace

• paper everywhere - post its galore!

• User observed walking to a locked room (with a broken ankle) to look up reference numbers

@Kel_Moran

CONTEXTUAL LESSONS FROM USERS

@Kel_Moran

Contextual Learnings

With a lot on their minds, Excel is king • High cognitive load • High use of Excel quick keys • Importing data of multiple types into Excel as images • Highlighting and/or circling important data

http://icons8.com@Kel_Moran

http://icons8.com

“It’s better than having it rejected back.”

@Kel_Moran

Contextual Learnings (continued)

Attachments and cover sheets • Manager preference for a single attachment • Printing out documents from multiple sources to scan

them into one attachment • Making a Custom “cover sheet” in Excel to sum up the

work

@Kel_Moran

“I look at this as basically

just a holding station.”

@Kel_Moran


“Roles” as defined in the software did not match the work-based roles of the users

• In several locations a “reconciler” also held the role of “administrator,” but juggling these in the software was cumbersome

• The administrator role could become overwhelming

Assumed User Role with Linear Workflow

Observed Interrupted User/Admin Workflow

StartAdministrative Work

Start EndUser Work

EndUser WorkUser Work

Interruption

@Kel_Moran

“I have to change gear… to move to that other role.”

@Kel_Moran

“Soon I won’t have any accounting work. It will all be [administrative] work.”

@Kel_Moran


The problem goes beyond the system • Lack of technical support • The “real work” was done outside the system

@Kel_Moran

Now what do we do with all that data?

@Kel_Moran

Synthesizing the Data - Major Themes1. Learning, training, &

support 2. Don’t make it hard on us 3. We know who we are and

what we need 4. We need an agile, smart

workflow and tracking system

5. Don’t add to our cognitive load

6. Be our partner 7. We have a lot going on

besides what you do for us 8. Collaboration, teamwork,

communication, & working with others

@Kel_Moran

Users viewed the system as supplementary, and part of a larger process.

Organize the system’s workstream to match the user’s view.

Cover sheets were created and standardized by each accounting group.

Renaming the “account home page” the “cover sheet” and emphasizing its similarities to the Excel-made coversheets.

High use of Excel.

Bringing in more Excel patterns.

Observations Lead to Design Ideas

@Kel_Moran

Observations Lead to Design Ideas (continued)

Uploading and viewing attachments was a common problem; and it didn’t match the rest of the experience.

Making both uploading and viewing attachments easier and more consistent with the rest of the UI.

Inconsistent UI was jarring to navigate.

Overall more consistent UI.

Users had to scan the page to find the most relevant information on each screen.

Putting the information the users needed most to keep in mind in a prominent, and persistent, place on every screen.

@Kel_Moran

Design Ideas get Prioritized

TRANSITIONING INTO DESIGN

@Kel_Moran

VALIDATION TESTING

@Kel_Moran

Three Pages Tested Two Versions Each

• Dashboard • List Page - results of search

• Refined filtering • Account Details - account home; i.e. cover sheet

@Kel_Moran

(revised and re-tested for a total of three testing rounds)

User-Led Validation and Changes

• Most users don’t need the 6-currency view panel • Changed the default to 3 currencies

• Accountants need to be sure their entries save • Ties back to the generative research • Added a “save” confirmation button instead of auto-saving

• Attaching files to the cover page was validated as preferable to using a separate attachments page

@Kel_Moran

BRINGING ETHNOGRAPHY TO YOUR PROJECTS

Start Small - Start Anywhere!Does not need to cover 7 regions across a continent (seeing a few users makes a difference)

Get into the environment (try Skype if needed)

Don’t go in with solutions in mind (don’t be too sure you know the problems already either)

@Kel_Moran

Obtain Informed Consent

@Kel_Moran

How to ObserveThe physical environment - open? cramped? hot? cold?

Lighting, noise - pleasant? distracting? manipulated by the user?

People - who interacts with whom?

Artifacts - equipment, paper notes, binders…

Document it all - notes, yes, but also photographs and audio/video if permitted.

@Kel_Moran

How to Ask Questions

You are not an expert in their work/play - even if you think you are

Rephrase what they say and ask if you got it right - let them correct you even if you know you did

Avoid leading questions - ask them to describe instead

Take note of their ideas and ask “What problem does this solve?” - they’re not the experts at finding solutions, but they tried their best

@Kel_Moran

Honor the Idea of Reciprocity

Your users (research participants) are giving you something, so be sure to give something back.

@Kel_Moran

ethnography in software design *updated for big design 2015*

Design

ethnography in software design updated for big design 2015