designing the next generation of search user experience - uxpa2015

© Design for Context

Duane Degler Principal [email protected] @ddegler

Lisa Ba.le Principal [email protected] @design4context

Designing the Next Genera/on of Search User Experience UXPA 2015 • June 24, 2015

@design4context NextGen Search #UXPA2015 2


Presenters

USER EXPERIENCE & USABILITY Understand user needs and tasks. Design applicaHons that work the way people expect them to.

Photo of addiConal presenter here (in

grayscale)

Lisa Ba.le

INFORMATION ARCHITECTURE / DESIGN Understand the models that make informaHon Hck, and help people get things done.

Duane Degler



The ubiquitous search box



Photo: 13628208 What can you do to influence the user’s experience with search?



Public Search Site Search Enterprise Search

DIFFERENT KINDS OF SEARCH FOR DIFFERENT USER NEEDS

What is a search experience?

@design4context NextGen Search #UXPA2015


Stages of seeking

Catalyst

Info Task Achieve in the search space

Goal Achieve in the world

Seek Type in the box

Evaluate Assess results

Review Assess content

Formulate Strategy Tactics

AcCvity

Use Apply what you gained

Keep Persistent resource

Outcome

REFINE

ITERATE







REFINE

User’s perspecCve “Mechanics” of search Trends for the future

Stages of seeking



http://en.wikipedia.org/wiki/Incandescent_light_bulb#mediaviewer/File:Edison_incandescent_lights.jpg

Users’ Mental Models for Search

The user’s mental models, moCvaCons and level of understanding affect their approach to search.

TesHng search has some nuances.

USERS’ PERSPECTIVE



SEARCH STYLE: Survey “ What are the

symptoms of arthri0s? ”

»  SaCsficing, "good enough" »  Usually single session »  Look for correlaCons between results to bolster confidence »  Focused on authoritaCve sources »  Preview to support decision-‐making

REFINE

Formulate Seek Evaluate Review



SEARCH STYLE: Targeted

"What is the maximum dosage for aspirin per day?"

»  Typically start with a known domain »  Clear, known expectaCon of outcome or nature of outcome »  High recogniCon when evaluaCng result set

REFINE


KEEP



“ I need to write a research paper on effec0veness of physical therapy vs. drugs for arthri0s treatment”

»  Longitudinal »  IteraCve, layered, evolving »  Search strategy is a criCcal component »  Inputs to search formulaCon must be acConable »  Outcomes must be acConable: capture, organize, save

REFINE


KEEP

SEARCH STYLE: Exploratory



SEARCH STYLE: Archival ”In what situa0ons has

methotrexate caused or contributed to pa0ent adverse events?"

»  Search has been comprehensive and complete »  Important to know you have found all that there is (within limit) »  Haven't missed criCcal findings »  Comprehensive search is itself a product; if I can't find it:

•  it likely doesn't exist, or •  It’s not relevant to the context that I have applied

REFINE


KEEP



SEARCH STYLE: RouCne "Keep me up-‐to-‐date on

hematological cancer involving Velcade as an agent"

»  Take a successful search strategy and give it an idenCty »  Focus on new and unique results »  Search parameters and filters are not necessarily related to a

specific result set

REFINE


KEEP



»  Agent for others »  Sharing »  Refinement might be mulC-‐session, iteraCve, not real-‐Cme »  FormulaCon task requires:

•  NegoCaCon to arCculate goals and scope •  TranslaCon

REFINE


KEEP

SEARCH STYLE: CollaboraCve

"What is the maximum dosage for aspirin per day?"

”Is this for an adult or a child?"

”A child."

” How many milligrams is your aspirin pill?"



Summary: Seeking Behaviors

Collabora/ve REFINE


KEEP

REFINE


KEEP

REFINE


KEEP

REFINE


KEEP

REFINE


KEEP

REFINE


Rou/ne

Archival

Exploratory

Targeted

Survey



●  Set the scene ● Let the user define scenarios whenever possible ● Be clear, together, about the context that you are tesCng for, so the person can get in that mental space

● Test in their environment whenever possible

•  Comfort

•  Secondary resources

●  IdenCfy expected outcomes ● Dig to learn what the person wants to do with informaCon

● Collect examples of work products / outputs they create now

ConsideraCons for tesCng search designs

16



●  It’s a thinking process ●  Test with real content – as much as you can get ● Use content that is meaningful / familiar ●  Encourage conversaCon – formaCve protocols encouraged

●  Iterate rouCnely and rapidly ● Mix your parCcipants – some regular, some new each Cme ● Work with real code in the back-‐end, lightweight interacCve prototypes on the front-‐end

•  The experience is tacCle and cogniCve

●  Include a longitudinal study ●  Search tools for large content environments and enterprise take Cme for a person to incorporate into a rouCne

●  ReflecCon is an important aspect of user feedback

ConsideraCons for tesCng search designs

17



Parsing

Indexing

Ranking

Filtering

Answering

MECHANICS

http://www.loc.gov/pictures/item/ny1273.photos.123483p/



The mechanics of search

Index

Query

Parser

Search Engine

Document

Unstructured Content

Models

Metadata, Terms, Thesaurus

Results & Facets List of items List of terms and counts

Results Page

Text, Image Voice,




Image

The mechanics of search . . . enhanced

Index

Query

Parser

Search Engine

Document Voice,

Models

Metadata, Taxonomy, Thesaurus, Ontology


Results Page

FUTURE

Text, ApplicaCon



Design ConsideraCons for Search

TRENDS

http://hubblesite.org/gallery/spacecraft/28/large_web



FORMULATE SECTION

Formulate Evaluate Review Seek Formulate



Deciding what to search for, and how CURRENT

What do I need to find? Where should I look? How is this different than what I searched for before? What words or phrases should I use to describe the subject?



Formula/ng aligns goals with concepts about informaCon

The Design of Browsing and Berrypicking Techniques for the Online Search Interface Marcia Bates, 1989, http://pages.gseis.ucla.edu/faculty/bates/berrypicking.html

Berry picking ê

Evolving understanding

How do we help users align their goals with the available informaCon space?



RecogniCon, not recall: Search suggesCons CURRENT

Consider how you retrieve and structure the terms you present.



Capture thinking process when it happens FUTURE

The increase in OA-related musculoskeletal pain especially in the growing elderly population has created heightened interest in effective treatments with better safety profiles. The Glucosamine/chondroitin Arthritis Intervention Trial (GAIT) was designed to examine the effects of the dietary supplements glucosamine and chondroitin sulfate given alone or in combination as compared to celecoxib or placebo on pain associated with osteoarthritis of the knee. The primary symptomatic outcome assessment of the GAIT study was after 24 weeks of randomized, double-blinded, placebo-controlled therapy.[6] An ancillary structural study describing the effects of the agents on radiographic joint space width loss for up to two years has also recently been reported.[7] To date, very few data have been reported for the long-term treatment of OA with any agent including the supplements and celecoxib studied here. Patients in the structural study continued to have safety and clinical efficacy assessments at all scheduled visits. This paper details the clinical efficacy and safety experience with these agents alone and in combination along with celecoxib as compared to placebo in patients from this subset of GAIT over 24 months offollow-up.

chondroitin sulfate celcoxib

radiographic joint space Add note, action Search on this now Tag for !Tag for search !

Prior search formulaCons and strategies are products

Search Phrase

Search Builder Assist:

Operator Toolbar Special Symbols Equations and Formulas

arthritis +4 and methotrexate

arthritis – synonyms and instances " Ankylosing spondylitis

þ Gonococcal arthritis " Gout " Juvenile rheumatoid arthritis

þ Psoriatic arthritis

þ Reactive arthritis (Reiter syndrome)

þ Rheumatoid arthritis (in adults) " Scleroderma " Systemic lupus erythematosus (SLE) + Add another



RouCne, automated search CURRENT

How will these searches evolve as a person’s interests and needs change? Think longitudinally.



Under the covers, using AI, this is about pa.ern-‐matching.

Seqng personalizaCon



Refine personalizaCon via richer models FUTURE



Refine personalizaCon via richer models FUTURE

Showing aspects of structure helps someone focus on their specific interests.




Image


Index

Query

Parser

Search Engine

Document Voice,

Models



Results Page

FUTURE

Profile Text,

SemanCc Processing

Save

User Model

Index/ Graph db

Formulate



● Help a person make needs and interests more explicit ● Capture what they are thinking before they start searching, in context of other tasks and applicaCons

●  Find streamlined ways to support refining and changing interests over Cme ● RouCne searches decay – refinement extends engagement

●  Surface underlying models ● ParCcularly for knowledge-‐intensive and longitudinal, mulC-‐session tasks

● Be humble – and create a humble experience

Design consideraCons Formulate



SEEK SECTION

Formulate Evaluate Review Seek



CURRENT Keywords, type ahead and scoping



CURRENT Mobile search relies heavily on type-‐ahead



CURRENT DisambiguaCon

Present informaCon that is recognizable and supports user decision-‐making.



Advanced search CURRENT

The common trend is to move detailed parameter refinement to Results page, as facet filters, to avoid “0 results”…



Parameter-‐driven “advanced” searches CURRENT

Parameters are the same (to the user, in the code) for the search and for facet management on the results page.

The OpCcal Society (OSA), h.ps://osapublishing.org



EnCty disambiguaCon from a model FUTURE



Parsing: Exposing thesaurus for user control

For very knowledgeable researchers, it builds trust when you are transparent and offer control.



Missouri populaCon

InterpretaCon of search phrases

41



What states surround Missouri?


42



What is the populaCon in the states surrounding Missouri?


43

Geographic bordering

relaConships

US State as focus

State (geographic)

Number of residents



Calculate difference

Specific Cme period for data

Geographic bordering

relaConships

US State as focus

State (geographic)

Number of residents

What is the change in populaCon in the states surrounding Missouri, between 2012-‐2014?


44

FUTURE

And this is sCll a simple quesCon, when compared with what researchers and analysts ask of their data today.



Smarter search boxes

(clinical decision support) (device or console or computer or apparatus) AND Q-22 «

OR OR OR SAME Q-17 »

“clinical decision” “decision support” “clinical decision support”

Phrase structures

support supported supporting supportive

Stemming Synonyms (thesauri)

Operator combinations Dynamic controls

FUTURE



Image search

46

FUTURE

Find examples of CultureCam, PixLogic

http://culturecam.eu

AlternaCve entry methods for large amounts of rich media should be explored.



Voice-‐acCvated search… a conversaCon FUTURE

Try building in comprehension check-‐points.



ProacCve search: Solving a real-‐world content problem

48

Use

r’s a

war

enes

s of

nee

d

Nature of content related to task

Prohibited Not available

Specific awareness of content and ability to articulate problem

Required Available

Awareness of problem but little ability to articulate

No awareness of need for information

“I don’t know

what I need to know”

“I know what I’m looking for-- get me to it!”

Identify via analyzing content and

term use

Identify via direct user

feedback

Identify via task

performance assessment

“I didn’t know I needed that”

Degler/Ba.le (2003). Can Topic Maps Describe Enterprise-‐Wide Context? h.p://www.designforcontext.com/publicaCons/dd-‐lb_topic-‐maps_context_extreme2003.pdf

“Just tell me what I

absolutely must know”




Image


Index

Query

Parser

Search Engine

Document Voice,

Models



Results Page

Speech Language Images

FUTURE

Profile

Context Parameters

Text,

SemanCc Processing

Terms, Tags

Interpreted meaning

NLP, Enrich, Auto-‐Tag


Semi-‐structured

Index/ Graph db

Seek



●  Simple is best . . . ● But no simpler that what supports a person to express what they need as precisely as they are able

● Allow someone to use examples wherever possible

● The trend is to move complexity to the results, because decisions are evaluated against real informaCon

●  Think iteraCvely . . . And longitudinally ●  In many cases, search is a mulC-‐step and mulC-‐session task (exploratory, archival, collaboraCve in parCcular)


Design consideraCons Seek



EVALUATE SECTION




Results page

Refinements

User choices Ac/ons View controls

Results

Meta-‐results

CURRENT

(Ranked Instances) (Facets)



Facets

53

CURRENT

Hierarchy Multi-select Incremental Disclosure Simple list

Scoping



Managing the evaluaCon process… CURRENT

These habits will remain – consider how to work with them.



Mobile results – answering direct quesCons

Shazam

CURRENT

Google Maps



Mobile results – could a conversaCon help?

iTriage

CURRENT

WebMD

How quickly do you go to individual results items?



Answers instead of “results” (“Things, not Strings” -‐ 2012) FUTURE



Graph traversing results display (DIVE)

58

http://dive.beeldengeluid.nl/

FUTURE



Exploratory: Blend results & topics (Intent Streams)

Salvatore Andolina, Khalil Klouche, Jaakko Peltonen, Mohammad Hoque, Tuukka Ruotsalo, Diogo Cabral, Arto Klami, Dorota Głowacka, Patrik Floréen, and Giulio Jacucci. 2015. IntentStreams: Smart Parallel Search Streams for Branching Exploratory Search. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI '15). ACM, New York, NY, USA, 300-305. DOI=10.1145/2678025.2701401 http://doi.acm.org/10.1145/2678025.2701401

FUTURE



VisualizaCons: User control is key FUTURE

Antti Kangasrääsiö, Dorota Glowacka, and Samuel Kaski. 2015. Improving Controllability and Predictability of Interactive Recommendation Interfaces for Exploratory Search. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI '15). ACM, New York, NY, USA, 247-251. DOI=10.1145/2678025.2701371 http://doi.acm.org/10.1145/2678025.2701371



Snippets

Snippets displayed and highlighted upon mouseover

Upon click, display snippet expansion

Transition display to single document view/layer, anchoring to the specific snippet in context

FUTURE



Expanded snippets Title Number Date Class Inventor Type Preview

Assisted Clinical Decision Support 20040091847 11/06/2003 434/353 Pandya, et al. Patent

Claim Text: assisted clinical decision support, the method comprising: identifying, with a processor, a plurality of associated terms from a medical ontology, the associated terms including associated drugs; generating... assisted clinical decision support, the system comprising: a memory operable to store a mining engine with a domain-knowledge base from associated terms in a medical ontology, the mining engine configured... assisted clinical decision support, the storage media comprising instructions for: creating a knowledge base from a medical ontology; and building a mining engine operable to search unstructured medical data... assisted clinical decision support, the method comprising: identifying a plurality of associated terms from a medical ontology; and training, with a processor, a model of relationships of the associated terms... assisted clinical decision support, the system comprising: a memory operable to store a probabilistic model having machine learned probabilities for relationships from a medical ontology, the machine learned... assisted clinical decision support, the method comprising: identifying, with a processor, a plurality of associated terms from a medical ontology; generating, with the processor, a domain-knowledge base from...

89 pages | Cover page | 12 drawings

#$

forward-backward citations

X

assisted clinical decision support, the system comprising: a memory operable to store a mining engine with a domain-knowledge base from associated terms in a medical ontology, the mining engine configured

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip

Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

#$

434/353: Specifications Pg XX

FUTURE

Help evaluaCon at different informaCon

levels.




Image


Index

Query

Parser

Search Engine

Document Voice,

Models



Results Page

FUTURE

Structured Data Panels Graph

Model

Images & MulCmedia

Profile

Context Parameters

Text,

SemanCc Processing

User Model

Terms, Tags

Interpreted meaning



Semi-‐structured

Index/ Graph db

Evaluate



●  The results page is a query interface ● Provide sufficient signposCng and user control

● Allow for quick skimming and, for knowledge-‐intensive tasks, incrementally deeper dives

● Consider capabiliCes for exploratory behaviors

● Design to suit the task and mental model ● Consider “how much is sufficient”

● Establish trust in the results – algorithms need transparency


Design consideraCons Evaluate



REVIEW SECTION




Snippets

Snippets displayed and highlighted upon mouseover

Upon click, display snippet expansion

Transition display to single document view/layer, anchoring to the specific snippet in context

FUTURE



People organize their thinking in different ways at different Cmes.

Sensemaking and organizing FUTURE

Marti A. Hearst and Duane Degler. 2013. Sewing the Seams of Sensemaking: A Practical Interface for Tagging and Organizing Saved Search Results. In Proceedings of the Symposium on Human-Computer Interaction and Information Retrieval (HCIR '13). ACM, New York, NY, USA, , Article 4 , 10 pages. DOI=10.1145/2528394.2528398 http://doi.acm.org/10.1145/2528394.2528398



Gathering relevant concepts and resources

haptic interactions CONCEPT

synonym1, synonym2, synonym3, synonym4, synonym5

material, palpable, physical, solid, tactual, tangible SEARCH TERMS

Snippets All Search Terms Documents Notes Sketches

FUTURE



The World Bank invests about one-‐quarter of its budget for country services in knowledge products… About 49 percent of the World Bank’s policy reports, which are published Economic and Sector Work or Technical Assistance reports, have the stated objecCve of informing the public debate or influencing the development community…

About 13 percent of policy reports were downloaded at least 250 Cmes while more than 31 percent of policy reports are never downloaded. Almost 87 percent of policy reports were never cited.

Granular content instead of “documents”

69

FUTURE

Doemeland, Doerte; Trevino, James. 2014. Which World Bank reports are widely read ? Policy Research working paper ; no. WPS 6851. Washington, DC: World Bank Group. http://documents.worldbank.org/

curated/en/2014/05/19456376/world-bank-reports-widely-read-world-bank-reports-widely-read



Craig S. Tashman and W. Keith Edwards. 2011. LiquidText: a flexible, multitouch environment to support active reading. In

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA,

3285-3294. DOI=10.1145/1978942.1979430 http://doi.acm.org/10.1145/1978942.1979430

Summarize & compress (LiquidText) FUTURE



FUTURE ReflecCng task, user and content relevance

71



Goal: Embed in applicaCon

72

FUTURE




Image


Index

Query

Parser

Search Engine

Document Voice,

Models



Fragment

Results Page

ApplicaCon Embedded Results Panels

FUTURE

Context Parameters

Text,

Graph Model

Terms, Tags

Interpreted meaning



Semi-‐structured

Index/ Graph db

Review



● Review is a learning process ●  It is iteraCve and longitudinal, so help a person understand what has previously been retrieved, viewed, saved, used

● Documents are part of the search experience

●  The take-‐away is the informaCon, not the document / desCnaCon ● Design should facilitate the next acCons, e.g. synthesis, sharing, ciCng, learning, doing


Design consideraCons Review



Take-‐aways

http://en.wikipedia.org/wiki/File:Libreria_San_Gines.jpg



Image


Index

Query

Parser

Search Engine

Document Voice,

Context Parameters

Models



Fragment

Results Page

Structured Data Panels

Speech Language Images

ApplicaCon Embedded Results Panels

FUTURE

Graph Model

Images & MulCmedia

Profile Text,

SemanCc Processing

Save

User Model

Terms, Tags

Interpreted meaning



Semi-‐structured

Index/ Graph db



User

System

•  Think •  Look at what you did before

•  Gather terms and ideas

•  Express keywords •  Pick from lists •  Enter parameters •  Speak sentences •  Change what you expressed (refine)

•  Review lists •  Read small amounts of informaCon

•  Apply filter parameters (refine)

•  Make selecCons (short-‐term keep)

•  Look at what you found •  Make decisions about what else to seek (refine)

•  Select and discard •  Take follow-‐on acCon •  Share •  Keep

•  Capture terms in notes •  Extract terms from documents

•  Provide reviews of previous materials

•  Provide examples of what might be available (proacCve)

•  Ingest and store content in a structured way so it can be searched

•  Incorporate what other people have searched for previously.

•  Provide example phrases (type ahead)

•  Parse language to try and understand intent

•  Gather context signals •  Apply thesaurus/addiConal model terms

•  Send requests to other services

•  IdenCfy appropriate items to return

•  Rank the returned items

•  Present the structured data

•  Present the returned items

•  Present the model of vocabulary for filtering

•  Present summaries and abstracCons to profile what is found

•  Analyze selected informaCon

•  Provide a place to store things

•  Provide a place to organize things

•  Provide a way to manage what you have

•  Provide a way to take what you have and use it elsewhere

•  Profile the user’s process when they did that






Duane Degler Principal [email protected] @ddegler

Lisa Ba.le Principal [email protected] @design4context

Designing the Next Genera/on of Search User Experience UXPA 2015 • June 24, 2015

designing the next generation of search user experience - uxpa2015

Design

design4contextnextgen

teshng search

uxpa20159 search style

uxpa201510 search style

uxpa201512 search style

uxpa201513 search style

search space goal

results review