exploring semantic means

© 2009 Endeca Technologies, Inc. All rights reserved.

exploring semantic means

Daniel TunkelangChief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.2

about me

• 1988 – 1992

• 1993 – 1998

• 1999 -

© 2009 Endeca Technologies, Inc. All rights reserved.3

overview

a means, not an end

exploratory search (HCIR)

searching for meaning

© 2009 Endeca Technologies, Inc. All rights reserved.4

do you like our owl?

© 2009 Endeca Technologies, Inc. All rights reserved.5

a means, not an end

The Semantic Web will improve the connections between knowledge on the web and software.

© 2009 Endeca Technologies, Inc. All rights reserved.6

so much religion, so little time

Skeptical reactions

• Practical feasibility

• An unrealized idea

• Censorship and privacy

• Doubling output formats

• Need

vs.

© 2009 Endeca Technologies, Inc. All rights reserved.7

what semantic means to (people who pay) me

using linguistic and statistical approaches to derive meaning from unstructured text

using semantic web approaches to represent meaning in content and query structure

© 2009 Endeca Technologies, Inc. All rights reserved.8

exploratory search

what i really care about

what you should care about too

especially if you see more toinformation seeking than google

© 2009 Endeca Technologies, Inc. All rights reserved.9

what would google do?

information Need query select from results

rank using IR model

USER:

SYSTEM:tf-idf PageRank

© 2009 Endeca Technologies, Inc. All rights reserved.10

assumptions of relevance-centric approach

• self-awareness

• self-expression

• model knows best

• answer is a document

• one-shot query

© 2009 Endeca Technologies, Inc. All rights reserved.11

human-computer information retrieval

• don’t just guess the user’s intent– optimize communication

• increase user responsibility and control– require and reward human intellectual effort

“Toward Human-Computer Information Retrieval”

Gary Marchionini

© 2009 Endeca Technologies, Inc. All rights reserved.12

let’s make this concrete

• Colleague:

Hey Daniel! You should check out what this guy Steve Pollitt’s been researching. Sounds right up your alley.

• Daniel:

Sure thing, I’ll look into it.

© 2009 Endeca Technologies, Inc. All rights reserved.13

google him!

© 2009 Endeca Technologies, Inc. All rights reserved.14

google scholar him?

© 2009 Endeca Technologies, Inc. All rights reserved.15

hcir-inspired interface

© 2009 Endeca Technologies, Inc. All rights reserved.16

tags provide summarization and guidance

© 2009 Endeca Technologies, Inc. All rights reserved.17

my information need evolves as i learn

© 2009 Endeca Technologies, Inc. All rights reserved.18

how did we do that?

• wisdom of crowds

• corpus analysis

• scope the problem

• target exploratory search

© 2009 Endeca Technologies, Inc. All rights reserved.19

if you prefer sports to computer science

© 2009 Endeca Technologies, Inc. All rights reserved.20

a more current slide…

© 2009 Endeca Technologies, Inc. All rights reserved.21

hcir can get fancy

© 2009 Endeca Technologies, Inc. All rights reserved.22

and there’s query structure side of semantics

© 2009 Endeca Technologies, Inc. All rights reserved.23

a more current example

All reviewers Reviewers Under 18

© 2009 Endeca Technologies, Inc. All rights reserved.24

one last example

“…As an example, I may be interested in the tensions between government and newly nationalized banks over the bonus culture that taxpayers are now funding (a hot topic in the UK currently)…”

--a commenter at The Noisy Channel, in a discussion of exploratory search for news

© 2009 Endeca Technologies, Inc. All rights reserved.25

still in private beta…

© 2009 Endeca Technologies, Inc. All rights reserved.26

following the information scent

© 2009 Endeca Technologies, Inc. All rights reserved.27

to explore the news

© 2009 Endeca Technologies, Inc. All rights reserved.28

in closing

let’s not quibble over protocols

exploration is the killer app for semantic tech

it’s real, and it’s fabulous!

© 2009 Endeca Technologies, Inc. All rights reserved.29

thank you

communication 1.0email: dt@endeca.com

communication 2.0blog: http://thenoisychannel.com

twitter: http://twitter.com/dtunkelang