taxonomy & ontology impact on search infrastructure john r. mcgrath sr. director, fast search...

11
Taxonomy & Ontology Impact Taxonomy & Ontology Impact on Search Infrastructure on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

Upload: emily-wagner

Post on 27-Mar-2015

214 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

Taxonomy & Ontology ImpactTaxonomy & Ontology Impacton Search Infrastructureon Search Infrastructure

John R. McGrath

Sr. Director, Fast Search & Transfer

Page 2: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

2

Mar

ket

Po

ten

tial

Search InnovationConventional Innovative

Ma

ture

Ma

rke

tG

row

th M

ark

et

General Site Search, Intranet

eTailing

BusinessIntelligence

EnterpriseeCommerce

Premium Content Delivery

InformationManagement

KnowledgeManagement

Complianceand ContentGovernance

Detection, Surveillance and Enforcement

Predictive Search• alerting through

business rules• tracking and monitoring• high order analytics• pattern matching

Next Generation Search• 360º View of Enterprise• Hyper-personalization• Contextual synergy• Technology transparency• Central governance

Conventional Search• search bar w/ results• fixed relevancy model• static navigation• few data sources

Advanced Search• tunable relevancy and navigation• wide array of sources (structured, unstructured)• static navigation

Extended Search• Extended platform (desktop, mobile)• Intelligent use of context (Web, geospatial)• rich media integration

The Real Search Solution SpaceEvolving more powerful capability

Page 3: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

3

Improving Search through Contextual AnalysisThe Importance of Context in Search

Usage PatternsQueriesInterest ProfilesLocation

USER CONTEXT

APPLICATION CONTEXTINFORMATION CONTEXTWhat does the data indicate is most important?

What matters most to the user?

What matters most to the business or organization providing the data?

Statistics

MetadataOrganization

Business RulesEditorial ControlProgram Control

Page 4: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

4

Improving Search through Conceptual AnalysisEmbedded Application Semantics

FAST Answers

Extreme precision applications:– Self-service, NLP, Mobile, Compliance– Information discovery, Intelligence– Web 2.0 – The semantic web

Rich Media

The Adaptive Information Warehouse

– Scene level discovery– Podcasts: Extreme precision access

– Linguistic cleansing– On-the-fly fact mining– 10-200 X query speed-ups– Low latency access to extreme volumes

When was D-day?

– Visualize implicit facts– Visualize uncertainty– Use embedded semantics:

–Ex: Patent claims–Ex: Blogs

Page 5: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

5

Applying Ontology to the Search Architecture

COLLECTIONS CONNECTORS REFINEMENT SEARCH & ALERT ENGINES PROCESSING SEARCH PROFILE USER

SEARCH & ADMINISTRATION MANAGEMENT

ALERT

CONTENTREFINEMENT

QUERY PROCESSING

RESULT PROCESSING

SEARCH

ST

RU

CT

UR

ED

DA

TA

RIC

H M

ED

IAU

NS

TR

UC

TU

RE

DD

AT

A

SEARCH

App. LogicTagging

ExtractionClassificationRelationship

PersonalizationTerm ExpansionQuery Control

Source SelectionRefinement Logic

PresentationSecurity

App logicNavigation

Page 6: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

6

Primary applications today in search

Classify documents – facilitate simple, keyword-based retrieval

Provide a common language, or thesaurus – offer terms to refine a search from a consistent, controlled vocabulary

Create browse-able directories – facilitate rapid navigation through defined hierarchies of information

Promote meaningful clustering – establish ‘fixed points’ for clustering results

Generate pick-list elements – select or combine terms to limit/define your search domain

Expedite query refinement – refine/exclude on similarly tagged items

Page 7: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

7

FAST Relevancy Framework

Business Rules

User Profiles

Core Algorithmic Model

Application Model•Sorting •Navigation•Feedback

Accessible to… Control Mechanisms

End Users

Business Managers

•Alert Parameters•Page “boosting”

Administrator •Rank Profile•Concept Security

Developer •Dynamic Algorithm “weights”

Levels o

f co

ntro

l

Multiple levels of control

Page 8: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

8

Human Factors Considerations

Limited user capacitance• Most business users do not navigate

deeper than 4 levels in a taxonomy• More than 10 choices/nodes per level

impacts willingness to move deeper, to next level

Multiple perceptions of value• Provides navigation for discovery

(40%)• Organizes disparate info (18%)• Structures KM repository (16%)• Automates classification & alerting

(14%)• Enhances searching(12%)

Distribution of expected benefits• Increased productivity (21%)• Reduced search time ( 20% )• Increased knowledge sharing (18%)• Shortened time to decision (16%)• Improved collaboration (13%)• Discover new opportunities (10%) Source : Delphi Group 2004 Survey – 300 respondents

Taxonomy Levels Navigated

17%

68%

11%

4%

0% 50% 100%

Source:Delphi Group 2004 Survey

>65-63-41-2

Page 9: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

9

Observed Trends

Taxonomies and ontologies are expense to develop and maintain.

• Published works and services• Social network product• Automatic generation

Page 10: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

10

Observed Trends

Primary application is “smart” navigation.

• Simple knowledge bases• Supervised clustering

Page 11: Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

11

Near Future?

Application of Knowledge bases is increasing to support advanced features.

• Extraction and association• Relationship analysis• Advanced personalization