Saurabh Sahni

Developer, hacker, evangelist

Build your Own Search Service

Yahoo! HackU IIIT Bangalore | 5th December 2009

Outline

•  About BOSS API – What?

– Why? – Features

•  What can be built with the BOSS API? •  How to use it

– BOSS API – Code example – BOSS Mashup framework

About BOSS API

What?

•  Open Yahoo’s core search features via web services to let 3rd parties revolutionize Search

http://developer.yahoo.com/search/boss

Usage

Opening the search technology stack

CRAWL

EXTRACT

SPAM <-> Gold

Analyze

Index

Rank Assist

Index

Web Map

Retrieve

Usage

Opening the search technology stack

CRAWL

EXTRACT

SPAM <-> Gold

Analyze

Index

Rank Assist

Index

Web Map

Retrieve

WEB API

Your App here

Why?

•  Removes entry barriers

•  Asset to Innovate –  Develop new relevance models –  Change presentation style

•  Search anywhere –  Improve Vertical Quality with Web comprehensiveness

BOSS API features

• No branding or attribution

• Ability to change presentation style • Ability to re-order results and blend-in additional

content

• Access to multiple verticals (web search, image, news)

• Keyword suggestions, spell checks • Semantic data, in-links, abstracts • Ability to monetize

What can be built with the BOSS API?

middlespot.com

10 

Inquisitor: Browser Extenstion

11 

Cluuz: http://cluuz.com

Keyword finder - http://keywordfinder.org/

askBOSS: http://ask-boss.appspot.com/

askBOSS: http://ask-boss.appspot.com/

askBOSS: http://ask-boss.appspot.com/

Webmeme: http://www.webmeme.in

17 

How to use it?

Get Started

•  Register for an application id http://developer.yahoo.com/wsregapp/

Get Started…

•  Documentation http://developer.yahoo.com/search/boss/boss_guide/

•  Code samples: Javascript, PHP and Python http://www.saurabhsahni.com/boss-examples.zip

BOSS Search API Interface

•  {query}: term to look for (url-encoded) •  {vert} := {web, news, images, spelling}

•  @ required – appid

•  @ optional – start, count, lang, region, format, callback,

sites, view

http://boss.yahooapis.com/ysearch/{vert}/v1/{query}

BOSS API

Searching Slumdog

Millionaire

(Source: http://en.wikipedia.org/wiki/File:Slumdog_Millionaire_poster.jpg)

BOSS API

•  Search for slumdog millionaire: –  http://boss.yahooapis.com/ysearch/web/v1/slumdog+millionaire?appid=xyz&format=xml

BOSS API: XML response

h"p://boss.yahooapis.com/ysearch/web/v1/slumdog+millionaire?appid=xyz&format=xml 

Site Restrict Search

•  Search for slumdog millionaire on selected movie sites – Add param

sites=indiatimes.com,movies.yahoo.com,imdb.com

–  http://boss.yahooapis.com/ysearch/web/v1/slumdog

+millionaire?appid=xyz&sites=indiatimes.com%2Cmovies.yahoo.com&format=xml

http://boss.yahooapis.com/ysearch/web/v1/slumdog+millionaire? appid=xyz&sites=indiatimes.com%2Cmovies.yahoo.com&format=xml

Search images

•  http://boss.yahooapis.com/ysearch/images/v1/slumdog +millionaire?dimensions=large

http://boss.yahooapis.com/ysearch/images/v1/ slumdog +millionaire

Search News

•  http://boss.yahooapis.com/ysearch/news/v1/slumdog +millionaire?age=15d

http://boss.yahooapis.com/ysearch/news/v1/ slumdog + millionaire?age=15d

Code Example Movie Search

Movie Search Code Example

http://www.saurabhsahni.com/boss-examples.zip

More with BOSS API

Related keywords

Add parameter view=keyterms –  http://boss.yahooapis.com/ysearch/web/v1/slumdog

+millionaire?appid=xyz&view=keyterms&format=xml

http://boss.yahooapis.com/ysearch/web/v1/slumdog +millionaire appid=xyz&view=keyterms&format=xml

Semantic Data

•  Access structured data acquired through SearchMonkey

Semantic Data

view=searchmonkey_feed view=searchmonkey_rdf

http://developer.yahoo.com/search/boss/stuctureddata.html

http://boss.yahooapis.com/ysearch/web/v1/slumdog +millionaire?appid=xyz& view=searchmonkey_feed&format=xml

Long abstracts

•  Add parameter abstract=long – get up to 300 characters instead of 130

http://boss.yahooapis.com/ysearch/spelling/v1/milionare?format=xml

Response

Spell Check 

Site Explorer

•  Get page inlinks –  http://boss.yahooapis.com/ysearch/se_inlink/v1/

{URL}?appid={APPID}

•  Page data: collection of subpages in a domain –  http://boss.yahooapis.com/ysearch/se_pagedata/v1/

{URL}?appid={APPID}

BOSS Mashup Framework

•  Python (v2.5+) library

•  BOSS Search SDK plus …

•  SQL for remixing arbitrary XML/JSON sources

http://developer.yahoo.com/search/boss/mashup.html

BMF + Google App Engine

•  Enhanced version of BMF to GAE platform

•  http://zooie.wordpress.com/2008/08/04/yahoo-boss-google-app-engine-integrated/

•  Enables quick deployment of BOSS applications online

More BOSS Implementations

•  http://mashable.com/boss/ •  http://delicious.com/tag/bossmashup

•  Add yours by tagging it with “bossmashup” on

Del.icio.us!

One more thing… One more thing…

BOSS in Academic Research

•  The biggest dataset available on web •  Very useful for Web-mining research

experiments – Natural language processing – Semantic extraction

– Related keywords – Similarity detection – Clustering algorithms – Spelling corrections

Learn more at: http://developer.yahoo.com/search/boss/

Slides at: http://www.slideshare.net/saurabhsahni/

Questions?

Appendix

BOSS Mashup Framework simplifies aggregating and presenting multiple data sources

Search UI Templates in BOSS Mashup Framework

BMF Features

•  select, group, sort, union, joins, udfs, where •  Text normalization and duplicate removal

•  All-in-memory storage and retrieval operations

•  Ability to join lists of tables via an arbitrary predicate function (map-like)

•  Search UI template framework •  Single search function provides total access to

BOSS REST API