Build your Own Search Service

Fred Meng Yahoo! Search Engineer

Outlines •  Search engines using BOSS •  Big milestones for BOSS •  What else can be built with BOSS •  BOSS API

– What – Why – Features

•  How to use it – BOSS API – BOSS Mashup framework

Search engines using

middlespot - http://middlespot.com

hakia - http://hakia.com

hakia - http://hakia.com

hakia - http://hakia.com

Tartin3 - http://www.tartin3.com

Vertical/niche search products

askBoss - http://ask-boss.appspot.com

askBoss - http://ask-boss.appspot.com

askBoss - http://ask-boss.appspot.com

askBoss - http://ask-boss.appspot.com

OneRiot - http://www.oneriot.com

New ranking models

tweetnews - http://tweetnews.appspot.com

New ranking models

spysee - http://spysee.com/

TechCrunch - http://search.techcrunch.com

TechCrunch - http://search.techcrunch.com

Big Milestone for

Big Milestone for BOSS

• 10 million queries per day in Dec 2009 = 300 million monthly queries

source: http://www.ysearchblog.com/2008/12/08/boss-reaches-a-milestone/

• 1 billion monthly queries in May 2009 source: http://www.ysearchblog.com/2008/12/08/boss-reaches-a-milestone/

Big Milestone for BOSS

What else can be built with

Get the pic - http://www.getthepicsure.com

Get the pic - http://www.getthepicsure.com

Get the pic - http://www.getthepicsure.com

What can be built with the BOSS API?

Client / Browser integrations

What can be built with the BOSS API?

Client / Browser integrations

What can be built with the BOSS API?

Client / Browser integrations

APIs About

Reality of Search: Huge Barriers of Entry

• Three players (G-Y-M) dominate the market, giving users limited choice

• Becoming a serious Search principal requires a massive capital investment and access to top technical talent

Reality of Search: Huge Barriers of Entry

•  Open up Yahoo!'s search infrastructure and enable third parties to build revolutionary search products.

•  Unrestricted!

•  http://developer.yahoo.com/search/boss

All crawled, ranked and presented in a

single API

What?

All crawled, ranked and presented in a

single API

Opening Yahoo! Search Tech

•  Removes entry barriers –  massive capital investment

•  Asset to innovate –  New relevancy models –  New presentation styles

•  Boom the search market –  Fragment the market, foster more players, choices,

competitions

Why?

•  Ability to re-rank results •  Blend results/content sources •  Ability to use any monetization platform •  Present results in any way •  No branding or attribution •  Unlimited queries per day* <BOSS Usage Fees>

•  Access to multiple verticals (web search, image, news) •  Spell checks, keyword suggestions (Key Terms) •  40+ supported language and region pairs •  SearchMonkey Structured Data, Delicious data •  Page data from Site Explorer •  Expanded Abstracts: 170 characters to 300! •  Mashup Framework

BOSS API Features

How to use

Getting Started with BOSS

1)  Info & Docs: http://developer.yahoo.com/boss 2)  Register for an BOSS App ID (free) 3)  Code samples: Javascript, PHP and Python

http://www.saurabhsahni.com/boss-examples.zip 4)  Join the BOSS Developer Group

http://tech.groups.yahoo.com/group/ysearchboss/ 5)  Starting your hacks

API Structure

The Basics:

http://boss.yahooapis.com/ysearch/{vert}/v1/{query}?appid={BOSSappid}&format=xml

•  {vert} = {web, news, images, spelling} •  {query} = {your search term} •  {appid} = required application id •  {format} = json is the default

API Structure

Optional Parameters: http://boss.yahooapis.com/ysearch/{vert}/v1/{query}?

appid={BOSSappid}&format=xml&optionalparameter={value} •  lang = {value} : 40+ languages •  count = {value} : number of results request •  start = {value} : first result requested •  sites = {value} : site restricts •  view=keyterms : related words/phrases for each result •  view = searchmonkey_rdf : SearchMonkey structured data •  view = delicious_toptags : most common public tags •  view = delicious_saves : # of delicious saves •  filter = porn / hate : filter out porn or hate-related content •  type = {value} : document format (e.g. word, pdf, etc) •  abstract = long : increases abstract length to 300 words

BOSS API: Web XML Response

http://boss.yahooapis.com/ysearch/web/v1/iphone?appid=…&format=xml&lang=tzh&region=tw

BOSS API: Find related keywords

http://boss.yahooapis.com/ysearch/web/v1/台北?appid=…&format=xml&lang=tzh&region=tw&age=1d-7d&view=keyterms

API Structure: Images

Optional Parameters for Images: http://boss.yahooapis.com/ysearch/images/v1/iphone?

appid={BOSSappid}&format=xml&optionalparameter={value} •  dimensions = {value}: all (default), small, medium, large, wallpaper,

widewallpaper •  refererurl = {value}: URL, no default. Search for this URL. Depending on

other query restrictions, returns all image objects with this referring URL. •  url = {value}: URL, no default. Search for this URL. Returns this exact image

result.

BOSS API: Images XML Response

http://boss.yahooapis.com/ysearch/images/v1/iphone?appid=…&format=xml&dimensions=small

API Structure: News

Optional Parameters for News: http://boss.yahooapis.com/ysearch/news/v1/台北?

appid={BOSSappid}&format=xml&optionalparameter={value} •  age = {value}: 7d is the default. (30d, 1w, 12h, 5d-10d, 2w-30d) •  orderby = {value}: relevance is the default. (add orderby=date to sort by

date)

BOSS API: News XML Response

http://boss.yahooapis.com/ysearch/news/v1/台北?appid=…&format=xml&lang=tzh&region=tw&age=1d-7d

BOSS API: Spell Check

http://boss.yahooapis.com/ysearch/spelling/v1/taipai?appid=…&format=xml

BOSS Mashup framework

An experimental Python library which provides SQL-like functions for mashing up the BOSS API with third-party data sources.

http://developer.yahoo.com/search/boss/mashup.html

BOSS Mashup framework

Simple code example:

BOSS Mashup framework

Output:

BMF+GAE

Enhanced version of BOSS Mashup Framework to Google App Engine:

http://zooie.wordpress.com/2008/08/04/yahoo-boss-google-app-engine-integrated/

Enables quick deployment of BOSS applications online

Thanks

Questions / Feedback?