yql & yahoo! apis

Saurabh Sahni YDN Developer, Hacker, Evangelist

YQL & YAHOO! APIS

Jai SanthoshFrontend Engineer, Hacker

Yahoo!

What We’re Going to Cover

• YQL Overview

• Making Queries and the Console Sandbox

• Demo

• Open Data Tables and Execute (Server-Side JavaScript)

• Final Lessons

Hacks = Data manipulation

+Data visualization

The web is a large pile of data!

ProgrammableWeb.com – 4480 Apis

Yahoo! has alsoopened up its data

http://developer.yahoo.com/everything.html

Trouble with Data

• Need to find data API • Get Access – Signup for key• Find data endpoint• Read docs to learn what parameters you

have• Get data in obscure format• Use data after converting and filtering • More APIs you use, more is your

annoyance

To make data access easy on the web,

Yahoo! created YQL

YQL turns web services and data on the web into

databases.

=

select {what} from {where} where {conditions}

You can select, filter, sort and limit data and you can

even insert, update and delete from it.

YQL: http://developer.yahoo.com/yql/console

Finding Videos about Bangalore

• SELECT * FROM youtube.search where query='bangalore'

Selecting Photos of RHOK

• SELECT * FROM flickr.photos.search where text="rhok"

Inserting data

INSERT INTO bitly.shorten (login, apiKey, longUrl)

VALUES ('ME', 'API_KEY', 'http://yahoo.com')

updating data

UPDATE social.profile.status

SET status="Using YQL UPDATE”

WHERE guid="NJFIDHVPVVISDX7UKED2WHU"

Accessing Private Data• http://query.yahooapis.com/v1/yql

Uses OAuth 1.0 for authorization

OAuth is complicated – use one of our SDKs at https://github.com/yahoo

Retrieving my profile

• SELECT * FROM social.contacts WHERE guid=me

Using the YQL Console makes it very easy to

use it.

url structure• http://query.yahooapis.com/v1/yql?[params]• http://query.yahooapis.com/v1/public/yql?

[params] Paramsq = [ YQL QUERY ]format = [ XML / JSON ]diagnostics = [ true / false ]debug = [ true / false ]callback = [ function name ]

Let’s See It

Query Samples

• SELECT * FROM twitter.search where q='hackathon’ limit 5

• select * from answers.search where query="cars" and type="resolved"

• select * from google.translate where q="this is a test" and target="hi";

• select * from google.translate where q in (select text from twitter.search where q="hackday") and target='hi';

• select * from search.termextract where context in (select description from rss where url='http://rss.news.yahoo.com/rss/topstories')

Query Samples• select * from yahoo.finance.quotes where symbol in

("YHOO","AAPL","GOOG","MSFT")

• insert into yahoo.shortener (url, keysize) values ('http://www.javarants.com', 5)

• select * from craigslist.search where location="bangalore" and type="apa" and query="indiranagar”

• select * from html where url="http://finance.yahoo.com/q?s=yhoo" and xpath='//div[@id="yfi_headlines"]/div[2]/ul/li/a’

Making Requests: Flickr URLs<photo farm="3" id="5708163920"

isfamily="0" isfriend="0" ispublic="1" owner="31832337@N04" secret="0075137487" server="2496" title="San Francisco"/>

Making Requests: Flickr URLs

http://farm{$farm}.static.flickr.com/{$server}/{$id}_{$secret}.jpg

Photo URL

http://www.flickr.com/photos/{$owner}/{$id}

Photo Page URL

http://www.flickr.com/photos/{$owner}

Photo Owner Profile URL

YQL is open – you can get your data tables in our system

Craigslist data table<table> <meta> <author>Yahoo! Inc.</author> <documentationURL>http://craigslist.org/</documentationURL> <sampleQuery> select * from {table} where location="sfbay" and type="sss" and query=“bike”

</sampleQuery> </meta> <bindings> … </bindings></table>

Craigslist data table<table> <meta> … </meta> <bindings> <select itemPath="" produces="XML"> <urls> <url>http://{location}.craigslist.org/search/{type}?format=rss</url> </urls> <inputs> <key id="location" type="xs:string" paramType="path" required="true"/> <key id="type" type="xs:string" paramType="path" required="true"/> <key id="query" type="xs:string" paramType="query" required="true"/> </inputs> </select> </bindings></table>

Upload XML to our Githubhttp://github.com/yql/yql-tables

Or to your own Site

• The USE clause

USE 'http://www.mysite.com/my_table.xml' AS mytable;

SELECT * FROM mytableWHERE user='john_doe'

You can even write server side javascript to build a webservice

Final Lessons

• YQL is a skeleton key for all Yahoo! APIs

• Open data tables allow you to create new data feeds out of one or many different raw sources

• There are 1200+ Tables in YQL covering all major APIs on the web

• Use SDKs when accessing private data that requires OAuth

Final Lessons: Links• All Yahoo! APIs and Services• http://developer.yahoo.com/everything.html

• YQL Documentation• http://developer.yahoo.com/yql• YQL Console• http://developer.yahoo.com/yql/console

• YQL Github Account (Contribute Tables)• http://github.com/yql/yql-tables

http://isithackday.com/hackday-toolbox/

Questions?

Jai Santhosh

Twitter: @jaisanthGithub: http://github.com/jaisanth

http://www.slideshare.net/jaisanth

THANK YOU!