yql & yahoo! apis
DESCRIPTION
The YQL & Yahoo! Apis presentation done in RHOK BangaloreTRANSCRIPT
Saurabh Sahni YDN Developer, Hacker, Evangelist
YQL & YAHOO! APIS
Jai SanthoshFrontend Engineer, Hacker
Yahoo!
What We’re Going to Cover
• YQL Overview
• Making Queries and the Console Sandbox
• Demo
• Open Data Tables and Execute (Server-Side JavaScript)
• Final Lessons
Hacks = Data manipulation
+Data visualization
The web is a large pile of data!
ProgrammableWeb.com – 4480 Apis
Yahoo! has alsoopened up its data
http://developer.yahoo.com/everything.html
Trouble with Data
• Need to find data API • Get Access – Signup for key• Find data endpoint• Read docs to learn what parameters you
have• Get data in obscure format• Use data after converting and filtering • More APIs you use, more is your
annoyance
To make data access easy on the web,
Yahoo! created YQL
YQL turns web services and data on the web into
databases.
=
select {what} from {where} where {conditions}
You can select, filter, sort and limit data and you can
even insert, update and delete from it.
YQL: http://developer.yahoo.com/yql/console
Finding Videos about Bangalore
• SELECT * FROM youtube.search where query='bangalore'
Selecting Photos of RHOK
• SELECT * FROM flickr.photos.search where text="rhok"
Inserting data
INSERT INTO bitly.shorten (login, apiKey, longUrl)
VALUES ('ME', 'API_KEY', 'http://yahoo.com')
updating data
UPDATE social.profile.status
SET status="Using YQL UPDATE”
WHERE guid="NJFIDHVPVVISDX7UKED2WHU"
Accessing Private Data• http://query.yahooapis.com/v1/yql
Uses OAuth 1.0 for authorization
OAuth is complicated – use one of our SDKs at https://github.com/yahoo
Retrieving my profile
• SELECT * FROM social.contacts WHERE guid=me
Using the YQL Console makes it very easy to
use it.
url structure• http://query.yahooapis.com/v1/yql?[params]• http://query.yahooapis.com/v1/public/yql?
[params] Paramsq = [ YQL QUERY ]format = [ XML / JSON ]diagnostics = [ true / false ]debug = [ true / false ]callback = [ function name ]
Let’s See It
Query Samples
• SELECT * FROM twitter.search where q='hackathon’ limit 5
• select * from answers.search where query="cars" and type="resolved"
• select * from google.translate where q="this is a test" and target="hi";
• select * from google.translate where q in (select text from twitter.search where q="hackday") and target='hi';
• select * from search.termextract where context in (select description from rss where url='http://rss.news.yahoo.com/rss/topstories')
Query Samples• select * from yahoo.finance.quotes where symbol in
("YHOO","AAPL","GOOG","MSFT")
• insert into yahoo.shortener (url, keysize) values ('http://www.javarants.com', 5)
• select * from craigslist.search where location="bangalore" and type="apa" and query="indiranagar”
• select * from html where url="http://finance.yahoo.com/q?s=yhoo" and xpath='//div[@id="yfi_headlines"]/div[2]/ul/li/a’
Making Requests: Flickr URLs<photo farm="3" id="5708163920"
isfamily="0" isfriend="0" ispublic="1" owner="31832337@N04" secret="0075137487" server="2496" title="San Francisco"/>
Making Requests: Flickr URLs
http://farm{$farm}.static.flickr.com/{$server}/{$id}_{$secret}.jpg
Photo URL
http://www.flickr.com/photos/{$owner}/{$id}
Photo Page URL
http://www.flickr.com/photos/{$owner}
Photo Owner Profile URL
YQL is open – you can get your data tables in our system
Craigslist data table<table> <meta> <author>Yahoo! Inc.</author> <documentationURL>http://craigslist.org/</documentationURL> <sampleQuery> select * from {table} where location="sfbay" and type="sss" and query=“bike”
</sampleQuery> </meta> <bindings> … </bindings></table>
Craigslist data table<table> <meta> … </meta> <bindings> <select itemPath="" produces="XML"> <urls> <url>http://{location}.craigslist.org/search/{type}?format=rss</url> </urls> <inputs> <key id="location" type="xs:string" paramType="path" required="true"/> <key id="type" type="xs:string" paramType="path" required="true"/> <key id="query" type="xs:string" paramType="query" required="true"/> </inputs> </select> </bindings></table>
Upload XML to our Githubhttp://github.com/yql/yql-tables
Or to your own Site
• The USE clause
USE 'http://www.mysite.com/my_table.xml' AS mytable;
SELECT * FROM mytableWHERE user='john_doe'
You can even write server side javascript to build a webservice
Final Lessons
• YQL is a skeleton key for all Yahoo! APIs
• Open data tables allow you to create new data feeds out of one or many different raw sources
• There are 1200+ Tables in YQL covering all major APIs on the web
• Use SDKs when accessing private data that requires OAuth
Final Lessons: Links• All Yahoo! APIs and Services• http://developer.yahoo.com/everything.html
• YQL Documentation• http://developer.yahoo.com/yql• YQL Console• http://developer.yahoo.com/yql/console
• YQL Github Account (Contribute Tables)• http://github.com/yql/yql-tables
http://isithackday.com/hackday-toolbox/
Questions?
Jai Santhosh
Twitter: @jaisanthGithub: http://github.com/jaisanth
http://www.slideshare.net/jaisanth
THANK YOU!