Social Media Programming Khanderao Kand
CTOGloMantra Inc
mybantu virtual personalized Assistance
@khanderaoNov 2, 2011 at JAX Conference, London
CTO of Glomantra and myBantu in Silicon Valley, CA, USA AI based Relevance platform / recommendation technology with Social
Intelligence Projects on Big Data, Computing, Machine Learning, Text Analytics, SOA,
User Interest profiling, Yield Optimization
BIO: 20 years in Industry (currently AI, SOA, Social, Mobile and Cloud
Technologies) Lead Oracle’s SOA and BPM Architecture Chief Consulting Architect for SOA and BPM to 50+ architects & hundreds of
developers of Oracle’s next Generation Fusion Apps Lead Architect of Peopletools Architect in CRM, Online Dialog Marketing, OLAP Hyperion Participated in Standards of BPEL and SCA at OASIS 9 Patents filled Frequent Speaker in conferences Authored articles in JDJ, WSJ, Cloud Computing
Khanderao Kand
APIs, Frameworks and Tools for Social Computing: Key Technologies, APIs Interfacing with Popular
Social media Storage Social Media Data Processing
Agenda
Foundational Protocols and Standards for Social Media
OAuth is open standard for Authorizing Access (typically data access / actions)
Oauth hands over access token and not user credentials
Oauth is essential to developing social-ready applications
OAuth protocol (Dance) involves series of 3 party (consumer browser, Service provider – Social media Site and App).
OAuth, including Twitter, Facebook, LinkedIn, TripIt, and Foursquare, as well as the Google and Yahoo APIs.
Different SM supports either Oauth 1.0 and Oauth 2.0
Visit : http://oauth.net
APIs: Jersery (JAX-RS), Spring Security API, Googlecode,
OAuth
Open ID is open source standard for multi-site authentication
OpenId is for authentication as against Oauth No need of adhoc and site specific authentication Oauth and OpenID can coexist Open ID providers : think of Google login Providers pass ID attributes to the relying party (app) However, Oauth is more popular and common in Social
Media interfaces Visit http://www.openid.org Java API: openid4java, GAE supports,
Open ID
Representational State Transfer (Roy Fielding’s paper for Architectural Style over HTTP)
Basic GET, PUT, POST, DELETE operations on a resource
Internet world REST is currently winning
Google, Yahoo, Facebook, etc etc.
Java: Jersey (JAX-RS), Restlet, RestFul Services, JAX-WS
REST
Javascript Simple Object Notation as lightweight data interchange format
Lighter than XML Can be manipulated on browsers No need for parsing into DOM http://www.json.org Java API: GSON, Jackson, JSONLib, FlexJSON XML to/from JSON conversion: JSONLib
JSON
Really Simple Syndication (originally RDF Site Summary)
Started with news, site, blogs update Simple and lightweight Popular Readers: Google Reader, Feedly etc Java API: ROME, Simple XML parsing
RSS
APIs of Popular Social Media
Representing objects and relationships in Facebook’s social network
Simple, consistent and REST style representation Objects: People, events, pages, and photos
People: http://graph.facebook.com/659690992/picture Events: http://graph.facebook.com/331218348435/picture Groups: http://graph.facebook.com/69048030774/picture Pages: http://graph.facebook.com/DoloresPark/picture Applications:
http://graph.facebook.com/2318966938/picture Photo Albums: http://graph.facebook.com/platform/picture
Access API: https://graph.facebook.com/ID/CONNECTION_TYPE
Facebook Graph API
https://graph.facebook.com/ID/CONNECTION_TYPE
Friends: https://graph.facebook.com/me/friends?access_token=... News feed: https://graph.facebook.com/me/home?access_token=... Profile feed (Wall): https://graph.facebook.com/me/feed?access_token=... Likes: https://graph.facebook.com/me/likes?access_token=... Movies: https://graph.facebook.com/me/movies?access_token=... Music: https://graph.facebook.com/me/music?access_token=... Books: https://graph.facebook.com/me/books?access_token=... Notes: https://graph.facebook.com/me/notes?access_token=... Permissions: https://graph.facebook.com/me/permissions?access_token=... Photo Tags: https://graph.facebook.com/me/photos?access_token=... Photo Albums: https://graph.facebook.com/me/albums?access_token=... Video Tags: https://graph.facebook.com/me/videos?access_token=... Video Uploads: https://graph.facebook.com/me/videos/uploaded?access_token=... Events: https://graph.facebook.com/me/events?access_token=... Groups: https://graph.facebook.com/me/groups?access_token=... Checkins: https://graph.facebook.com/me/checkins?access_token=…
Facebook Graph API: Access
All public objects https://graph.facebook.com/search?q=QUERY&type=OBJECT_TYPE
All public posts: https://graph.facebook.com/search?q=socialcommerce&type=post
People: https://graph.facebook.com/search?q=Khanderao&type=user Pages: https://graph.facebook.com/search?q=glomantra&type=page Events: https://graph.facebook.com/search?q=conference&type=event Groups: https://graph.facebook.com/search?q=programming&type=group
Places: https://graph.facebook.com/search?q=coffee&type=place¢er=37.76,122.427&distance=1000
Checkins: https://graph.facebook.com/search?type=checkin
Facebook Search using Graph API
HTTP Post Access token curl -F 'access_token=...' \
https://graph.facebook.com/313449204401/likes
Facebook: Publishing on Wall
Example String url = "https://graph.facebook.com/"+facebook ID+"/feed";String data = URLEncoder.encode("access_token", "UTF-8") + "=" + URLEncoder.encode(“ACCESS TOKEN STRING”, "UTF-8");
data += "&" + URLEncoder.encode("message", "UTF-8") + "=" + URLEncoder.encode(“wallMessageText”, "UTF-8");
data += "&" + URLEncoder.encode("picture", "UTF-8") + "=" + URLEncoder.encode(”IMAGE URL”, "UTF-8");
data += "&" + URLEncoder.encode("name", "UTF-8") + "=" + URLEncoder.encode(“POST TITLE STRING”, "UTF-8");
data += "&" + URLEncoder.encode("caption", "UTF-8") + "=" + URLEncoder.encode(“CAPTION STRING”, "UTF-8");
data += "&" + URLEncoder.encode("description", "UTF-8") + "=" + URLEncoder.encode(“DESCRIPTION”, "UTF-8");
data += "&" + URLEncoder.encode("actions", "UTF-8") + "=" + URLEncoder.encode("{\"name\": \”action_name\", \"link\": \” http://192.169.0.46/app/valid url1\"}", "UTF-8");
data += "&" + URLEncoder.encode("link", "UTF-8") + "=" + URLEncoder.encode(“LINK URL”, "UTF-8");
// Constructing action linksJSONObject obj = new JSONObject();LinkedHashMap<String, Object> m1 = new LinkedHashMap<String, Object>();m1.put("text“,”LINK1 TEXT”);m1.put("href", “http://192.169.0.46/app/valid action url1”);obj.put(”LINK1 LABEL”, m1);
Twitter is ranked as one of the ten-most-visited websites worldwide by Alexa's web traffic analysis.
March 2010 10 billionth tweet
Twitter has a user retention rate of forty percent.
Value: Conversational 30%, Pointless Babble 30%, Promotion 8%, News 8%, news 7-8% and spam
Good source: news validation, trend, topic updates
"Twitter Revolutions" and which include the 2011 Egyptian revolution, 2010–2011 Tunisian protests, 2009–2010 Iranian election protests, and 2009 Moldova civil unrest
Twitter is also increasingly used for making TV more interactive and social
Most of the celebrities are on Twitter President Obama and many others effectively used Twitter for
campaigning Twitter has become a great source for latest news Twitter is an indicator for buzz around latest topics and trends Brands are using twitters for promotions of products and
spreading deals
Twitter Usage
Twitter4J : Open Source BSD
Java-Twitter
Jtwitter : Open Source
Twitter Client Streaming API
3rd party Twitter Java APIs
Twitter API allows to Tweet Direct message Search Get tweets
Twitter4J (third party – open source) Access token / consumer key in twitter.properties
Twitter twitter = new TwitterFactory().getInstance();Query query = new Query(”mybantu");QueryResult result = twitter.search(query);// uodate status Status status = twitter.updateStatus(latestStatus);
Twitter4J API
GNIP 50% at approximately 360K per year For analysis but not for resale or display tweets
Spritzer 2% random for free
Twitter Fire hose
"For the question about Google plans to allow developers to create Google+ apps deploying the OpenSocial APIs, a spokeswoman of Google said that by means of email nowadays the Google+ platform can’t support the OpenSocial APIs.
She also added that nevertheless, they are utilizing many of the technology which was designed as part of OpenSocial, along with the gadget application packaging model, and power Google+ games and the Portable Contacts JSON schema. Because they define the +Platform APIs, they are drawing close attention to the future tendency of the OpenSocial APIs, as well as converging everything possible."
Open Social Project
Apache Opensource project in incubation Shindig
Developing Social Network
Great initiative with huge promise A common “open source” API for social media launched in
2007 Early movers as open social containers: Google Orkut, hi5,
LinkedIn, MySpace, Netlog, Ning, orkut, and Yahoo! Opensource OpenSocial Based container : Apache Shindig
Bummer… Except Facebook Google+ is not yet committed to Open Social
Open Social
By supporting open Web standards for describing connections between people, web sites can add to the social infrastructure of the web
The API returns web addresses of public pages and publicly declared connections between them.
Allows developers to create a button for a Web site that would allow a registered user to easily "add friends.”
index the public Web for Friends of A Friend (FOAF) and Friends Network
Google Social Graph API
The Social Graph API looks for two types of publicly declared connections:
1. It looks for all public URLs that belong to you and are interconnected. This could be your blog (a1), your LiveJournal page (a2), and your Twitter account (a3).
2. It looks for publicly declared connections between people. For example, a1 may link to b's blog while a1 and c link to each other.
Google’s Public Social Graph
Searching Public posts GET https://www.googleapis.com/plus/v1/activities?query=cookie%20recipes&orderBy=best&key=[yourAPIKey]
People Search
GET https://www.googleapis.com/plus/v1/people?query=Khanderao%20kand&key=[yourAPIKey]
View People Interacting with PostsGET https://www.googleapis.com/plus/v1/activities/{activityId}/people/resharers?key=[yourAPIKey] GET https://www.googleapis.com/plus/v1/activities/{activityId}/people/plusoners?key=[yourAPIKey]
Google+ API https://developers.google.com/+/api/
Spring Social provides templates forREST based API handling OAUth and providing operations specific to Social Media
TwitterTemplate FacebookTemplate LinkedInTemplate TripItTemplate
http://blog.springsource.com/2010/11/03/socializing-spring-applications/
http://greenhouse.springsource.org
Spring Social
Technologies for Building Applications on Social
Media
Social Media Contains: Huge Data, Often available via JSON, XML , RSS
Requires different type of DBs…. Couch DB MongoDB Cassandra Hbase Neo4J
Hadoop for Big Data Processing
Social Media Data Often Needs Different Type of Database
CouchDB MongoDB
Data Model Document Oriented Document Oriented
Data Format JSON BSON
Interface REST TCP/IP
Query MapReduce Java Script Jmap Reduce ava Script + Query
Fault Tolerance Master Master Master Slave
Concurrency MVCC (Versioning) Updates in place
Performance Comparatively lower Higher
Cassandra HBase
Type Column DB Colletcion- column DB
Type Big Table Big Table
Protocol Binary (Thrift) Binary (Thrift) HTTP/REST
Map Reduce Via Hadoop Via Hadoop
Dealing with Big Data
1. The presentation though quotes examples of my Company and products, the presentation is based on my knowledge and experience and my company is not liable.
2. Attributed to most of the known source, if any attribution is missing, it is not due to intention but might not have got the information of the original source or might be be negligence in my due diligence. It can be corrected if you contact
3. Feel free to share some / all content however please attribute to Khanderao Kand
4. Social Media Networks mentioned in the presentation are more foe example but the approach is applicable to most of all other social media and there is no intention to promote / discourage any media
5. User usage data and financial numbers are dated and would be stale / outdated over a period of time. You may need to look for the latest data.
6. APIs , tools and libraries are third party and not from me / my company. Though they are listed based on my experience and knowledge, use them at your risk.
7. We have products in Social Commerce, Social CRM, and recommendation. We also take projects around the space and can be done outsource / jointly if interested in.
Contact : [email protected] or [email protected] Twitter @khanderao
For original Slide deck (ppt) feel free to contact.
Disclaimer and Request
Q & A