Social Media Programming Khanderao Kand

CTOGloMantra Inc

mybantu virtual personalized Assistance

@khanderaoNov 2, 2011 at JAX Conference, London

CTO of Glomantra and myBantu in Silicon Valley, CA, USA AI based Relevance platform / recommendation technology with Social

Intelligence Projects on Big Data, Computing, Machine Learning, Text Analytics, SOA,

User Interest profiling, Yield Optimization

BIO: 20 years in Industry (currently AI, SOA, Social, Mobile and Cloud

Technologies) Lead Oracle’s SOA and BPM Architecture Chief Consulting Architect for SOA and BPM to 50+ architects & hundreds of

developers of Oracle’s next Generation Fusion Apps Lead Architect of Peopletools Architect in CRM, Online Dialog Marketing, OLAP Hyperion Participated in Standards of BPEL and SCA at OASIS 9 Patents filled Frequent Speaker in conferences Authored articles in JDJ, WSJ, Cloud Computing

Khanderao Kand

APIs, Frameworks and Tools for Social Computing: Key Technologies, APIs Interfacing with Popular

Social media Storage Social Media Data Processing

Agenda

Foundational Protocols and Standards for Social Media

OAuth is open standard for Authorizing Access (typically data access / actions)

Oauth hands over access token and not user credentials

Oauth is essential to developing social-ready applications

OAuth protocol (Dance) involves series of 3 party (consumer browser, Service provider – Social media Site and App).

OAuth, including Twitter, Facebook, LinkedIn, TripIt, and Foursquare, as well as the Google and Yahoo APIs.

Different SM supports either Oauth 1.0 and Oauth 2.0

Visit : http://oauth.net

APIs: Jersery (JAX-RS), Spring Security API, Googlecode,

OAuth

Open ID is open source standard for multi-site authentication

OpenId is for authentication as against Oauth No need of adhoc and site specific authentication Oauth and OpenID can coexist Open ID providers : think of Google login Providers pass ID attributes to the relying party (app) However, Oauth is more popular and common in Social

Media interfaces Visit http://www.openid.org Java API: openid4java, GAE supports,

Open ID

Representational State Transfer (Roy Fielding’s paper for Architectural Style over HTTP)

Basic GET, PUT, POST, DELETE operations on a resource

Internet world REST is currently winning

Google, Yahoo, Facebook, etc etc.

Java: Jersey (JAX-RS), Restlet, RestFul Services, JAX-WS

REST

Javascript Simple Object Notation as lightweight data interchange format

Lighter than XML Can be manipulated on browsers No need for parsing into DOM http://www.json.org Java API: GSON, Jackson, JSONLib, FlexJSON XML to/from JSON conversion: JSONLib

JSON

Really Simple Syndication (originally RDF Site Summary)

Started with news, site, blogs update Simple and lightweight Popular Readers: Google Reader, Feedly etc Java API: ROME, Simple XML parsing

RSS

APIs of Popular Social Media

Representing objects and relationships in Facebook’s social network

Simple, consistent and REST style representation Objects: People, events, pages, and photos

People: http://graph.facebook.com/659690992/picture Events: http://graph.facebook.com/331218348435/picture Groups: http://graph.facebook.com/69048030774/picture Pages: http://graph.facebook.com/DoloresPark/picture Applications:

http://graph.facebook.com/2318966938/picture Photo Albums: http://graph.facebook.com/platform/picture

Access API: https://graph.facebook.com/ID/CONNECTION_TYPE

Facebook Graph API

https://graph.facebook.com/ID/CONNECTION_TYPE

Friends: https://graph.facebook.com/me/friends?access_token=... News feed: https://graph.facebook.com/me/home?access_token=... Profile feed (Wall): https://graph.facebook.com/me/feed?access_token=... Likes: https://graph.facebook.com/me/likes?access_token=... Movies: https://graph.facebook.com/me/movies?access_token=... Music: https://graph.facebook.com/me/music?access_token=... Books: https://graph.facebook.com/me/books?access_token=... Notes: https://graph.facebook.com/me/notes?access_token=... Permissions: https://graph.facebook.com/me/permissions?access_token=... Photo Tags: https://graph.facebook.com/me/photos?access_token=... Photo Albums: https://graph.facebook.com/me/albums?access_token=... Video Tags: https://graph.facebook.com/me/videos?access_token=... Video Uploads: https://graph.facebook.com/me/videos/uploaded?access_token=... Events: https://graph.facebook.com/me/events?access_token=... Groups: https://graph.facebook.com/me/groups?access_token=... Checkins: https://graph.facebook.com/me/checkins?access_token=…

Facebook Graph API: Access

All public objects https://graph.facebook.com/search?q=QUERY&type=OBJECT_TYPE

All public posts: https://graph.facebook.com/search?q=socialcommerce&type=post

People: https://graph.facebook.com/search?q=Khanderao&type=user Pages: https://graph.facebook.com/search?q=glomantra&type=page Events: https://graph.facebook.com/search?q=conference&type=event Groups: https://graph.facebook.com/search?q=programming&type=group

Places: https://graph.facebook.com/search?q=coffee&type=place&center=37.76,122.427&distance=1000

Checkins: https://graph.facebook.com/search?type=checkin

Facebook Search using Graph API

HTTP Post Access token curl -F 'access_token=...' \

https://graph.facebook.com/313449204401/likes

Facebook: Publishing on Wall

Example String url = "https://graph.facebook.com/"+facebook ID+"/feed";String data = URLEncoder.encode("access_token", "UTF-8") + "=" + URLEncoder.encode(“ACCESS TOKEN STRING”, "UTF-8");

data += "&" + URLEncoder.encode("message", "UTF-8") + "=" + URLEncoder.encode(“wallMessageText”, "UTF-8");

data += "&" + URLEncoder.encode("picture", "UTF-8") + "=" + URLEncoder.encode(”IMAGE URL”, "UTF-8");

data += "&" + URLEncoder.encode("name", "UTF-8") + "=" + URLEncoder.encode(“POST TITLE STRING”, "UTF-8");

data += "&" + URLEncoder.encode("caption", "UTF-8") + "=" + URLEncoder.encode(“CAPTION STRING”, "UTF-8");

data += "&" + URLEncoder.encode("description", "UTF-8") + "=" + URLEncoder.encode(“DESCRIPTION”, "UTF-8");

data += "&" + URLEncoder.encode("actions", "UTF-8") + "=" + URLEncoder.encode("{\"name\": \”action_name\", \"link\": \” http://192.169.0.46/app/valid url1\"}", "UTF-8");

data += "&" + URLEncoder.encode("link", "UTF-8") + "=" + URLEncoder.encode(“LINK URL”, "UTF-8");

// Constructing action linksJSONObject obj = new JSONObject();LinkedHashMap<String, Object> m1 = new LinkedHashMap<String, Object>();m1.put("text“,”LINK1 TEXT”);m1.put("href", “http://192.169.0.46/app/valid action url1”);obj.put(”LINK1 LABEL”, m1);

Twitter is ranked as one of the ten-most-visited websites worldwide by Alexa's web traffic analysis.

March 2010 10 billionth tweet

Twitter has a user retention rate of forty percent.

Value: Conversational 30%, Pointless Babble 30%, Promotion 8%, News 8%, news 7-8% and spam

Good source: news validation, trend, topic updates

Twitter

"Twitter Revolutions" and which include the 2011 Egyptian revolution, 2010–2011 Tunisian protests, 2009–2010 Iranian election protests, and 2009 Moldova civil unrest

Twitter is also increasingly used for making TV more interactive and social

Most of the celebrities are on Twitter President Obama and many others effectively used Twitter for

campaigning Twitter has become a great source for latest news Twitter is an indicator for buzz around latest topics and trends Brands are using twitters for promotions of products and

spreading deals

Twitter Usage

Twitter4J : Open Source BSD

Java-Twitter

Jtwitter : Open Source

Twitter Client Streaming API

3rd party Twitter Java APIs

Twitter API allows to Tweet Direct message Search Get tweets

Twitter

Twitter4J (third party – open source) Access token / consumer key in twitter.properties

Twitter twitter = new TwitterFactory().getInstance();Query query = new Query(”mybantu");QueryResult result = twitter.search(query);// uodate status Status status = twitter.updateStatus(latestStatus);

Twitter4J API

GNIP 50% at approximately 360K per year For analysis but not for resale or display tweets

Spritzer 2% random for free

Twitter Fire hose

"For the question about Google plans to allow developers to create Google+ apps deploying the OpenSocial APIs, a spokeswoman of Google said that by means of email nowadays the Google+ platform can’t support the OpenSocial APIs.

She also added that nevertheless, they are utilizing many of the technology which was designed as part of OpenSocial, along with the gadget application packaging model, and power Google+ games and the Portable Contacts JSON schema. Because they define the +Platform APIs, they are drawing close attention to the future tendency of the OpenSocial APIs, as well as converging everything possible."

Open Social Project

Apache Opensource project in incubation Shindig

Developing Social Network

Great initiative with huge promise A common “open source” API for social media launched in

2007 Early movers as open social containers: Google Orkut, hi5,

LinkedIn, MySpace, Netlog, Ning, orkut, and Yahoo! Opensource OpenSocial Based container : Apache Shindig

Bummer… Except Facebook Google+ is not yet committed to Open Social

Open Social

By supporting open Web standards for describing connections between people, web sites can add to the social infrastructure of the web

The API returns web addresses of public pages and publicly declared connections between them.

Allows developers to create a button for a Web site that would allow a registered user to easily "add friends.”

index the public Web for Friends of A Friend (FOAF) and Friends Network

Google Social Graph API

The Social Graph API looks for two types of publicly declared connections:

1. It looks for all public URLs that belong to you and are interconnected. This could be your blog (a1), your LiveJournal page (a2), and your Twitter account (a3).

2. It looks for publicly declared connections between people. For example, a1 may link to b's blog while a1 and c link to each other.

Google’s Public Social Graph

Searching Public posts GET https://www.googleapis.com/plus/v1/activities?query=cookie%20recipes&orderBy=best&key=[yourAPIKey]

People Search

GET https://www.googleapis.com/plus/v1/people?query=Khanderao%20kand&key=[yourAPIKey]

View People Interacting with PostsGET https://www.googleapis.com/plus/v1/activities/{activityId}/people/resharers?key=[yourAPIKey] GET https://www.googleapis.com/plus/v1/activities/{activityId}/people/plusoners?key=[yourAPIKey]

Google+ API https://developers.google.com/+/api/

Spring Social provides templates forREST based API handling OAUth and providing operations specific to Social Media

TwitterTemplate FacebookTemplate LinkedInTemplate TripItTemplate

http://blog.springsource.com/2010/11/03/socializing-spring-applications/

http://greenhouse.springsource.org

Spring Social

Technologies for Building Applications on Social

Media

Social Media Contains: Huge Data, Often available via JSON, XML , RSS

Requires different type of DBs…. Couch DB MongoDB Cassandra Hbase Neo4J

Hadoop for Big Data Processing

Social Media Data Often Needs Different Type of Database

CouchDB MongoDB

Data Model Document Oriented Document Oriented

Data Format JSON BSON

Interface REST TCP/IP

Query MapReduce Java Script Jmap Reduce ava Script + Query

Fault Tolerance Master Master Master Slave

Concurrency MVCC (Versioning) Updates in place

Performance Comparatively lower Higher

Cassandra HBase

Type Column DB Colletcion- column DB

Type Big Table Big Table

Protocol Binary (Thrift) Binary (Thrift) HTTP/REST

Map Reduce Via Hadoop Via Hadoop

Dealing with Big Data

1. The presentation though quotes examples of my Company and products, the presentation is based on my knowledge and experience and my company is not liable.

2. Attributed to most of the known source, if any attribution is missing, it is not due to intention but might not have got the information of the original source or might be be negligence in my due diligence. It can be corrected if you contact

3. Feel free to share some / all content however please attribute to Khanderao Kand

4. Social Media Networks mentioned in the presentation are more foe example but the approach is applicable to most of all other social media and there is no intention to promote / discourage any media

5. User usage data and financial numbers are dated and would be stale / outdated over a period of time. You may need to look for the latest data.

6. APIs , tools and libraries are third party and not from me / my company. Though they are listed based on my experience and knowledge, use them at your risk.

7. We have products in Social Commerce, Social CRM, and recommendation. We also take projects around the space and can be done outsource / jointly if interested in.

Contact : khanderao.kand@glomantra.com or khanderao@yahoo.com Twitter @khanderao

For original Slide deck (ppt) feel free to contact.

Disclaimer and Request

Q & A