Blogging, Googling, Blogging, Googling, Syndication: What’s the Web Syndication: What’s the Web coming to?coming to?

Ian GrahamIan Graham

Enterprise IT Strategy & BMO Connect, T&SEnterprise IT Strategy & BMO Connect, T&S

T: 416.513.5656T: 416.513.5656

E: ian.graham@bmo.comE: ian.graham@bmo.com

W: http://www.utoronto.ca/ian/talks/W: http://www.utoronto.ca/ian/talks/

What this talk is about What this talk is about

The The cultureculture of web applications of web applications– WhatWhat are the new applications about? are the new applications about?– HowHow are they ‘designed’ and put together are they ‘designed’ and put together – WhereWhere are they taking the ‘public’ Web are they taking the ‘public’ Web

Examples (Lots) Examples (Lots) – That help to illustrate the aboveThat help to illustrate the above

Some conclusions Some conclusions

What the talk is What the talk is notnot about about– Banking/financial services, software Banking/financial services, software

development, Web Services (tech details)development, Web Services (tech details)

Web culture Web culture

LocationLocation ConnectivityConnectivity DiscoveryDiscovery Communication & Communication &

PresencePresence CommunityCommunity

Blogging Blogging Syndication Syndication RSS RSS GoogleGoogle Jabber, ICQJabber, ICQ

Ebay, Yahoo, Ebay, Yahoo, Slashdot, Amazon, Slashdot, Amazon, Blogspots, … Blogspots, …

The right side lists just a few examples: The right side lists just a few examples: there are many others … there are many others …

A) Having a home: A) Having a home: BloggingBlogging Short for ‘Weblogging’Short for ‘Weblogging’

– A ‘log’ authored on / accessible over, the WebA ‘log’ authored on / accessible over, the Web» Simple Web-site generationSimple Web-site generation

– Personal (or group) publishing & communications:Personal (or group) publishing & communications:» Users often publish regularly (daily, or weekly), and link Users often publish regularly (daily, or weekly), and link

to friends / colleagues / coworkersto friends / colleagues / coworkers» Links establish virtual ‘communities’ with shared Links establish virtual ‘communities’ with shared

interestsinterests» Categorization for topics, etc. Categorization for topics, etc.

– Software is often free to download (and play with)Software is often free to download (and play with)» Free (or near) hosting available (see list at end)Free (or near) hosting available (see list at end)

– Software often supports user-level scripting to Software often supports user-level scripting to building custom functions, behaviorsbuilding custom functions, behaviors

Some Example BlogsSome Example Blogs

http://www.nerdgirl.com http://www.nerdgirl.com – A ‘normal’ person (purely social site)

http://weblogs.mozillazine.org/roadmap/ http://weblogs.mozillazine.org/roadmap/ [1][1]– Inventor of JavaScript – roadmap for development team

http://blogs.msdn.com/dmassy/archive/http://blogs.msdn.com/dmassy/archive/2004/06/16/157263.aspx2004/06/16/157263.aspx– Internet Explorer development team member

http://www.joelonsoftware.com/http://www.joelonsoftware.com/– Articles on software development

http://dear_raed.blogspot.com/http://dear_raed.blogspot.com/2003_02_01_dear_raed_archive.html 2003_02_01_dear_raed_archive.html [2][2]– Reported from inside Iraq before and during last year’s war

Uses of WeblogsUses of Weblogs

Pure social networkingPure social networking– Post musings and share them with friendsPost musings and share them with friends

(Semi-) professional information sharing(Semi-) professional information sharing– Post information for more general Post information for more general

communication (e.g. news, current affairs, communication (e.g. news, current affairs, professional commentary)professional commentary)

Work-related networkingWork-related networking– Post status reports, decisions and analyses, Post status reports, decisions and analyses,

etc, for sharing / cross-linking with others.etc, for sharing / cross-linking with others. Popularity?Popularity?

– There are There are millionsmillions of blogsites! of blogsites!

Blogspace Blogspace

• The collection of blog sites all across the Internet• Unconnected except for hand-created links in the pages

B) Two issues: B) Two issues: 1.1. Letting people know what you haveLetting people know what you have

– Publishing information ‘about’ blog postings Publishing information ‘about’ blog postings » RSS, Atom: XML languages for summary publishingRSS, Atom: XML languages for summary publishing

RSS RSS Resource Site Summary; RDF Site Summary; Really Resource Site Summary; RDF Site Summary; Really Simple Syndication …) – [ example later ]Simple Syndication …) – [ example later ]

– Also used by regular web sites, news sites, etc. Also used by regular web sites, news sites, etc.

2.2. Finding interesting/relevant stuffFinding interesting/relevant stuff– Aggregation sites and clientsAggregation sites and clients

» aggregating info ‘about’ published blogs / other sitesaggregating info ‘about’ published blogs / other sites» Headliner clients, topic-specific Web sites Headliner clients, topic-specific Web sites

– Categorization and rating systems Categorization and rating systems » Categorize blog items, Web page/sitesCategorize blog items, Web page/sites» separate wheat from chaffseparate wheat from chaff

B.1 RSS Feeds from a B.1 RSS Feeds from a blogsiteblogsite

Time-stamped Time-stamped ‘summary’ ‘summary’ about a blog about a blog itemitem

Metadata (for Metadata (for cataloging, cataloging, indexing)indexing)

Headline textHeadline text A link back to A link back to

the originalthe original

… … An ‘ad’ for An ‘ad’ for the original the original content.content.

<item><dc:creator>Dave Massy [MSFT] </dc:creator><title>Security, Security, Security</title><link>http://blogs.msdn.com/dmassy/archive/2004/06/20/160649.aspx</link><pubDate>Sun, 20 Jun 2004 12:56:00 GMT</pubDate><description><P>This topic seems to be coming up <P>&nbsp;</P><img src ="http://blogs.msdn.com/dmassy/aggbug/160649.aspx" width = "1" height = "1" /> …more text …</description></item>

E.g.: http://blogs.msdn.com/dmassy/ RSS link [3]

B.2. Aggregators & B.2. Aggregators & Syndicate Syndicate

Retrieves and categorizes RSS from blog (and other) sitesAnd provides a categorized feed to users, or a Web listing: http://aggregator.weblogs.co.uk/http://aggregator.weblogs.co.uk/ (a Blog aggregator) http://meerkat.oreilly.comhttp://meerkat.oreilly.com (an RSS feed aggregator) [4] http://www.newzcrawler.comhttp://www.newzcrawler.com (an RSS desktop client) [5]

RSS

RSS

RSSRSS

RSS or HTML

Aggregator APIs:Aggregator APIs:

APIs for talking to aggregation sitesAPIs for talking to aggregation sites– Simple, easy to use, often URL-basedSimple, easy to use, often URL-based– E.g. Meerkat API -- E.g. Meerkat API --

» http://www.oreillynet.com/meerkat/ http://www.oreillynet.com/meerkat/ [6][6]http://www.oreillynet.com/pub/a/rss/2000/05/09/meerkat_http://www.oreillynet.com/pub/a/rss/2000/05/09/meerkat_api.html -- \ API documentation)api.html -- \ API documentation)

– API based on API based on argumentsarguments in the URL (docs above in the URL (docs above explain how it works). Here are some examples:explain how it works). Here are some examples:» http://www.oreillynet.com/meerkat/?http://www.oreillynet.com/meerkat/?p=1065&_fl=rssp=1065&_fl=rss

[6.1,6.2][6.1,6.2]» http://www.oreillynet.com/meerkat/?http://www.oreillynet.com/meerkat/?p=1065&_fl=rss10p=1065&_fl=rss10

RSS feed of items in profilecategory 1065

Same, but use RSS 1.0 format for feed

And Embeddable in Pages:And Embeddable in Pages:( . . . demo.html) [6.3]

<html><head><title> Example of inclusion of RSS feeds using

JavaScript </title></head><body><h1> Heading For the Example </h1><p> Here is some regular HTML -- ok, i didn't have time to make it pretty </p><p> And here is the JavaScript example code:<script language="JavaScript"

src="http://meerkat.oreillynet.com/?_fl=js"></script></body> </html> Returned script sent by Meerkat;

writes out HTML summaries into file

SummarySummary Blogging is for Blogging is for individualsindividuals RSS (and syndication) help build RSS (and syndication) help build connectivityconnectivity, and , and

communitiescommunities Aggregators support simple APIs so it’s Aggregators support simple APIs so it’s easyeasy to build to build

functions that integrate these tools with others functions that integrate these tools with others (blogsites, portals, other..) (blogsites, portals, other..) – Richer communities via Richer communities via richer functionalityricher functionality– No conscious high-level No conscious high-level architecturearchitecture: people just try new : people just try new

ideas, and seeing what works (or doesn’t)ideas, and seeing what works (or doesn’t)– A heterogenous collection of apps, sharing using simple A heterogenous collection of apps, sharing using simple

protocols, APIs.protocols, APIs. Each application and protocol is Each application and protocol is simplesimple (limited (limited

scope, easy to use ), and scope, easy to use ), and openopen (encourages reuse, (encourages reuse, extension)extension) URL as a reference / API is key to this model URL as a reference / API is key to this model

C. GooglingC. Googling

A web search engineA web search engine– http://www.google.comhttp://www.google.com [7][7]– Other stuff:Other stuff:

» Blogger.com; Google groups, special search services, Blogger.com; Google groups, special search services, maps, Gmail (email) etc. maps, Gmail (email) etc.

Some numbers:Some numbers:– Estimated spaceEstimated space > 2,000,000 GB> 2,000,000 GB– Number of serversNumber of servers 100,000 (Linux) 100,000 (Linux)

Access to the index:Access to the index:– Via several APIs, some simple, and some Via several APIs, some simple, and some

complexcomplex

Google as a serviceGoogle as a service

Via APIs that let you reuse the Via APIs that let you reuse the google engine as part of your own google engine as part of your own application:application:– A URL-based API (like Meerkat)A URL-based API (like Meerkat)– A full-blown Web services interface A full-blown Web services interface – The latter lets you, essentially, use The latter lets you, essentially, use

Google as the search engine / Google as the search engine / database within your own database within your own applications (subject to restrictions) applications (subject to restrictions)

Google’s URL-based APIGoogle’s URL-based API ( . . . demo-google.html) [7.1]

<form id="search" method="get" action="http://www.google.com/custom">

<input type="hidden" name="cof" value="LW:600;LH:58;L:http://www.bmo.com/images/

...banner.gif;GIMP:#cc0000;T:black; ...;AWFID:9262c37cefe23a86;"> <input type="hidden" name="domains" value=“bmo.com"> <input type="hidden" name="sitesearch" value=“bmo.com"> <input type="text" id="q" name="q"

accesskey="s" size="30"> <input type="submit" id="submit" value="go"></form>

----- Info -----------------------http://www.google.com/faq_freewebsearch.htmlhttp://cyberzoide.developpez.com/html/google/

Google’s Web Services APIGoogle’s Web Services API

Experimental interface, but widely usedExperimental interface, but widely used– Requires registration for access key (simple, Requires registration for access key (simple,

and free)and free)– Provides library and tools for building simple Provides library and tools for building simple

applications, using SOAP-based web applications, using SOAP-based web services. services. » Java and .NET examples providedJava and .NET examples provided

API coversAPI covers– Ways to query the google databaseWays to query the google database– Response formats / error responsesResponse formats / error responses

Google ExampleGoogle Example[7.2][7.2]

public class GoogleAPIDemo { public static void main(String[] args) { … // Create a Google Search object, set our authorization key GoogleSearch s = new GoogleSearch(); s.setKey(clientKey); // do search or cache query, then print out result try { if (directive.equalsIgnoreCase("search")) { s.setQueryString(directiveArg); GoogleSearchResult r = s.doSearch(); System.out.println("Google Search Results:"); System.out.println(r.toString()); } else if (directive.equalsIgnoreCase("cached")) { …. } else if (directive.equalsIgnoreCase("spell")) { … } else { printUsageAndExit(); } } catch (GoogleSearchFault f) { System.out.println("The call to the Google Web APIs failed:"); System.out.println(f.toString()); } } private static void printUsageAndExit() { … } }}

Google SummaryGoogle Summary

Simple API focused on single functionSimple API focused on single function– Querying the google database.Querying the google database.

Loose couplingLoose coupling – Google doesn’t care about you, or what Google doesn’t care about you, or what

you’re doing. you’re doing. Coarse-grainedCoarse-grained

– Here’s a query, and there’s your answer. No Here’s a query, and there’s your answer. No details beyond those needed for business details beyond those needed for business function.function.

These two characteristics are core to These two characteristics are core to almost all ‘Web’ development models. almost all ‘Web’ development models.

D. Jabber: Messaging & D. Jabber: Messaging & presencepresence

Jabberserver

Jabberserver

Jabber clients

• Presence• User directory• Proxys to Yahoo, ICQ• Other services

Jabber servers

Jabber ExampleJabber Example

Jabberserver

Jabberserver

Jabber client

• Connect register presence

• Lookup user contact database

• Send text message contact database

Jabber client

• Requests and responses all sent in XML

• Generic XML protocol for exchanging messages, plus some services.

• Can be extended to non-text messaging applications

• Now used for virtual presence: lets you know who else is at a blog or other web site

Jabber client [8.1]

Virtual PresenceVirtual Presence(Jabbber extension: http://www.lluna.de/ [8.2])(Jabbber extension: http://www.lluna.de/ [8.2])

1.1. Web Server registers Web Server registers with virtual presence with virtual presence (VP) server, using a URL(VP) server, using a URL

2.2. VP client gets name of VP client gets name of VP server (URL) from VP server (URL) from Web server.Web server.

3.3. VP client registers with VP client registers with VP server. VP server.

4.4. VP server tells VP client VP server tells VP client who else is there.who else is there.

5.5. VP client provides chat VP client provides chat service to other users of service to other users of the site. the site.

VPserverWS 11

VPclient

22 4433

55

Key featuresKey features

Simple, single-purposeSimple, single-purpose URL-based referencing schemeURL-based referencing scheme

– Underlying protocol uses XML Underlying protocol uses XML Loose couplingLoose coupling between between

componentscomponents– servers, clients – failure is expectedservers, clients – failure is expected

Less-simple protocol / APILess-simple protocol / API– May explain why it isn’t as popular as May explain why it isn’t as popular as

other technologies. other technologies.

And many other examplesAnd many other examples

Bittorrent - file sharing / transfersBittorrent - file sharing / transfers [9][9]– http://bitconjurer.org/BitTorrent/http://bitconjurer.org/BitTorrent/

Wikis collaborative web authoring Wikis collaborative web authoring – http://en.wikipedia.org/wiki/Main_Pagehttp://en.wikipedia.org/wiki/Main_Page– http://en.wikipedia.org/wiki/Wikihttp://en.wikipedia.org/wiki/Wiki [10][10]

… …

An example applicationAn example application

Web page (from web server) that Web page (from web server) that includes data from an aggregator,includes data from an aggregator,– uses google to provide searchinguses google to provide searching– publishes new content info using RSSpublishes new content info using RSS– Uses jabber to provide presence Uses jabber to provide presence

functionality to visitorsfunctionality to visitors– Provides bittorrent links to downloadable Provides bittorrent links to downloadable

datadata

All integrated using URLs, and simple All integrated using URLs, and simple web protocols. web protocols.

ConclusionsConclusions

Social Social focus on community, focus on community, networking, connection, sharingnetworking, connection, sharing

Technical Technical Software design focused on Software design focused on– Simplicity of interfaces, and applications Simplicity of interfaces, and applications – URL as the key reference for an application, URL as the key reference for an application,

function, methodfunction, method– Loose coupling between applicationsLoose coupling between applications

» Construction of Web applications by integrating Construction of Web applications by integrating these different ‘application services’ together.these different ‘application services’ together.

Conclusion IIConclusion II““A large part of how we think about music is A large part of how we think about music is

influenced by the methods by with which it has influenced by the methods by with which it has conventionally been distributed. We think of pop conventionally been distributed. We think of pop songs as being three or four minutes long songs as being three or four minutes long because 40 years ago that was all that could fit because 40 years ago that was all that could fit on one side of a vinyl single.”on one side of a vinyl single.”

MobyMoby

We often think of Internet-based computing is We often think of Internet-based computing is the same way -- in terms of what we know -- not the same way -- in terms of what we know -- not what it is becoming. It is different, and what it is becoming. It is different, and challenges our thinking!challenges our thinking!

Couldn’t be a more fun time to be doing Couldn’t be a more fun time to be doing software!software!

Blogging, Googling, Blogging, Googling, Syndication: What’s the Web Syndication: What’s the Web coming to?coming to?

--- The End ------ The End ---

Ian GrahamIan Graham

Enterprise IT Strategy & BMO Connect, T&SEnterprise IT Strategy & BMO Connect, T&S

T: 416.513.5656T: 416.513.5656

E: ian.graham@bmo.comE: ian.graham@bmo.com

W: http://www.utoronto.ca/ian/talks/W: http://www.utoronto.ca/ian/talks/

Some Tools and SitesSome Tools and Sites Blogging software

– http://www.moveabletype.com– http://www.livejournal.org/– http://manila.userland.com/– http://www.slashcode.com/

Free blogging sites – http://www.blogger.com http://newrandom.blogspot.com (Ian’s site)– http://www.livejournal.com

RSS / Atom Aggregation sites– http://aggregator.weblogs.co.uk/– http://www.bloglines.com/– http://www.syndic8.com/– http://meerkat.oreilly.com (Open Wire Service)

» API Documentation: http://www.oreillynet.com/pub/a/rss/2000/05/09/meerkat_api.html

RSS / Atom Desktop clients– http://www.newzcrawler.com– http://www.headlineviewer.com

Some Tools and SitesSome Tools and Sites Google tools and APIs

– Google Hacks, Calishain and Dornfest, O’Reilly and Associates, 2003

» http://www.oreilly.com/catalog/googlehks/index.html– http://searchenginewatch.com/searchday/article.php/2161301– http://www.google.com/faq_freewebsearch.html (Free websearch)– http://www.google.com/services/custom.html (custom search)– http://cyberzoide.developpez.com/html/google/– http://www.google.ca/apis/ (Google Web service APIs)

Jabber– http://www.jabber.org– http://www.lluna.de (virtual presence extension)

Bittorrent– http://bitconjurer.org/BitTorrent/

Wikis– http://en.wikipedia.org/wiki/Wiki (all about Wikis)

Java Development Kit– http://java.sun.com/j2se/1.4.2/download.html