monitoring and maximising organisational impact using the social web
DESCRIPTION
http://www.ukoln.ac.uk/web-focus/events/conferences/ili-2010/. Session. C102. Acceptable Use Policy Recording/broadcasting of this talk, taking photographs, discussing the content using Twitter, blogs, etc. is permitted providing distractions to others is minimised. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/1.jpg)
UKOLN is supported by:
Monitoring and Maximising Organisational Impact using the Social Web
Brian KellyUKOLNUniversity of BathBath, UK
http://www.ukoln.ac.uk/web-focus/events/conferences/ili-2010/http://www.ukoln.ac.uk/web-focus/events/conferences/ili-2010/
This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence (but note caveat)
Acceptable Use PolicyRecording/broadcasting of this talk, taking photographs, discussing the content using Twitter, blogs, etc. is permitted providing distractions to others is minimised.
Acceptable Use PolicyRecording/broadcasting of this talk, taking photographs, discussing the content using Twitter, blogs, etc. is permitted providing distractions to others is minimised.
Tag for del.icio.us ‘ili-2010-talk'Tag for del.icio.us ‘ili-2010-talk'
Email:[email protected]
Twitter:http://twitter.com/briankelly/
Blog:http://ukwebfocus.wordpress.com/
C102Session
![Page 2: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/2.jpg)
2
About Me
Brian Kelly:• UK Web Focus: a national advisory post• Long-standing Web evangelist (since Jan
1993)• Based at UKOLN, University of Bath, with
remit to advise HE/FE sector• 790+ blog posts since Nov 2006• Over 350 presentations given since 1997• Speaker at all ILI conferences except ILI
2008
Intr
od
uct
ion
![Page 3: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/3.jpg)
About This TalkSummary
• The Social Web is becoming accepted as a valuable tool for use by information professionals.
• But how do we measure the effectiveness of our use of the Social Web?
• How do we provide evidence of value of our services?
• What are the limitations of gathering and using metrics?
• What should we be doing?
3
Intr
od
uct
ion
![Page 4: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/4.jpg)
Contents
Structure of this talk
4
![Page 5: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/5.jpg)
Reshaping The Gartner Curve
Need to (a) avoid the chasm; (b) manage expectations and (c) move to productive service
5
Advocacy
Supporting early mainstream adopters
Listening to users
Gathering evidence
Demonstrating ROI
![Page 6: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/6.jpg)
Some Statistics
Statistics about the UK Web Focus blog (from 1 Nov 2007- 23 Aug 2010)
6
770 41
270,978
561,401
3,584
Impressive?
: no. of posts : Wikio ranking
: no. of comments
: no. of views
: no. of spam comments (since reset)
Not really! Nos. meaningless without context
![Page 7: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/7.jpg)
What’s The Point?
Arguments against gathering statistics:• We know we’re doing a good job• Our users are happy with our services• The effort with distract from core business
Arguments for gathering statistics:• The government is unconvinced!• The Daily Mail is unconvinced!• Evidence may indicate that services are no
longer being widely used• Evidence may indicate that services are
popular and required extra resources 7
![Page 8: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/8.jpg)
Political Context• David Camefon
8
X
![Page 9: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/9.jpg)
Need To Understand Trends
9
There’s a context to answers. Evidence can help inform decisions.
How effective are mailing lists?• Very: everyone uses
mailing lists• Not much: we now use
RSS, blogs, Twitter, …
Not much: Correct! Evidence shows significant decrease in Web support team lists over past 5 years
Very: Correct! lis-* lists still popular despite small
![Page 10: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/10.jpg)
Who Are We Doing This For?Who should we be providing impact data for?
• Decision makers: so they have evidence on which service to fund from smaller pot
• Our funders: they can mandate this• Our peers: they can learn – and we should
support such learning and sharing• Our users: if our services are well-used, they’ll
be interested in this• Ourselves: we can learn from our successes -
and our failures
10
Moves towards greater transparency (e.g. salaries of senior managers; MP’s expenses; etc.) should be reflected by those working in libraries
![Page 11: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/11.jpg)
Case Studies
Let’s look at:• Metrics for use of various Social Web
services
And explore:• Metrics which can be determined for little
effort• Significance of the numbers• How the metrics can be gamed
Before:• Reviewing limitations• Making suggestions
11
![Page 12: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/12.jpg)
Blog Usage StatisticsUK Web Focus blog usage since launch in Nov 2007
12
Stories: • Preparation for workshop on SEO• Running tool to create PDF copy of blog posts
What does this tell us:• Ways of measuring & influencing nos. of visits (but
not necessarily indicative of purpose of service)
2010: 224 visits/day2009: 247 visits/day
![Page 13: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/13.jpg)
Blog Metrics: TechnoratiTechnorati indexes ~1 M English language blogs
• Nos. of links from other (indexed) blogs weighted by their own link popularity (cf Google Page Rank)
• Comparisons with peers
But:• Need to register blog• Sometimes breaks
13
![Page 14: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/14.jpg)
•Evidence of the importance of Twitter for driving traffic to blogs
The Evidence
14
UK Web Focus Blog: Total of 250,00+ views.Average 250/day over 3.5 yearsBut how do they arrive at the blog?
UK Web Focus Blog: Total of 250,00+ views.Average 250/day over 3.5 yearsBut how do they arrive at the blog?
![Page 15: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/15.jpg)
15
“The Power Of Passed Links”The Value Of Twitter Is In “The Power Of Passed Links” Wilson predicts that at current growth rates, Twitter “will surpass Google for many websites in the next year.” And that just as nearly every site on the Web has become addicted to Google juice, they will increasingly try to find ways to get more links from Twitter. Because Twitter equals traffic. …Moreover, he asserts that these Twitter links “convert better” than search links because they are often pre-filtered and come in the form of a recommendation from someone you are following.
•TechCrunch, June 2009
![Page 16: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/16.jpg)
Claiming a blog in Technorati
Note need to publish (temporarily) unique code to demonstrate ownership (useful to do at start).
16
Thank you for submitting your blog claim on Technorati. Technorati will need to verify that you are an author of the site http://blogs.ukoln.ac.uk/good-apis-jisc/ by looking for a unique code. We have just assigned the claim token 2XAVJZX29XXX to this claim.
![Page 17: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/17.jpg)
Blog Metrics: Wikio
Wikio: similar tool for blog rankings
Three UK HE blogs in top ~50 technology blogs
17
![Page 18: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/18.jpg)
18
Claiming a blog in Wikio
![Page 19: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/19.jpg)
There Are Limitations …League tables can be flawed
• Non-existent blog
• Excellent blog• Blog for
completed project (no recent posts)
19
Need for scepticism
![Page 20: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/20.jpg)
… And Benefits
Monitoring usage statistics can provide indications of successful strategies for user engagement:
• Something good happened on 19 Feb 2007! • This was date of post on email subscription
service to blog posts20
![Page 21: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/21.jpg)
Summary on Blog Metrics
We have seen:• There are services for providing blog
metrics• Signup up at blog launch can be useful• The services can be flawed• Monitoring and keeping records can help
to spot success stories
But:• What about limitations, ethics, …?• What about other Social Web services?
21
![Page 22: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/22.jpg)
Case Study 2a: Twitter
Twitoaster provides summaries of Twitter responses & RTs
22
![Page 23: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/23.jpg)
Exploring The EvidenceTwitter can be a significant driver of traffic
23
Where would blog be now without Twitter? Twitter seems to have decimated RSS traffic.
![Page 24: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/24.jpg)
Summarizr
24
Summarizr can provide statistics on event hashtag usage e.g. top twitterers, hashtags & URLs tweeted, etc.
Summarizr processes data stored in Twapper Keeper Twitter archiving service
![Page 25: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/25.jpg)
SummarizrSummarizr also provides information on geo-located tweets
25
Possible benefits:• Are they local
or remote?• Is geo-
location used?
• What are barriers?
![Page 26: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/26.jpg)
WikipediaThere is a lack of awareness of the usage statistics for Wikipedia articles
26
As in other examples the statistics may be flawed
Missing data
![Page 27: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/27.jpg)
Slideshare
27
How effective is Slideshare?
![Page 28: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/28.jpg)
Slideshare Evidence (1) •Carol Gole’s slides
28
Slides uploaded on behalf of Professor Carole Goble
![Page 29: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/29.jpg)
Slideshare Evidence (2) •Tweet from Steve Wheeler, 25 August 2010
29
What is a seminar? A mechanism for creating & delivering content which is consumed by others? When does the digital resource become more impact than the physical equivalent
![Page 30: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/30.jpg)
JISC VfM Report (2005)“Whilst a Value for Money exercise measures and compares costs, it must also take account of the mix of quality, resource use, fitness for purpose, timeliness and convenience to judge whether or not together, they constitute good value. As such, Value for Money is one measurement of good practice. Some elements under consideration may be subjective, difficult to measure, unavailable, intangible or misunderstood. Judgement is therefore required when considering whether value for money has been achieved.”
30
![Page 31: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/31.jpg)
TASI Example‘Value for Money’ exercise performed in 2005 suggested that advice from TASI saved user time and effort and that large savings were being made.
The following calculation was proposed: • Suppose each workshop attendance or enquiry
leads to a saving of one/person day. Suppose that each report read saves someone half an hour. Value a person/day at £109.
• A conservative estimate suggests that in 2004-05, for every pound of funding received, TASI saved the community at least £16 pounds.
31
An illustration of the need for evidence on which to base scenarios
![Page 32: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/32.jpg)
Facebook TrendsJun 2008:
“The Open University Facebook page is the top of all University pages, with 7,539 fans (2,874 a month ago). The other most popular UK Universities are Aston University (2,976 fans), Royal Holloway (1,765), Aberystwyth University (1,655 fans), University of Central Lancashire (1,475 fans), Keele University (1,420 fans), Cardiff University (1,357 fans) and the University of Surrey (1,166 fans).”
28 Sep 2010 (~start of academic year):
OU: 28,949 Aston: 8,445
Royal Holloway: 9,093 Aberystwyth: 7,326
UCLan: 7,982 Keele Uni: 6,716
Cardiff Uni: 18,689 University of Surrey: 8,06332
![Page 33: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/33.jpg)
Understanding Trends
What can be learn from Facebook trends • Evidence which can address ideology• Diversity of approaches (e.g. University
Freshers Fb page)• Identification of fragmentation of Facebook
presence• Inform discussions on resourcing, policies,
etc.
33
![Page 34: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/34.jpg)
Stories + StatisticsWe need (a) Evidence to support scenarios and (b) Stories to make evidence comprehensible
34
AssertionTwitter can help to develop professional relationships
StoryFollowing a Twitter response from @slewth I noticed her interests & a link to her blog.I read her blog posts – and exchanged tweets & emails.A few months later we had collaborated on a paper.At W4A 2010 paper was award a prize as the best communications paper
ResponseReally?Where’s the evidence?What is the cost?
![Page 35: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/35.jpg)
Bigger PictureMetrics are flawed as they don’t take into account:
• Diversity of uses of Social Web (e.g. blog for reflective thinking vs dissemination)
• Size of organisation and levels of resources• Temptation to ‘game the system’ and aim to
maximise metrics rather than underlying purpose
We could:• Refuse to engage with current tools & approaches• Wait for the perfect analytic framework
But:• We’ll lose out in understanding current approaches• Is there a perfect framework?• What if evidence is needed to justify existence?
35
![Page 36: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/36.jpg)
The Bigger Picture: OU“… it’s a whole new generation of engaging digital content. 6.12m downloads, 64k visitors, 180k downloads a week. Lots of top-20 hits. Over 50% outside the UK. How much does it cost? Very cost-effective versus putting signs on the side of buses.”
Martin Bean, OU VC36
BBC News, 29 June 2010
![Page 37: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/37.jpg)
Amazon Rankings
Amazon provides rankings for its books, even though they have different purposes, audiences, …
37
![Page 38: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/38.jpg)
Manipulating Statistics
Some suggestions:• Create blogs which have links back to
your services• Automatically submit comments to open
Social Web services containing links• Use keywords for your competitors• Give your content away with a CC
attribution licence• Keep an eye out for queries and link to
your services when appropriate• Include your URL in your email signature
38
![Page 39: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/39.jpg)
EthicsConcerns:
• There are lies, dammed lies and statistics• Metrics can be gamed
Ethical approaches:• Openness:
Being honest about limitations of metrics Being open about the purposes of the services
• Diversity of approaches to evaluation: Provide a range of statistics Statistics plus stories
• Staff development: Ensure staff are aware of need for metrics Ensure that are aware of ethical approaches39
![Page 40: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/40.jpg)
Policies
40
Can you measure the effectiveness of a service if you don’t have a clear idea of its purpose?
There’s a need for (simple) policies on purpose of Social Web Services
![Page 41: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/41.jpg)
Maximising Impact
How do you go about maximising impact?• Create content!• Empower your staff• Join in the discussions• Go where the discussions take place• Monitor the impact and identify success
stories
41
![Page 42: Monitoring and Maximising Organisational Impact using the Social Web](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813714550346895d9e9de2/html5/thumbnails/42.jpg)
42
Questions
Questions are welcome
Name: Brian Kelly Address: UKOLN, University of Bath, BATH, UKEmail: [email protected] site: http://www.ukoln.ac.uk/Blog: http://ukwebfocus./wordpress.com/Twitter: briankelly and ukwebfocus (automated feed)
Name: Brian Kelly Address: UKOLN, University of Bath, BATH, UKEmail: [email protected] site: http://www.ukoln.ac.uk/Blog: http://ukwebfocus./wordpress.com/Twitter: briankelly and ukwebfocus (automated feed)