Now, about that filter...Managing scientific information overload on the web

NFAIS Meeting 28 February 2010, Philadelphia

Some of the people who contributed to this presentation...

Helen BermanLorie LeJeune

Iain Emsley

Neil Saunders

Brian Kelly

Harry Collins

Michael Nielsen

Jen Dodd

Greg Wilson

Timo Hannay

Maxine Clarke

Jenny RohnRicardo Vidal Paulo Nuin

Jenny HalePeter Murray-Rust

Deepak Singh

Jon Udell

Tim O’Reilly

David Crotty

Rafael Sidi

Richard Akerman

Jean-Claude Bradley

Mike Ellis

Liz Lyons

Andy Powell

Gavin Baker Peter Suber

Victor Henning

Sabine HossenfelderFlickr

Google

Steve Wilson

Andrew Milsted

Frank Norman

Dave de RoureJeremy Frey

John Cumbers

Bill Flanagan

ISIS LSS Group

Lakshmi Shastry

Catherine Jones

ISIS Computing GroupSTFC

Plausible AccuracyJohn

Dupuis

Chad Orzel

Ken Shankland

Martyn Bull

Jonathan Gray

Rufus Pollock

Clay Shirky

Kevin Kelly

Gavin Bell

Shirley Wu

Euan Adie

Richard Curry Ian Mulvany

Jamie McQuay

Atilla Csordas

Pawel Szcsesny Gabriel Cavalli

Matt Wood

TIM HUBBARD

DUNCAN HULL

Richard Grant Branwen Hide

PLoS

Friendfeed

Bora Zivkovic

Peter Binfield

John Wilbanks

Kaitlin ThaneyThe BioGang

Tony WilliamsEgon  Willighagen

Martin FennerYaroslav Nikolaev

Jon EisenMichael Eisen

Richard Akerman

Jeremiah FaithMichael Barton

Lee Smolin

Garret Lisi

Victoria  Stodden

Simon ColesTony Hey

Noel Gorelick

Jon Tansley

Benjamin Good Dorothea Salo

Paul Walk

Mitch Waldrop

Björn Brembs

Rich Apodaca

Bill Hooker

Pedro Beltrao

Mat Todd

SciFoo 2008/9

campers

Stephen Brenner

Brian Matthews

Allyson Lister

Phil Lord

Steve Koch

Koch Lab

Carole Goble

Stephen FriendEva Amsen

JOHN WILLINSKY

TIM HUBBARDTIM HUBBARDTIM HUBBARD

Steph Hannon

Rebecca Goulding

Leigh Dodds

Paul Miller

Mark BorkumDan Hagon

Jim Downing

Nico Adams

@gnatFabiana Kubke

Hope LemanLisa Green

Ariel Waldmann

@tGrace Baynes

Simon Philips

Matt Johnson

Lee Dirks

Microsoft

NPG Ben Goldacre

Arfon Smith

Nicholas Cole

Chris Leonard

PIERRE LINDENBAUM

ALAN CANN

Jo Badge

Mummi Thorissson

Andrew Kasarskis

Glyn Moody

@communicating

PT Sefton

Andrew Farke

Back to them later...

Me: A brief history

Finished highschool 1990...

http://www.flickr.com/photos/elsie/105716382 CC-BY

Undergrad 1991-94

First email addresss -1991

http://www.flickr.com/photos/bike/3046589822 CC-BY-SA

“You need to spend half a day a week in the library reading the new journals”

My project supervisor, 1994

http://www.flickr.com/photos/stevecadman/486261295 CC-BY-SA

PhD 1995-99

Discovered web - 1995

http://web.archive.org/web/19990421174831/www.sciencemag.org/

http://web.archive.org/web/19990208214440/http://ncbi.nlm.nih.gov/pubmed/

Over the course of my PhD...

...day to day search went from...

http://www.flickr.com/photos/stevecadman/486261295 CC-BY-SA

...to...

http://web.archive.org/web/19990208214440/http://ncbi.nlm.nih.gov/pubmed/

And on top of that...

...data. No longer just papers

0

25,000,000,000

50,000,000,000

75,000,000,000

100,000,000,000

1982 1986 1990 1994 1998 2002 2006

Submissions to Genbank

0

1.25

2.50

3.75

5.00

1982 1986 1990 1994 1998 2002 2006

Average Capacity of Human Scientist

By 2001/2...

...everyone I know is subscribed to TOC alerts

...and no-one I know is reading them...

Information overload...

http://www.flickr.com/photos/dylanroscover/3450505729 CC-BY-SA

So how are those filters getting on?

http://www.flickr.com/photos/daveyll/332723930

Search is (by far) the dominant filter

So the state of the art is an RSS feed of a text search...

...and that’s only on the abstracts not the full text

...which is a bit

http://www.flickr.com/photos/contortyourself/3902224062 CC-BY-SA

So do I..?http://www.flickr.com/photos/vagawi/3155400274 CC-BY

Absolutely not!

Sidestep: Let me...

An assertion.

In the area of social web tools for scientists I am confident that every signif icant (public) document and announcement crosses my attention stream.

Without active searching.

http://friendfeed.com

http://tinyurl.com/dku869

A shared social net

Sharing the loadhttp://www.flickr.com/photos/joi/2941559903 CC-BY

Social aggregation...

...but still a filtering problem

Each interaction adds value...

http://www.flickr.com/photos/qnr/1263648697 CC-BY-SA

...and each interaction measures value...

http://www.flickr.com/photos/pinksherbet/3209939998 CC-BY

Collaborative aggregationCollaborative abstraction

Collaborative indexingCollaborative prioritisation

http://www.flickr.com/photos/luc/2515255357

Based on the network I built

...but...

...will only work where a community exists

http://www.flickr.com/photos/mararie/3313582639/ CC-BY-SA

...and shares a set of tools

http://www.flickr.com/photos/batega/1596898776 CC-BY

...or rather...a frameworkhttp://www.flickr.com/photos/sparker/290754127 CC-BY

We’re a long way from the ideal...

So where next?

http://www.flickr.com/photos/davidmasters/2884480103 CC-BY

We need...

...better tools for aggregation, summarisation,

integration...

•Rapid•Relevant•Filtered•Digestible•Interoperable•Comprehensive

...tools for building, maintaining, and measuring networks...

http://www.flickr.com/photos/luc/2515255357

...if I’m relying on the network...

http://www.flickr.com/photos/mgifford/3558463424 CC-BY-SA

...tools to summarise, filter, and integrate diverse

sources of information...

...abstracts, summaries, indexes of people and the

tools to help me use them to build the network I want...

Who might be well placed to do that?

http://www.flickr.com/photos/richardmoross/3947406286 CC-BY

Cameron.Neylon@stfc.ac.ukhttp://blog.openwetware.org/scienceintheopenhttp://slideshare.net/cameronneylonTwitter: @cameronneylonFriendfeed: cameronneylon

Thanks to:Sciencetwists, Friendfeeders, and the wider online community for ideas, criticism, and conversations.

Deepak Singh, Larry Lessig, Andy Powell, and John Wilbanks for presentation inspiration.

Flickr for images