lightning talk, weber: seeing is believing, but measuring is knowing: quantifying the impact o…

<TongueInCheek>Seeing is believing, but measuring is knowing</TongueInCheek>

Nic Weber

Thursday, April 4, 13

2 things


First thing


There is nothing “soft” about metric incentives


History of Political Economy 20:4 0 1988 by Duke University Press CCC 00 18-2702/88/$1.50

The market for fame and fortune David M. Levy

The Problem The modem discussion of the economics of fame treats fame as an instru- ment to material income. This view I should like to contrast with the classical Greek view of the maker. While the classical Greek discussions sometimes do look upon fame as a means to material income, fame itself is seen as a good thing; indeed, it is such a good thing that it can allow mortal man to rival even the undying gods themselves. Although fame cannot compensate for death, fame is the closest mortal man can come to real immortality.’ If fame and material income are both ultimate ends, might not there be a tradeoff between the two?

The thrust of the article is to explore the possibility that there is a com- plicated tradeoff between scientific fame and material income. The specific problem addressed is whether scientists in an amateur period of their dis- cipline will behave differently than scientists in a professional period.

The Model Modem economic research on the incentives to publish economic ar-

ticles has established an important point which adds considerably to the Greek discussion. Scientific fame can be viewed as an investment: one

Correspondence may be addressed to the author, Center for Study of Public Choice, George Mason University, Fairfax VA 22030.

1. Vermeule (1979, p 23): “Unfortunately, the psyche usually lost creative powers after death, and so was helpless in comparison with divinity and nature. This was the source of the familiar old quarrel between Simonides and Kleoboulos of Lindos. Kleoboulos thought men could make things-a bronze statue, a carved stele, a poem-which would not waste away or be extinguished ‘so long as water flows or the tall trees bloom, so long as the sun goes up to shine, and the brilliant moon, so long as rivers flow and the sea-waves surge on the shore.’ That was immortal memory, fame, kleos uphthiton, the oldest ambition in Greek poetry. Creativity was a magic drug, a pharmukon against death and against being forgot- ten. Simonides though he saw a tougher truth, that rivers, flowers, sun, moon and waves were immeasurably stronger than human creation because they were divine. It was idiocy to ‘set the strength of a stele against the flame of the sun and the gold moon and the whirling sea’ because our work is inescapably mortal. Still, in the short time between the Greeks and ourselves, rivers have dried up and oceans shifted while many Greek pharmaka against death are still potent.”

2. My interest in this problem was stimulated by Stigler 1949. Stigler rightly observed that the classical economists focused on a different sort of problem than did those economists who followed.

615

Science is driven by curiosity, AND fame and fortune

Science as a gift giving economy

Google “Not Hagstrom”


Vanity (fame) metrics are powerful


Don’t we already do this?


Sort of...


We don’t do it that well


We certainly don’t do this


For the most part, we treat repository metrics

like this not like this


Second thing


To do this well we should focus on...

Usage Metrics!=

Data Citation Metrics


Hold up...


“We’ve been promoting data citations for this!!!!!!!!!!!!”


Good


other people are on that...


but citations (alone) won’t do that...(no no, no they won’t do that)


The Data Usage Index


Indicator ExplanationUnique Users (UU) Unique users that downloaded data during a time window

Unique Users - Advanced UUs that accessed data programmatically

Unique Users - Assisted UUs that accessed data via GUI or Service

Number of Datasets Number of Datasets assigned DS number Files DS Number of files in Dataset per time window

Download Frequency Total number of files downloaded per time window

Download Frequency - Advanced Files downloaded by Advanced users

Download Frequency - Assisted Files downloaded by Assisted users

Homepage Hits Dataset Homepage Hits per time window

Homepage Hits - Direct Access Dataset Homepage Hits per time window by users with direct access (link not indexed or retrieved by search)

Homepage Hits - With Link Dataset Homepage Hits per time window by users with link (from indexed list or retrieved by search)

Subset Requests Subsets Requests per time window

Download Density Average number of files downloaded per UU

Usage Impact Total number of downloaded files over total files in dataset

Usage Impact - Advanced “

Usage Impact - Assisted “

Interest Impact Total homepage hits per number of files in dataset

Usage Balance Files downloaded by number of homepage hits per time window

Subset Ratio Number of subset requests over total number files downloaded per time window

Secondary Interest Impact Homepage over UU

Coi

nci

den

tLe

adin

gLa

gg

ing


And the thing about usage metrics are...


0"

50"

100"

150"

200"

250"

300"

Numbe

r'of'fi

les'd

ownloa

ded''(n

='433)'

Change'in'download''frequency'(3'month'skip)'

Files"downloaded"by"Advanced"users"

Files"downloaded"by"Assisted"users"

Subsets"Requests"of"Dataset"Per"?me"window"

0"

2"

4"

6"

8"

10"

12"

14"

16"

18"

Usage&Im

pact&Score&&

Usage&Impact&(3&Month&Skip)&&

ds083.2"

ds083.2","advanced"

ds083.2","assisted"

... (alone) they can measure curation impact


Thanks!

[email protected]

@nniiicc


mailto:[email protected]

mailto:[email protected]

lightning talk, weber: seeing is believing, but measuring is knowing: quantifying the impact o…

Documents

scientific fame

economics of fame

fortuneand fame

fame itselfscience

link dataset homepage

number of homepage hits

vanity fame metrics

searchhomepage hits