citation metrics and the stories they tell
TRANSCRIPT
International Symposium on the Science of Science���Library of Congress���
March 21st 2016
Citation networks and the stories they tell
Carl T. BergstromUniversity of Washington
Jevin West Martin Rosvall
SciSIP
Jennifer Jacquet Jacob Foster Shelley CorrellMolly King Daril Vilhena Ted BergstromJames Evans Ben Althouse Moritz StefanerDaniel Edler Ian Wesley-Smith Rodney GarrettMichael Jensen Morton Bech Ralph DandreaGregg Gordon
Eigenfactor.org/projects/well-‐formed/
Citation is a core institution of ���academic science, tracing the flow of
ideas over time.
The sum of all citations create a vast network of more than a billion citations among more than 100 million papers
Eigenfactor.org/projects/well-‐formed/
Every one of those citations represents a careful decision by domain experts.
Eigenfactor.org/projects/well-‐formed/
The citation network of science holds a wealth of information about how science
works, and how it can work better.
Eigenfactor.org/projects/well-‐formed/
How can we extract ���this information?
Eigenfactor.org/projects/well-‐formed/
The first step is to assemble the data. ������
We have compiled citation networks ���from many sources:
���How important is any particular paper, or any particular journal,���
in the network?
Mapequation.org
Count incoming links���(Impact Factor)
Mapequation.org
Count incoming links���(Impact Factor)
Use the whole network(Eigenfactor)
Mapequation.org
Important websites���are linked to by���
important websites.
Important papers���are cited by ���important papers
Important journals���are cited by ���
important journals
Eigenfactor algorithmP = α H + (1 − α ) a.eT
Matrix representing therandom walk over citations Probability of
not teleportingCross-citation Matrixdictating the structureof the citation network
Probability of teleportingto completely new journalweighted by the numberof articles in that journal
EF =100 Hπ[Hπ ]ii∑
Leading eigenvectorof the random walkmatrix P.
Normalization
Bergstrom (2007); West et al (2010)
Applet coding: Daniel EdlerMapequation.org
The Eigenfactor Algorithm
Study, and publicize, the cost-effectiveness of journal subscriptions
Eigenfactor.org Bergstrom and Bergstrom 2004 PNAS
Study, and publicize, the cost-effectiveness of open access publishing
Eigenfactor.org
Ranking authors
“Author-level Eigenfactor performs best in identifying high-impact authors”���
- Dunaiski et al. ��� J. Informetrics May 2016
West et al 2013 JASIST
Ranking articles: ���The Article-Level Eigenfactor (ALEF) Algorithm
Time
Olderpapers
Newer papers
Wesley-Smith et al 2016; West et al in prep.
Image Courtesy of Mark Newman
Small networks reveal structuredirectly.
Dating network in a Michigan high school
Large networks could use some assistance.
Yeast protein interaction network
Ho et al. (2002) Nature
good maps simplify ���and highlight��� relevant structures
Boston MTAGoogle maps
Network community detection ������
We want a modular description of a weighted, directed network: ���
���Most flow on the network occurs within, ���
not between, local modules.
DataCompressing Finding patterns
If we can find a good code for describing flow on a network, we will have solved the dual problem of finding the important structures with respect to that flow.
The map equation tells us the description length for a particular modular structure
The map equation
We conclude that the infomap method by Rosvall and Bergstrom is the best performing… ���
Among other things, the method can be applied to weighted and directed graphs as well, with excellent performances, so it has a large spectrum of potential applications.”
- Lancichinetti and Fortunato (2009)
“
Rosvall and Bergstrom (2008) PNAS
Althouse et al (2009) JASIST
“coverage” Impact factor
Althouse et al (2009) JASIST
1995 2004
1. Determine which structures are statistically significant.���
2. Visualize changes in those structures.
Rosvall and Bergstrom (2010) ���PLoS One
The emergence of neuroscience
The map equation tells us the description length for a particular hierarchical structure
The hierarchical map equation
Rosvall and Bergstrom (2011) ���PLoS One
Rosvall and Bergstrom (2011) PLoS One
Revealing hierarchical structure
Rosvall and Bergstrom (2011) PLoS One
Revealing hierarchical structure
Rosvall and Bergstrom (2011) PLoS One
Revealing hierarchical structure
Rosvall and Bergstrom (2011) PLoS One
Revealing hierarchical structure
Rosvall and Bergstrom (2011) ���PLoS One
Revealing hierarchical structure
Using hierarchical structure for scholarly recommendation
West et al (2016) In press
http://babel.eigenfactor.org
Using hierarchical structure for scholarly recommendation
1920 1940 1960 1980 2000
0.10
0.15
0.20
0.25
0.30
perc
enta
ge o
f wom
en
What gender disparities still exist across academia?
first author
West et al. 2013 PLoS One
1920 1940 1960 1980 2000
0.10
0.15
0.20
0.25
0.30
perc
enta
ge o
f wom
en
What gender disparities remain ���in scholarly publishing?
last author
first author
West et al. 2013 PLoS One
Eigenfactor.org
Self-citation rate by gender
●
●
●●●
●●
●
●●●
●●
●●
●●●
●
●●●●
●●●
●●●●
●●●●●●●●●
●●●
●●●●●●●●●●●●●●●
●
●●
●●
●●●
●●●
●●●●●●
●
●●●●●
●●●●●
●●●●●●
●●●●●●●
●●●●●●●●●●●●●
●●●●●●
●●●
●
●
●●
Women's and men's rates of self-citation
���� ���� ���� ���� ���� ��������
���
���
���
���
���
���
����-����� / ����������
Based on > 3 million papers from JSTOR King et al. in prep.
Rates of rates of self-citation do make a difference to impact metrics, particularly the h-index.
– Cameron et al 2016 Bioscience
Self-citations per authorship
”“
King et al. in prep.
