cobaltmetrics en - cloudinary€¦ · journal to journal: web of science, scopus doi to doi:...
TRANSCRIPT
![Page 1: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/1.jpg)
CobaltmetricsLuc Boruta & Damien Vannson — Thunken Inc.
[email protected] — @thunkenizerPUBMET2019, Zadar, 2019/09/20
Web-Scale Citation Tracking
cobaltmetrics.com
http
://gp
h.is
/XI8
Wen
![Page 2: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/2.jpg)
cobaltmetrics.com
http
://gp
h.is
/XI8
Wen
![Page 3: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/3.jpg)
Dear Santa
cobaltmetrics.com
http
://th
einc
lusi
ve.n
et/a
rticl
e.ph
p?id
=268
![Page 4: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/4.jpg)
cobaltmetrics.com
http
://gp
h.is
/1N
XR
Xtc
![Page 5: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/5.jpg)
Attention vs. Impact
Citations and altmetrics are proxies for impact.
Citations and altmetrics measure attention.
Attention correlates w/ impact. So do influence and privilege.
Mentions and events are merely newish types of citations.
cobaltmetrics.com
![Page 6: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/6.jpg)
A partial landscape of citation aggregators
● Journal to journal: Web of Science, Scopus● DOI to DOI: OpenCitations● URL to DOI: ALM/Lagotto, Crossref Event data● URL to URL: Altmetric, Plum, Cobaltmetrics
cobaltmetrics.com
![Page 7: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/7.jpg)
Common issues with citation aggregators
● Imbalanced datasets○ Predefined lists of supported research outputs○ Predefined lists of supported languages
● Irreproducible indicators○ Dependency on 3rd party servers (short URLs, APIs)
cobaltmetrics.com
![Page 8: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/8.jpg)
Why should we care?
cobaltmetrics.com
Metrics are a sampling game.
Imbalanced datasets reinforce discrimination.
We are interested in low-frequency phenomena,and in distinguishing structural zeros from sampling zeros.
![Page 9: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/9.jpg)
Weapons of math destruction
cobaltmetrics.com
“There is a moral obligation to challenge machine biases.”— Heather Staines, PIDapalooza’19
Algorithmic bias reflects the values of the humans involved in designing the algorithm and/or collecting the data.
![Page 10: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/10.jpg)
http
s://g
ph.is
/2xg
F3te
cobaltmetrics.com
![Page 11: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/11.jpg)
Cobaltmetrics
It is not up to citation aggregators to decide what is citable, our role is to observe all citation patterns on the web.
The web is not FAIR (and will most likely never be)and that is just fine.
cobaltmetrics.com
![Page 12: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/12.jpg)
Cobaltmetrics
Cobaltmetrics crawls the web to indexhyperlinks and PIDs as first-class citations.
The web is our corpus, and our URI transmutation API collates citations to all known versions of a document.
cobaltmetrics.com
![Page 13: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/13.jpg)
Design rationale
Cobaltmetrics tracks all URIs, URLs, and typed PIDs.
Cobaltmetrics can only be queried by URIs.
Cobaltmetrics will never create new identifiers.
Cobaltmetrics will never create new metrics.
cobaltmetrics.com
![Page 14: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/14.jpg)
Design rationale
✔ Lawrence et al., 2001, https://doi.org/10.1109/2.901164✔ http://dx.doi.org/10.1109/2.901164✔ doi:10.1109/2.901164✔ https://ieeexplore.ieee.org/document/901164/✔ https://bit.ly/2kEavO1✘ Lawrence et al., 2001
cobaltmetrics.com
![Page 15: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/15.jpg)
Better a URL today than a PID tomorrow
cobaltmetrics.com
The ideal identifier should be persistent,findable, accessible, interoperable, and reusable...
...we all copy-paste from the address bar of our browser.
![Page 16: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/16.jpg)
PIDs are not silver bullets
cobaltmetrics.com
There are billions of documentsthat will never get DOIs or any other fancy PID:old documents, grey literature, and the rest of the web.
There are tons of documents with PIDs that are citedwith no mention of their PIDs.
![Page 17: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/17.jpg)
Compact IDs vs. good old URLs
cobaltmetrics.com
Cobaltmetrics’ citation index (February 2019):
● HTTP+HTTPS+FTP: 256 million URLs (98%)● Every other scheme: 4 million IDs
![Page 18: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/18.jpg)
cobaltmetrics.com
http
://gp
h.is
/2O
XLM
RE
![Page 19: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/19.jpg)
Are your metrics alt- enough?
cobaltmetrics.com
NO.
![Page 20: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/20.jpg)
Are your metrics alt- enough?
● Bias in favor of English● Bias in favor of traditional publication venues● Bias in favor of traditional publication formats● Bias in favor of short-term rewards (vs. long-term goals)● …?
cobaltmetrics.com
![Page 21: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/21.jpg)
Selection biases: Wikipedia languages
cobaltmetrics.com
Altmetric: 3 languages (en, fi, sv)
PlumX Metrics: 3 languages (en, es, pt)
ALM: 25 most popular languages
Cobaltmetrics: 180+ languages!
![Page 22: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/22.jpg)
Selection biases: document types
cobaltmetrics.com
Strong focus on traditional peer-reviewed publications. Preprints are still treated as second-class documents.
What about patents, clinical trials, law articles, etc.?What about non-textual objects, e.g. datasets or software?
In Cobaltmetrics a URL is a URL, we do not discriminate.
![Page 23: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/23.jpg)
Selection biases: PIDs vs. URLs
cobaltmetrics.com
http
s://g
ph.is
/2N
ehB
G5
Nothing lasts forever on the web:
● Link rot!● Content drift!● Outages!
![Page 24: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/24.jpg)
Non-canonical URIs
cobaltmetrics.com
Non-canonical URI ≈ any ID that is not 100% FAIR,including but not limited to:
● Short URLs● Proxy URLs● Sci-Hub URLs
![Page 25: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/25.jpg)
URI transmutation
cobaltmetrics.com
Transmutation = normalization + conversion
● Equivalencies we can compute (e.g. ORCID⇄ISNI)● Equivalencies we must learn (e.g. short URL⇄URL)
Our transmutation API is open and free, try it out!
![Page 26: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/26.jpg)
URI transmutation example
cobaltmetrics.com
We remix 4M cliques of IDs from ORCID’s Public Data File.
Example:
● orcid:0000-0003-0557-1155 → {scopus:55148973700}● scopus:55148973700 → {orcid:0000-0003-0557-1155}● mailto:[email protected] → {orcid:0000-0003-0557-1155, scopus:55148973700}
![Page 27: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/27.jpg)
A note on reproducibility
cobaltmetrics.com
Because we aggregate data from different sources,there are many moving parts.
Our default strategy is to ingest the entire datasets,so that we control when and how data gets updated.
Our API can return a fingerprint of the whole database,as well as the log of all the web resources we remix.
![Page 28: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/28.jpg)
cobaltmetrics.com
http
://gp
h.is
/2JC
xAbw
![Page 29: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/29.jpg)
Web-scale citation tracking
cobaltmetrics.com
● Wikimedia (all projects, all languages)● StackExchange/StackOverflow (all projects, all languages)● US legal opinions (via CourtListener)● Hypothes.is annotations● Usenet posts (via the Internet Archive)● CommonCrawl (3.1 billion webpages)
http
s://c
obal
tmet
rics.
com
/doc
s/pa
ge/d
ata-
sour
ces
![Page 30: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/30.jpg)
Web-scale citation tracking: transmutation
cobaltmetrics.com
● Crossref● ORCID● PMC● Terror of Tiny Town● Unpaywall● Wikidata● ...
http
s://c
obal
tmet
rics.
com
/doc
s/pa
ge/d
ata-
sour
ces
![Page 31: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/31.jpg)
Cobaltmetrics in the context of open science
cobaltmetrics.com
● Currently mostly closed-source, but...● Everything on the website (data/docs) is now CC BY 4.0● Coming soon:
○ No more third party trackers○ Pricing transparency
![Page 32: Cobaltmetrics en - Cloudinary€¦ · Journal to journal: Web of Science, Scopus DOI to DOI: OpenCitations URL to DOI: ALM/Lagotto, Crossref Event data URL to URL: Altmetric, Plum,](https://reader033.vdocuments.us/reader033/viewer/2022050214/5f6033b1f185517a462549ad/html5/thumbnails/32.jpg)
cobaltmetrics.com
http
://gp
h.is
/XI8
Wen