Download - Trusting wikipedia
Trusting WikipediaSu-Laine Yeo Brodsky
December 2014
Today’s Talk
• How does
Wikipedia vet its
own content?
• Questions are
welcome
anytime, or at the
end
Image:Robert Lawton, distributed under a CC BY-SA 2.5 license
The Encyclopedia Anyone
Can Edit
• Hosted by the non-profit
Wikimedia Foundation,
in 250+ languages
• Almost anyone is
encouraged to edit
almost all text and
media on the site
• ~10% of edits are
reverted (rejected)
Image: xkcd.com, distributed under a CC BY-NC 2.5 license
What Happens When You
Edit This Page?
Right Away
Exceptions: Edits You Can’t
Save
• URLs on Wikipedia’s spam blacklist
• Use of certain images, except in particular articles
• Certain types of blatant vandalism
Page History
User Contributions Page
Accounts used only
for vandalism are
easy to spot
Summary: Wiki Mechanics
• All edits are tracked, and
summarized in the article
History page
• All edits can be tracked to an
IP address or username
• Edits can be quickly
reverted by another editor
People and Processes
Roles at Wikipedia
• Editor: Anyone who writes or changes articles, or
uploads images
• Administrator: 1,386 elected administrators have
additional tools:
• Can delete/undelete articles or past versions of
articles
• Can block editors or other administrators
• Can “protect” an article to prevent editing
Reality: For the most part,
everybody has equal privileges
when it comes to controlling article
content
Assumption: Moderators decide
what changes to an article will be
accepted
How Wikipedia Monitors
Edits: A 2-Stage Process
Stage 1: Recent Changes
Page
30 seconds worth
of edits to all
English Wikipedia
articles
Link to “diff”
showing what text
was changed in this
edit
Link to list of editor’s
other edits
Heading of the article
section that was
changed
Editor’s
explanation for
the edit
Red indicates
editor is new
Recent Changes Patrol
• Volunteer patrollers and
robots monitor all
changes to the site
• Quickly revert blatantly
inappropriate changes
Stage 2: Watchlists
Changes to articles
I’ve chosen to
watch, grouped by
date
Watchlists
• Each user has a watchlist of
articles they are interested in
• The Watchlist page shows
recent changes in those
articles
• Primary tool for fact-
checking
Will an Edit Stick?
• Usually decided by the editors who watch the
page
• If no consensus, the editors who watch the page
ask the wider Wikipedia community for input and
mediation
Reality: Wikipedia articles summarize
what reliable published sources say
on the subject…
…as determined by the consensus of
editors who show up
Assumption: Wikipedia articles
summarize what a majority of its
editors on the subject believe
How Editors Scrutinize
Changes
Consider:
• My own knowledge
• Sources cited?
• Who is the editor?
• What does the source say?
Userpage of a Wikipedia
editor
Image: Userpage of Kim Dent-Brown on Wikipedia
• Improve instead of reverting
Dispute Resolution
• Discuss the issue on
the article Talk Page
• Ask the wider
community for input
• Problematic editors
can be blocked by
administrators
What Can Go Wrong?• Patrollers/watchlisters
not paying attention
• Watchlisters lacking the
necessary expertise
• Persistent, highly
motivated agenda-
pushers
Perspective: Core Facts
In an article with a reasonable number of watchers, the core
facts of the article tend to be reliably monitored.
“West Bengal
is a state in
the eastern
region of
India.”
(no source given)
Wikipedians: Question 1Wikipedia currently has 31,000 active, registered
editors. According to surveys of Wikipedia editors,
what percentage are male?
A. 85%
B. 60%
C. 45%
Wikipedians: Question 2Which of the following statements is false?
A. According to editor surveys, more than 10% of
Wikipedia editors are under 18 years old
B. Anyone can track down the IP address that
each edit comes from
C. According to editor surveys, about half of
Wikipedia editors have some post-secondary
education
Systemic Bias• Demographic bias:
• Technical
• Male
• Childless
• First world
• Recentism: Recent
news coverage &
online sources
Summary: People and
Processes
• All editors are equal, in
theory
• Articles should draw only
from reliable published
sources
• 2-stage review process
means that subtle problems
last longer than obvious
ones
Image: Adam Novak, distributed under a CC BY-SA 3.0 license
Wikimania Conference 2013
Summary (cont’d)
• In an article with a
reasonable number of
watchers, the core facts
tend to be relatively
reliable
• Wikipedia has systemic
biases stemming from
community demographics
Trends in Trust
Image: Mariuszjbie, distributed under a CC BY-SA 3.0 license
How Many Watchers?
Article# of
Watchers
Barack Obama 2,592
Angela Merkel 225
Influenza 228
Cephalexin 46
Many articles on living people <5
Four Trends
1. More complaints from the subjects of Wikipedia
articles, even articles with very low readership
2. Higher expectation to cite sources when adding
new content
3. More complete content -> Ratio of good to bad
edits changes
4. More sophisticated PR from organizations
Problematic Edits in Medical
Articles
Source: https://en.wikipedia.org/wiki/Wikipedia:Conflicts_of_interest_%28medicine%29
Looking Out for Bias• Would any personal,
corporate, or ideological
interest benefit from a
certain article slant?
• Does the article Talk page
and/or history show any
concerns with conflict-of-
interest editing or disputes
over neutrality?
Buy!
• Libel
• Hoax
• Advertising
• Pseudoscience
• Propaganda
Don’t bite the
newbies
Reject questionable
edits
Many edits by new
editors are imperfect
and do not cite
sources
A Reversible Trend?
• Community needs to
grow to:
• Maintain quality and
update facts
• Reduce systemic bias
• Requires Wikipedia
community to be skilled
in both skepticism and
openness
Enjoy Wikipedia
• Reliability is variable
• Project reflects huge
volunteer effort
• Future depends on a
strong editor
community
Image: Takeaway, distributed under a CC BY-SA 3.0 license
Thank You!Any questions?
Su-Laine Yeo Brodsky
www.interelement.ca
@sulaineyeo