scraping the social graph with ushahidi and swiftriver
DESCRIPTION
As delivered by Jon Gosier at Georgetown University on June 28th.TRANSCRIPT
![Page 2: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/2.jpg)
About UshahidiUshahidi is a free, open-source platform used for crowdsourcing and visualizing data geospatially. It was born out of the 2008 election unrest when founders Juliana Rotich, Erik Hersman, Ory Okolloh and David Kobia wanted to allow Kenyan citizens a way to SMS reports of incident to know what was occurring around them. This was one of the earliest uses of crowdsourcing for crisis response.
Notable UsesUshahidi has been deployed in major global crisis scenarios, allowing organizations to draw situational awareness from the crowd. To date it ’s been downloaded over 15,000 times.
Some of the more notable deployments include recently in Egypt, the Haiti earthquakes, the fires in Russia, the Queensland floods in Australia.
The ChallengeA s t h e a m o u n t s o f d a t a aggregated by Ushahidi users grows, they face a common problem. How do they effectively manage this realtime data? How can we help them discover credible and actionable info from the deluge of reports they’ll get from the public? The SwiftRiver initiative was created to begin to answer some of these questions for Ushahidi deployers.
![Page 3: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/3.jpg)
![Page 4: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/4.jpg)
USHAHIDI HAITI
![Page 5: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/5.jpg)
OILSPILL CRISIS MAP
![Page 6: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/6.jpg)
UCHAGUZI
![Page 7: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/7.jpg)
RUSSIAN FIRES “HELP MAP”
![Page 8: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/8.jpg)
PAKREPORT
![Page 9: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/9.jpg)
TUBESTRIKE CROWDMAP
![Page 10: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/10.jpg)
PRAGUEWATCH
![Page 11: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/11.jpg)
HARASSMAP
![Page 12: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/12.jpg)
U-SHAHID
![Page 13: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/13.jpg)
CHRISTCHURCH
![Page 14: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/14.jpg)
SINSAI.INFO
![Page 15: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/15.jpg)
“It’s not information overload. It’s filter failure.”
- Clay Shirky
![Page 16: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/16.jpg)
PLATFORM GOALS
Consider the context, relevance defined by the user
Offer an opt-in global database of trust and authority
Algorithms augment, but not define, human decision making
Work across media channels (Twitter, Email, Feeds, SMS)
Be accessible (offline/online/mobile)
Index massive amounts of the mobile/social web
![Page 17: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/17.jpg)
KNC AWARD & RIVER ID
final component of the veracity algorithm
needs to be able to scale massively
changing the backend (Hadoop & Mongo DB)
research by data scientists
use-cases at scale and iterative improvements
![Page 18: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/18.jpg)
THIS IS A DATA PROBLEM
![Page 19: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/19.jpg)
![Page 20: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/20.jpg)
PROGRESS
7,000+ downloads in 6 months
7,000+ API Users
100,000+ Lines of code
5 APIs and 2 Apps
Data Items Processed - 70,000,000 (liberal extrapolation)
![Page 21: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/21.jpg)
![Page 22: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/22.jpg)
![Page 23: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/23.jpg)
Sweeper - User Interface
![Page 24: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/24.jpg)
NETWORK DYNAMICS
Good crowdsourcing campaigns build upon the existing ties between people and their networks. There’s a natural mult-iplier, where the people in the original network become nodes for new networks and so on.
![Page 25: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/25.jpg)
❖ Participation is permission❖ Consent is not carte blanche❖ Clarity is critical❖ Trust is Earned or Burned❖ Transparency is hard to teach
EARNING TRUST
![Page 26: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/26.jpg)
❖ Protection of data is different than the protection of people/identity❖ Standards like HTTPS or SSL❖ Encryption❖ Anonymity is not a given (TOR Project)❖ The usual fail-points are still threats (weak passwords, compromised servers, careless employees)
PRIVACY
![Page 27: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/27.jpg)
❖ Verify factual occurrences (location, time, date)❖ Verify contributor identity (who?)❖ Verify contributor credentials
VALIDATION
Everything beyond these three points is an educated guess. Anyone looking to game the campaign will only be affective if they are able to compromise the aforementioned.
![Page 28: Scraping the Social Graph with Ushahidi and SwiftRiver](https://reader034.vdocuments.us/reader034/viewer/2022052523/55516350b4c905a8768b524b/html5/thumbnails/28.jpg)
❖ Ease of participation❖ Low risk of failure or shame❖ Social Capital ❖ Repute & Accolade❖ Barter❖ Strategic Spending ($)❖ Data Sharing❖ Altruism & Charity
MOTIVATION