![Page 1: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/1.jpg)
Aakar Gupta*, William Thies#,
Edward Cutrell#, Ravin Balakrishnan*
*University of Toronto #Microsoft Research India
mClerk: Enabling Mobile Crowdsourcing in Developing Regions
![Page 2: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/2.jpg)
Prior efforts either have middle-class workers..
Paid Crowdsourcing has yet to Deliver
on its Potential in Developing Regions
2
..or see barriers to scale in low-income contexts
![Page 3: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/3.jpg)
mClerk
3
![Page 4: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/4.jpg)
Lack of access to computers & Internet
Limited skills for tasks
Lack of payment mechanisms
Low awareness
Overcoming the Barriers
to Scalable Crowdsourcing in Dev. Regions
4
![Page 5: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/5.jpg)
Lack of access to computers & Internet
1. Send visual tasks via SMS
Overcoming the Barriers
to Scalable Crowdsourcing in Dev. Regions
5
![Page 6: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/6.jpg)
1. Send Visual Tasks via SMS
Nokia’s Smart Messaging
• Binary Images
• 74x28 pixels
• Same cost as 3 SMSs!
6
![Page 7: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/7.jpg)
Lack of access to computers & Internet
Limited skills for tasks
Lack of payment mechanisms
Low awareness
1. Send visual tasks via SMS
2. Local-language digitization
Overcoming the Barriers
to Scalable Crowdsourcing in Dev. Regions
7
![Page 8: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/8.jpg)
(i) Document Segmentation
(ii) Crowdsourced Digitization
(iii) Response Verification
2. Local-Language Digitization
Goal: Digitize paper documents in local language
8
![Page 9: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/9.jpg)
(i) Document Segmentation
Clean, Segment, Binarize, Resize
9
![Page 10: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/10.jpg)
(ii) Crowdsourced Digitization
• But local language fonts difficult or unsupported!
• Solution: back and forth transliteration
ಮೂರ್ತಿ murthy
User sends English transliteration
Server transliterates back
1. Send worker a word image via picture SMS
2. Worker replies with text SMS
10
![Page 11: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/11.jpg)
(iii) Response Verification
murthy
≡
moorthi
Verify agreement of transliterated text
ಮೂರ್ತಿ ಮೂರ್ತಿ Accepted
11
![Page 12: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/12.jpg)
Lack of access to computers & Internet
Limited skills for tasks
Lack of payment mechanisms
Low awareness
1. Send visual tasks via SMS
2. Local-language digitization
3. Pay with mobile airtime
Overcoming the Barriers
to Scalable Crowdsourcing in Dev. Regions
12
![Page 13: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/13.jpg)
3. Pay With Mobile Airtime
• Manual recharge via mobile shop
13
![Page 14: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/14.jpg)
Lack of access to computers & Internet
Limited skills for tasks
Lack of payment mechanisms
Low awareness
1. Send visual tasks via SMS
2. Local-language digitization
3. Pay with mobile airtime
4. Incentivize viral spread
Overcoming the Barriers
to Scalable Crowdsourcing in Dev. Regions
14
![Page 15: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/15.jpg)
4. Incentivize Viral Spread
(i) Referral system
Each worker earns 10% of their referrals’ earnings
(ii) Leaderboard messages
(iii) Feedback & motivational messages
15
![Page 16: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/16.jpg)
Field Study
Will users adopt the system and use it willingly in a real-world setting?
16
![Page 17: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/17.jpg)
Field Study
• 5 Week study, divided into 2 phases
• Phase 1 (3 Weeks): Paid INR 0.5 / task (~1 ¢)
• Phase 2 (2 Weeks): Paid INR 0.2 / task
• Semi-urban location, 4 hours from Bangalore
• Language: Kannada
17
![Page 18: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/18.jpg)
Results: Diffusion Network
239 Users, 64000 Responses, 25000 Digitized Words
We contacted only 10 users. Five weeks later:
18
![Page 19: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/19.jpg)
Results: Group Effects
Social interactions drive usage
19
![Page 20: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/20.jpg)
Qualitative Themes
Time Pass
“I have to wait 20mins for bus. I stand and do at bus-stop. I have stopped going to the recharge shop, I get enough.”
Skepticism
“It is like some code sending. What do you do using this?”
“Is it legal? What’s your profit? I don’t want any trouble.”
Flip side
“We sit at back in class and message during lecture.”
“Earlier we [friends] used to message poetry, jokes etc. Now no one does that. Everyone is busy with this.”
20
![Page 21: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/21.jpg)
Performance: Accuracy
• Fraction of words digitized correctly: 90.1%
• Improvement of Accuracy
65
70
75
80
85
90
95
2 3 4 5 6
Accuracy %
No. of Responses per word (to get an agreement)
21
![Page 22: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/22.jpg)
Performance: Agreements
2
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
Avg. no. of responses per word
Word Length (px) before scaling the length
Screen Width(74px)
22
![Page 23: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/23.jpg)
• With some optimization, could be market viable
• Partner with telcos to decrease payment overhead
• Identify more accurate workers to improve accuracy
Business Sustainability
mClerk Market
Costs Phase 1: 2.4 ¢/ word Phase 2: 1 ¢/ word
2 ¢/ word
Accuracy 90% 97%
23
![Page 24: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/24.jpg)
Conclusions
• mClerk enables scalable crowdsourcing in developing regions by:
• Future opportunities in
- Optimizing accuracy and costs
- Finding more tasks amenable to picture-SMS
1. Sending visual tasks via SMS
2. Leveraging local language skills
3. Paying with mobile airtime
4. Incentivizing viral spread
24
![Page 25: mClerk: Enabling Mobile Crowdsourcing in Developing Regions · 2018-01-29 · tasks via SMS 2. Local-language digitization 3. Pay with mobile airtime 4. Incentivize viral spread Overcoming](https://reader033.vdocuments.us/reader033/viewer/2022050513/5f9d4824d6f593623055c21b/html5/thumbnails/25.jpg)
Thank You
Contact: [email protected]
To Nithya Sambasivan, Richard T. Guy, James Davis, Indrani Medhi To all of our users, as well as their..
Competitors - “All my friends have become leaders [at least once]. Now I sleep at 12, so that I can do fast messages at night.”
Family members - “I gave my phone to my wife. She is free at home. She can do more SMS. I take it in evening when I get free with friends”
Collaborators - “Coming back from college in the bus, all of us do messaging and ask each other meanings of the words for fun. One time no one knew so we thought we’ll ask the Kannada lecturer in college and if he does not know that will be fun.. but he knew.”
25