Presentation is loading. Please wait.

Presentation is loading. Please wait.

Alex Quinn Crowdsourcing to Build Lists of Things.

Similar presentations


Presentation on theme: "Alex Quinn Crowdsourcing to Build Lists of Things."— Presentation transcript:

1 Alex Quinn Crowdsourcing to Build Lists of Things

2 make a list of ….

3 Photo: Wikipedia Relay: leader / ramen

4 Mechanical Turk – menu

5 Mechanical Turk – easy site

6 Relay Relay: overview

7 Relay: setup

8 Relay: naïve (Google)

9 Relay: sources

10 Relay: source, Google

11 Relay: source, another site

12 Relay: source, list

13 Relay: report progress

14 Relay: continue

15 Photo: Wikipedia, “Schellack” Results In about 3 hours… 105  69 ramen noodle shops in Kobe 34 culled automatically, 2 culled manually 661  420 articles about food safety 18 culled automatically, 6 culled manually ​155  119 professors studying online education 18 culled automatically, 6 culled manually ​ ​498  ≤258 donut shops in DC issues in specification led to some ambiguity Relay: raw results

16 Ramen noodle shops in Kobe, Japan Relay: results, ramen 1/2

17 Ramen noodle shops in Kobe, Japan Relay: results, ramen 2/2

18 Professors researching online education Relay: results, profs 1/2

19 Professors researching online education Relay: results, profs 2/2

20 Articles about food safety Relay: articles, profs 1/2

21 Articles about food safety Relay: articles, profs 2/2

22 Cost # of itemscost per itemhourly rate Ramen entered 105$0.15 / item $6.95 / hour good 69$0.28 / item Professors entered 155$0.20 / item $6.91 / hour good 131$0.27 / item Articles entered 661$0.20 / item $13.93 / hour good 420$0.32 / item

23 Ideas Design to facilitate clean, unique data items For unbounded tasks, give workers just enough information about what the others are doing Crowdsourcing lets you buy time with money Crowd workers are workers; pay fairly


Download ppt "Alex Quinn Crowdsourcing to Build Lists of Things."

Similar presentations


Ads by Google