BudgetFix: budget limited crowdsourcing for interdependent task allocation with quality guarantees
OA Location
Author(s)
Tran-Thanh, Long
Huynh, Trung Dong
Rosenfeld, A
Ramchurn, Sarvapali
Jennings, Nicholas R
Type
Conference Paper
Abstract
Copyright © 2014, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.Crowdsourcing is a multi-agent task allocation paradigm that involves up to millions of workers, of varying reliability and availability, performing large numbers of micro-tasks. A key challenge is to crowdsource, at minimal cost and with predictable accuracy, complex tasks that involve different types of interdependent micro-tasks structured into complex workflows. In this paper, we propose the first crowdsourcing algorithm that solves this problem. Our algorithm, called BudgetFix, determines the number of interdependent micro-tasks and the price to pay for each task given budget constraints. Moreover, BudgetFix provides quality guarantees on the accuracy of the output of each phase of a given workflow. BudgetFix is empirically evaluated on a well-known crowdsourcing-based text correction workflow using Amazon Mechanical Turk, and is shown that BudgetFix can provide similar accuracy, compared to the state-of-the-art algorithm for this workflow, but is on average 32% cheaper.
Date Issued
2014
Citation
2014, pp.477-484
Publisher
International Foundation for Autonomous Agents and Multiagent Systems
Start Page
477
End Page
484
Identifier
http://eprints.soton.ac.uk/362321/
Source
13th International Conference on Autonomous Agents and Multi-Agent Systems
Publication Status
Unpublished