Efficient Budget Allocation with Accuracy Guarantees for Crowdsourcing Classification Tasks
OA Location
Author(s)
Tran-Thanh, Long
Venanzi, Matteo
Rogers, Alex
Jennings, Nicholas R
Type
Conference Paper
Abstract
In this paper we address the problem of budget allocation for redundantly crowdsourcing a set of classification tasks where a key challenge is to find a trade?off between the total cost and the accuracy of estimation. We propose CrowdBudget, an agent?based budget allocation algorithm, that efficiently divides a given budget among different tasks in order to achieve low estimation error. In particular, we prove that CrowdBudget can achieve at most max0, K/2 ? O (?B) estimation error with high probability, where K is the num- ber of tasks and B is the budget size. This result significantly outperforms the current best theoretical guarantee from Karger et al. In addition, we demonstrate that our algorithm outperforms existing methods by up to 40% in experiments based on real?world data from a prominent database of crowdsourced classification responses.
Date Issued
2013-05
Citation
2013, pp.901-908
Publisher
International Foundation for Autonomous Agents and Multiagent Systems
Start Page
901
End Page
908
Identifier
http://eprints.soton.ac.uk/346675/
Source
AAMAS ’13 Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Publication Status
Unpublished