| Published Version Download ( PDF | 751kB) | License: Creative Commons Attribution 4.0 |
Comparing Bayesian Models of Annotation
Article
Paun, S., Carpenter, B., Chamberlain, J. D., Hovy, D., Kruschwitz, Udo
and Poesio, Massimo
(2018)
Comparing Bayesian Models of Annotation.
Transactions of the Association for Computational Linguistics 6, pp. 571-585.
DOI to cite this document: 10.5283/epub.40304
Abstract
Crowdsourcing has revolutionised the way tasks can be completed but the process is frequently inefficient, costing practitioners time and money. This research investigates whether crowdsourcing can be optimised with a validation process, as measured by four criteria: quality; cost; noise; and speed. A validation model is described, simulated and tested on real data from an online crowdsourcing ...
Crowdsourcing has revolutionised the way tasks can be completed but the process is frequently inefficient, costing practitioners time and money. This research investigates whether crowdsourcing can be optimised with a validation process, as measured by four criteria: quality; cost; noise; and speed. A validation model is described, simulated and tested on real data from an online crowdsourcing game to collect data about human language. Results show that by adding an agreement validation (or a like/upvote) step fewer annotations are required, noise and collection time are reduced and quality may be improved.
Alternative links to fulltext
Involved Institutions
Details
| Item type | Article | ||||
| Journal or Publication Title | Transactions of the Association for Computational Linguistics | ||||
| Publisher | Association for Computational Linguistics | ||||
| Volume | 6 | ||||
| Page Range | pp. 571-585 | ||||
| Date | 2018 | ||||
| Date of publication | 11 Jun 2019 09:37 | ||||
| Institutions | Languages and Literatures > Institut für Information und Medien, Sprache und Kultur (I:IMSK) > Lehrstuhl für Informationswissenschaft (Prof. Dr. Udo Kruschwitz) Informatics and Data Science > Department Human-Centered Computing > Lehrstuhl für Informationswissenschaft (Prof. Dr. Udo Kruschwitz) | ||||
| Identification Number |
| ||||
| Dewey Decimal Classification | 000 Computer science, information & general works > 020 Library & information sciences | ||||
| Status | Published | ||||
| Refereed | Yes, this version has been refereed | ||||
| Created at the University of Regensburg | Unknown | ||||
| URN of the UB Regensburg | urn:nbn:de:bvb:355-epub-403048 | ||||
| Item ID | 40304 |
Download Statistics
Download Statistics