Direkt zum Inhalt

Paun, S. ; Carpenter, B. ; Chamberlain, J. D. ; Hovy, D. ; Kruschwitz, Udo ; Poesio, Massimo

Comparing Bayesian Models of Annotation

Article

Paun, S., Carpenter, B., Chamberlain, J. D., Hovy, D., Kruschwitz, Udo and Poesio, Massimo (2018) Comparing Bayesian Models of Annotation. Transactions of the Association for Computational Linguistics 6, pp. 571-585.

DOI to cite this document: 10.5283/epub.40304


Abstract

Crowdsourcing has revolutionised the way tasks can be completed but the process is frequently inefficient, costing practitioners time and money. This research investigates whether crowdsourcing can be optimised with a validation process, as measured by four criteria: quality; cost; noise; and speed. A validation model is described, simulated and tested on real data from an online crowdsourcing ...

Crowdsourcing has revolutionised the way tasks can be completed but the process is frequently inefficient, costing practitioners time and money. This research investigates whether crowdsourcing can be optimised with a validation process, as measured by four criteria: quality; cost; noise; and speed. A validation model is described, simulated and tested on real data from an online crowdsourcing game to collect data about human language. Results show that by adding an agreement validation (or a like/upvote) step fewer annotations are required, noise and collection time are reduced and quality may be improved.



Involved Institutions


Details

Item typeArticle
Journal or Publication TitleTransactions of the Association for Computational Linguistics
PublisherAssociation for Computational Linguistics
Volume6
Page Rangepp. 571-585
Date2018
Date of publication11 Jun 2019 09:37
InstitutionsLanguages and Literatures > Institut für Information und Medien, Sprache und Kultur (I:IMSK) > Lehrstuhl für Informationswissenschaft (Prof. Dr. Udo Kruschwitz)
Informatics and Data Science > Department Human-Centered Computing > Lehrstuhl für Informationswissenschaft (Prof. Dr. Udo Kruschwitz)
Identification Number
ValueType
10.1162/tacl_a_00040DOI
Dewey Decimal Classification000 Computer science, information & general works > 020 Library & information sciences
StatusPublished
RefereedYes, this version has been refereed
Created at the University of RegensburgUnknown
URN of the UB Regensburgurn:nbn:de:bvb:355-epub-403048
Item ID40304

Export bibliographical data

Owner only: item control page

nach oben