Inter-Rater Agreement and Usability: A Comparative Evaluation of Annotation Tools for Sentiment Annotation

URN to cite this document:: urn:nbn:de:bvb:355-epub-435757

Schmidt, Thomas ; Winterl, Brigitte ; Maul, Milena ; Schark, Alina ; Vlad, Andrea ; Wolff, Christian

Preview

License: Creative Commons: Attribution 3.0
PDF - Published Version
(233kB)

Date of publication of this fulltext: 07 Aug 2020 05:25

Alternative links to fulltext:DOI Publisher

Details

Preview

Export bibliographical data

Abstract

We present the results of a comparative evaluation study of five annotation tools with 50 participants in the context of sentiment and emotion annotation of literary texts. Ten participants per tool annotated 50 speeches of the play Emilia Galotti by G. E. Lessing. We evaluate the tools via standard usability and user experience questionnaires, by measuring the time needed for the annotation, and ...