A4 Vertaisarvioitu artikkeli konferenssijulkaisussa
Online hate ratings vary by extremes: A statistical analysis
Tekijät: Salminen J., Almerekhi H., Kamel A., Jung S., Jansen B.
Konferenssin vakiintunut nimi: International ACM SIGIR Conference on Research and Development in Information Retrieval
Kustantaja: Association for Computing Machinery, Inc
Julkaisuvuosi: 2019
Journal: International ACM SIGIR Conference on Research and Development in Information Retrieval
Kokoomateoksen nimi: CHIIR 2019 - Proceedings of the 2019 Conference on Human Information Interaction and Retrieval
Tietokannassa oleva lehden nimi: CHIIR 2019 - Proceedings of the 2019 Conference on Human Information Interaction and Retrieval
Aloitussivu: 213
Lopetussivu: 217
Sivujen määrä: 5
ISBN: 978-1-4503-6025-8
DOI: https://doi.org/10.1145/3295750.3298954
Analyzing 5,665 crowd ratings on 1,133 social media comments, we find
that individuals tend to agree on the extremes of a hate rating scale
more than in the middle when evaluating the hatefulness of online
comments. The agreement is higher for less hateful comments and lowest
on moderately hateful comments. The results have implications for
researchers developing machine learning models for online hate
processing, as the extreme classes are likely to require fewer
annotations for reaching statistical stability. Our findings suggest
that the models developed in this domain should consider the
distributions of hate ratings rather than average hate scores.
