A4 Article in conference proceedings
Online hate ratings vary by extremes: A statistical analysis




List of Authors: Salminen J., Almerekhi H., Kamel A., Jung S., Jansen B.
Publisher: Association for Computing Machinery, Inc
Publication year: 2019
Book title *: CHIIR 2019 - Proceedings of the 2019 Conference on Human Information Interaction and Retrieval
Journal name in source: CHIIR 2019 - Proceedings of the 2019 Conference on Human Information Interaction and Retrieval
ISBN: 978-145036025-8

Abstract

Analyzing 5,665 crowd ratings on 1,133 social media comments, we find
that individuals tend to agree on the extremes of a hate rating scale
more than in the middle when evaluating the hatefulness of online
comments. The agreement is higher for less hateful comments and lowest
on moderately hateful comments. The results have implications for
researchers developing machine learning models for online hate
processing, as the extreme classes are likely to require fewer
annotations for reaching statistical stability. Our findings suggest
that the models developed in this domain should consider the
distributions of hate ratings rather than average hate scores.



Internal Authors/Editors

Last updated on 2019-19-07 at 19:35