The purpose of this research is to test the validity of commonly used measures of exposure to and
production of online extremism. Specifically, we investigate if a definition of hate influences survey
responses about the production of and exposure to online hate. To explore the effects of a
definition, we used a split experimental design on a sample of 18 to 25-year-old Americans where
half of the respondents were exposed to the European Union’s definition of hate speech and the
other half were not. Then, all respondents completed a survey with commonly used items
measuring exposure to and perpetration of online hate. The results reveal that providing a
definition affects self-reported levels of exposure and perpetration, but the effects are dependent
on race. The findings provide evidence that survey responses about online hate may be conditioned
by social desirability and framing biases. The findings that group differences exist in how
questions about hate are interpreted when definitions of it are not provided mean we must be
careful when using measures that try to capture exposure to and the production of hate. While
more research is needed, we recommend providing a clear, unambiguous definition when using
surveys to measure online hate.