Common reporting styles for statistical results in scientific articles,
such as \pvalues\ and confidence intervals (CI), have been reported to
be prone to dichotomous interpretations, especially with respect to the
null hypothesis significance testing framework. For example when the
p-value is small enough or the CIs of the mean effects of a studied drug
and a placebo are not overlapping, scientists tend to claim significant
differences while often disregarding the magnitudes and absolute
differences in the effect sizes. This type of reasoning has been shown
to be potentially harmful to science. Techniques relying on the visual
estimation of the strength of evidence have been recommended to reduce
such dichotomous interpretations but their effectiveness has also been
challenged. We ran two experiments on researchers with expertise in
statistical analysis to compare several alternative representations of
confidence intervals and used Bayesian multilevel models to estimate the
effects of the representation styles on differences in researchers'
subjective confidence in the results. We also asked the respondents'
opinions and preferences in representation styles. Our results suggest
that adding visual information to classic CI representation can decrease
the tendency towards dichotomous interpretations measured as the cliff
effect: the sudden drop in confidence around p-value 0.05 compared with
classic CI visualization and textual representation of the CI with
p-values. All data and analyses are publicly available at
https://github.com/helske/statvis.