A1 Refereed original research article in a scientific journal

Can visualization alleviate dichotomous thinking Effects of visual representations on the cliff effect




AuthorsHelske Jouni, Helske Satu, Cooper Matthew, Ynnerman Anders, Besançon Lonni

PublisherInstitute of Electrical and Electronics Engineers

Publication year2021

JournalIEEE Transactions on Visualization and Computer Graphics

eISSN1941-0506

DOIhttps://doi.org/10.1109/TVCG.2021.3073466

Web address https://ieeexplore.ieee.org/document/9405484/authors#authors

Self-archived copy’s web addresshttps://research.utu.fi/converis/portal/detail/Publication/56055113


Abstract

Common reporting styles for statistical results in scientific articles,
such as \pvalues\ and confidence intervals (CI), have been reported to
be prone to dichotomous interpretations, especially with respect to the
null hypothesis significance testing framework. For example when the
p-value is small enough or the CIs of the mean effects of a studied drug
and a placebo are not overlapping, scientists tend to claim significant
differences while often disregarding the magnitudes and absolute
differences in the effect sizes. This type of reasoning has been shown
to be potentially harmful to science. Techniques relying on the visual
estimation of the strength of evidence have been recommended to reduce
such dichotomous interpretations but their effectiveness has also been
challenged. We ran two experiments on researchers with expertise in
statistical analysis to compare several alternative representations of
confidence intervals and used Bayesian multilevel models to estimate the
effects of the representation styles on differences in researchers'
subjective confidence in the results. We also asked the respondents'
opinions and preferences in representation styles. Our results suggest
that adding visual information to classic CI representation can decrease
the tendency towards dichotomous interpretations measured as the cliff
effect: the sudden drop in confidence around p-value 0.05 compared with
classic CI visualization and textual representation of the CI with
p-values. All data and analyses are publicly available at
https://github.com/helske/statvis.


Downloadable publication

This is an electronic reprint of the original article.
This reprint may differ from the original in pagination and typographic detail. Please cite the original version.





Last updated on 2024-26-11 at 23:23