A1 Vertaisarvioitu alkuperäisartikkeli tieteellisessä lehdessä
How to Make Sense of Reliability? Common Language Interpretation of Reliability and the Relation of Reliability to Effect Size
Tekijät: Metsämuuronen, Jari; Niemensivu, Timi
Kustantaja: SAGE Publications
Kustannuspaikka: THOUSAND OAKS
Julkaisuvuosi: 2025
Journal: Applied Psychological Measurement
Tietokannassa oleva lehden nimi: Applied Psychological Measurement
Lehden akronyymi: APPL PSYCH MEAS
Sivujen määrä: 21
ISSN: 0146-6216
eISSN: 1552-3497
DOI: https://doi.org/10.1177/01466216251350159
Verkko-osoite: https://doi.org/10.1177/01466216251350159
Rinnakkaistallenteen osoite: https://research.utu.fi/converis/portal/detail/Publication/498940181
Communicating the factual meaning of a particular reliability estimate is sometimes difficult. What does a specific reliability estimate of 0.80 or 0.95 mean in common language? Deflation-corrected estimates of reliability (DCER) using Somers' D or Goodman-Kruskal G as the item-score correlations are transformed into forms where specific estimates from the family of common language effect sizes are visible. This makes it possible to communicate reliability estimates using a common language and to evaluate the magnitude of a particular reliability estimate in the same way and with the same metric as we do with effect size estimates. Using a DCER, we can say that with k = 40 items, if the reliability is 0.95, in 80 out of 100 random pairs of test takers from different subpopulations on all items combined, those with a higher item response will also score higher on the test. In this case, using the thresholds familiar from effect sizes, we can say that the reliability is "very high." The transformation of the reliability estimate into a common language effect size depends on the size of the item-score association estimates and the number of items, so no closed-form equations for the transformations are given. However, relevant thresholds are provided for practical use.
Ladattava julkaisu This is an electronic reprint of the original article. |  |
Julkaisussa olevat rahoitustiedot:
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study has been completed as part of the EDUCA Flagship project funded by the Research Council of Finland (#358924, #358947).
