Advancing Equitable Research Evaluation: Introducing Keyword Impact Assessment (KIA) and Case Study Insights through Open-Access Software




Maleki, Ashraf; Holmberg, Kim

Nordic Workshop on Bibliometrics & Research Policy 2024

2024

Book of Oral Abstracts 29th Nordic Workshop on Bibliometrics & Research Policy

13

13



As scientific domains continue to evolve, traditional citation metrics alone are increasingly inadequate for timely capturing the full scope of research impact, particularly across diverse topics and communities. This presentation introduces a new methodology that integrates altmetrics with traditional bibliometrics, promoting more equitable and inclusive approaches to research evaluation. Central to this work is the introduction of the newly developed Keyword Impact Assessment (KIA) method, supported by open-access software, which quantifies altmetrics at the topic level by analyzing citation and altmetric indicators at the keyword level. This method provides a more nuanced understanding of research impact, extending beyond conventional metrics. The effectiveness of KIA is demonstrated through a case study in the Information and Computer Science domain, examining thematic shifts in research focus across 177,225 publications from 1980 to 2023. It reveals how research themes evolve and how different topics gain or lose attention over time, highlighting the importance of diverse metric counts in research assessment. By integrating VOSviewer clustering with KIA, overlays of citation and altmetric-derived variables are created on a fixed layout of keyword co-occurrence maps, revealing thematic shifts and trends. Further, we showcase the use of Multivariate Scientometric Mapping (MSM) techniques, which facilitate the comparison of citations and altmetrics across multiple variables. A central focus of this presentation is on how this method can promote inclusiveness in research evaluation and how altmetrics can serve as complementary proxies for impact assessment. This presentation offers practical insights and tools to implement advanced bibliometric and altmetric methods in research evaluation.



Last updated on 2025-27-01 at 18:56