The abelian critical exponent of an infinite word $w$ is defined as the maximum ratio between the exponent and the period of an abelian power occurring in $w$. It was shown by Fici et al. that the set of finite abelian critical exponents of Sturmian words coincides with the Lagrange spectrum. This spectrum contains every large enough positive real number. We construct words whose abelian critical exponents fill the remaining gaps, that is, we prove that for each nonnegative real number $\theta$ there exists an infinite word having abelian critical exponent $\theta$. We also extend this result to the $k$-abelian setting.