A/B testing is a popular tool for guiding mobile game development. The
developer releases different versions of a game to different test
cohorts, and observes which version has the best player retention or
monetization. Correctly determining whether the differences are
statistically significant is however challenging. Typically the analysis
needs to be done on small and heterogeneous player cohorts, with
differing follow-up times and unknown player churn. In this paper, we
show for the first time how these issues can be properly addressed using
the Cox model for recurrent events. The method enables a multivariate
A/B-test, that allows determining which game version has the highest
player retention or purchase rate, with confidence intervals provided.
We demonstrate the benefits of the approach in multiple game development
problems, on real-world free-to-play mobile game data.