A4 Article in conference proceedings
A/B-Test of Retention and Monetization Using the Cox Model

List of Authors: Markus Viljanen, Antti Airola, Jukka Heikkonen, Tapio Pahikkala
Place: Palo Alto, California
Publication year: 2017
Book title *: Proceedings, The Thirteenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE-17)
Number of pages: 7
ISBN: 978-1-57735-791-9
ISSN: 2326-909X


A/B testing is a popular tool for guiding mobile game development. The
developer releases different versions of a game to different test
cohorts, and observes which version has the best player retention or
monetization. Correctly determining whether the differences are
statistically significant is however challenging. Typically the analysis
needs to be done on small and heterogeneous player cohorts, with
differing follow-up times and unknown player churn. In this paper, we
show for the first time how these issues can be properly addressed using
the Cox model for recurrent events. The method enables a multivariate
A/B-test, that allows determining which game version has the highest
player retention or purchase rate, with confidence intervals provided.
We demonstrate the benefits of the approach in multiple game development
problems, on real-world free-to-play mobile game data.

Last updated on 2019-29-01 at 23:28