A4 Refereed article in a conference publication
A/B-Test of Retention and Monetization Using the Cox Model
Authors: Markus Viljanen, Antti Airola, Jukka Heikkonen, Tapio Pahikkala
Editors: Brian Magerko, Jonathan Rowe
Conference name: Artificial Intelligence and Interactive Digital Entertainment
Publishing place: Palo Alto, California
Publication year: 2017
Book title : Proceedings, The Thirteenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE-17)
Series title: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
Volume: 13
Issue: 1
First page : 248
Last page: 254
Number of pages: 7
ISBN: 978-1-57735-791-9
ISSN: 2326-909X
DOI: https://doi.org/10.1609/aiide.v13i1.12941
Web address : https://doi.org/10.1609/aiide.v13i1.12941
A/B testing is a popular tool for guiding mobile game development. The
developer releases different versions of a game to different test
cohorts, and observes which version has the best player retention or
monetization. Correctly determining whether the differences are
statistically significant is however challenging. Typically the analysis
needs to be done on small and heterogeneous player cohorts, with
differing follow-up times and unknown player churn. In this paper, we
show for the first time how these issues can be properly addressed using
the Cox model for recurrent events. The method enables a multivariate
A/B-test, that allows determining which game version has the highest
player retention or purchase rate, with confidence intervals provided.
We demonstrate the benefits of the approach in multiple game development
problems, on real-world free-to-play mobile game data.
