Refereed journal article or data article (A1)

The spatial leave-pair-out cross-validation method for reliable AUC estimation of spatial classifiers

List of Authors: Antti Airola, Jonne Pohjankukka, Johanna Torppa, Maarit Middleton, Vesa Nykänen, Jukka Heikkonen, Tapio Pahikkala

Publisher: Springer New York LLC

Publication year: 2019

Journal: Data Mining and Knowledge Discovery

Journal name in source: Data Mining and Knowledge Discovery

Volume number: 33

Issue number: 3

Number of pages: 18

ISSN: 1384-5810

eISSN: 1573-756X



Self-archived copy’s web address:


Machine learning based classification methods are widely used in geoscience applications, including mineral prospectivity mapping. Typical characteristics of the data, such as small number of positive instances, imbalanced class distributions and lack of verified negative instances make ROC analysis and cross-validation natural choices for classifier evaluation. However, recent literature has identified two sources of bias, that can affect reliability of area under ROC curve estimation via cross-validation on spatial data. The pooling procedure performed by methods such as leave-one-out can introduce a substantial negative bias to results. At the same time, spatial dependencies leading to spatial autocorrelation can result in overoptimistic results, if not corrected for. In this work, we introduce the spatial leave-pair-out cross-validation method, that corrects for both of these biases simultaneously. The methodology is used to benchmark a number of classification methods on mineral prospectivity mapping data from the Central Lapland greenstone belt. The evaluation highlights the dangers of obtaining misleading results on spatial data and demonstrates how these problems can be avoided. Further, the results show the advantages of simple linear models for this classification task.

Downloadable publication

This is an electronic reprint of the original article.
This reprint may differ from the original in pagination and typographic detail. Please cite the original version.

Last updated on 2022-07-04 at 17:10