Estimation of predictive performance in high-dimensional data settings using learning curves

Research output: Contribution to journalArticleAcademicpeer-review


In high-dimensional prediction settings, it remains challenging to reliably estimate the test performance. To address this challenge, a novel performance estimation framework is presented. This framework, called Learn2Evaluate, is based on learning curves by fitting a smooth monotone curve depicting test performance as a function of the sample size. Learn2Evaluate has several advantages compared to commonly applied performance estimation methodologies. Firstly, a learning curve offers a graphical overview of a learner. This overview assists in assessing the potential benefit of adding training samples and it provides a more complete comparison between learners than performance estimates at a fixed subsample size. Secondly, a learning curve facilitates in estimating the performance at the total sample size rather than a subsample size. Thirdly, Learn2Evaluate allows the computation of a theoretically justified and useful lower confidence bound. Furthermore, this bound may be tightened by performing a bias correction. The benefits of Learn2Evaluate are illustrated by a simulation study and applications to omics data.

Original languageEnglish
Article number107622
JournalComputational Statistics and Data Analysis
Early online date2022
Publication statusE-pub ahead of print - 2022


  • Area under the receiver operating curve
  • Bootstrap
  • Cross-validation
  • High-dimensional data
  • Omics
  • Predictive performance

Cite this