Retrain or not retrain: Conformal test martingales for change-point detection

Vladimir Vovk, Ivan Petej, Ilia Nouretdinov, Ernst Ahlberg, Lars Carlsson, Alex Gammerman

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

We argue for supplementing the process of training a prediction algorithm by setting up a scheme for detecting the moment when the distribution of the data changes and the algorithm needs to be retrained. Our proposed schemes are based on exchangeability martingales, i.e., processes that are martingales under any exchangeable distribution for the data. Our method, based on conformal prediction, is general and can be applied on top of any modern prediction algorithm. Its validity is guaranteed, and in this paper we make first steps in exploring its efficiency.
Original languageEnglish
Title of host publicationProceedings of Machine Learning Research
EditorsLars Carlsson, Zhiyuan Luo, Giovanni Cherubin, Khuong Nguyen
Pages191-210
Number of pages20
Volume152
Publication statusPublished - 2021

Keywords

  • conformal test martingales
  • dataset shift
  • IID assumption
  • retraining

Cite this