Abstract

This paper describes probability forecasting systems that are universal, or universally consistent, in the sense of being consistent under any data-generating distribution, assuming that the observations are produced independently in the IID fashion. The notion of universal consistency is asymptotic and does not imply any small-sample guarantees of validity. On the other hand, the method of conformal prediction has been recently adapted to producing predictive distributions that satisfy a natural property of small-sample validity, namely they are automatically probabilistically calibrated. The main result of the paper is the existence of universal conformal predictive systems, which output predictive distributions that are both probabilistically calibrated and universally consistent.
Original languageEnglish
Article number108536
Number of pages12
JournalPattern Recognition
Volume126
Early online date1 Feb 2022
DOIs
Publication statusPublished - Jun 2022

Keywords

  • conformal prediction
  • predictive distribution
  • probabilistic calibration
  • universal consistency

Cite this