Large-scale probabilistic predictors with and without guarantees of validity

This paper studies theoretically and empirically a method of turning machine learning algorithms into probabilistic predictors that automatically enjoys a property of validity (perfect calibration), is computationally efficient, and preserves predictive efficiency. The price to pay for perfect calibration is that these probabilistic predictors produce imprecise (in practice, almost precise for large data sets) probabilities. When these imprecise probabilities are merged into precise probabilities, the resulting predictors, while losing the theoretical property of perfect calibration, consistently outperform the existing methods in empirical studies.