Training conformal predictors. / Colombo, Nicolo; Vovk, Vladimir.

Proceedings of Machine Learning Research. ed. / Alex Gammerman; Vladimir Vovk; Zhiyuan Luo; Evgueni Smirnov; Giovanni Cherubin. Vol. 128 2020. p. 55-64.

Research output: Chapter in Book/Report/Conference proceedingChapter

Published

Standard

Training conformal predictors. / Colombo, Nicolo; Vovk, Vladimir.

Proceedings of Machine Learning Research. ed. / Alex Gammerman; Vladimir Vovk; Zhiyuan Luo; Evgueni Smirnov; Giovanni Cherubin. Vol. 128 2020. p. 55-64.

Research output: Chapter in Book/Report/Conference proceedingChapter

Harvard

Colombo, N & Vovk, V 2020, Training conformal predictors. in A Gammerman, V Vovk, Z Luo, E Smirnov & G Cherubin (eds), Proceedings of Machine Learning Research. vol. 128, pp. 55-64.

APA

Colombo, N., & Vovk, V. (2020). Training conformal predictors. In A. Gammerman, V. Vovk, Z. Luo, E. Smirnov, & G. Cherubin (Eds.), Proceedings of Machine Learning Research (Vol. 128, pp. 55-64)

Vancouver

Colombo N, Vovk V. Training conformal predictors. In Gammerman A, Vovk V, Luo Z, Smirnov E, Cherubin G, editors, Proceedings of Machine Learning Research. Vol. 128. 2020. p. 55-64

Author

Colombo, Nicolo ; Vovk, Vladimir. / Training conformal predictors. Proceedings of Machine Learning Research. editor / Alex Gammerman ; Vladimir Vovk ; Zhiyuan Luo ; Evgueni Smirnov ; Giovanni Cherubin. Vol. 128 2020. pp. 55-64

BibTeX

@inbook{2f7be284d0164335a6bd08b9457268c2,
title = "Training conformal predictors",
abstract = "Efficiency criteria for conformal prediction, such as observed fuzziness (i.e., the sum of p-values associated with false labels), are commonly used to evaluate the performance of given conformal predictors. Here, we investigate whether it is possible to exploit efficiency criteria to learn classifiers, both conformal predictors and point classifiers, by using such criteria as training objective functions.The proposed idea is implemented for the problem of binary classification of hand-written digits. By choosing a 1-dimensional model class (with one real-valued free parameter), we can solve the optimization problems through an (approximate) exhaustive search over (a discrete version of) the parameter space. Our empirical results suggest that conformal predictors trained by minimizing their observed fuzziness perform better than conformal predictors trained in the traditional way by minimizing the prediction error of the corresponding point classifier. They also have reasonable performance in terms of their prediction error on the test set.",
keywords = "classification, criterion of efficiency, inductive conformal prediction, observed fuzziness",
author = "Nicolo Colombo and Vladimir Vovk",
year = "2020",
month = sep,
language = "English",
volume = "128",
pages = "55--64",
editor = "Alex Gammerman and Vladimir Vovk and Zhiyuan Luo and Evgueni Smirnov and Giovanni Cherubin",
booktitle = "Proceedings of Machine Learning Research",

}

RIS

TY - CHAP

T1 - Training conformal predictors

AU - Colombo, Nicolo

AU - Vovk, Vladimir

PY - 2020/9

Y1 - 2020/9

N2 - Efficiency criteria for conformal prediction, such as observed fuzziness (i.e., the sum of p-values associated with false labels), are commonly used to evaluate the performance of given conformal predictors. Here, we investigate whether it is possible to exploit efficiency criteria to learn classifiers, both conformal predictors and point classifiers, by using such criteria as training objective functions.The proposed idea is implemented for the problem of binary classification of hand-written digits. By choosing a 1-dimensional model class (with one real-valued free parameter), we can solve the optimization problems through an (approximate) exhaustive search over (a discrete version of) the parameter space. Our empirical results suggest that conformal predictors trained by minimizing their observed fuzziness perform better than conformal predictors trained in the traditional way by minimizing the prediction error of the corresponding point classifier. They also have reasonable performance in terms of their prediction error on the test set.

AB - Efficiency criteria for conformal prediction, such as observed fuzziness (i.e., the sum of p-values associated with false labels), are commonly used to evaluate the performance of given conformal predictors. Here, we investigate whether it is possible to exploit efficiency criteria to learn classifiers, both conformal predictors and point classifiers, by using such criteria as training objective functions.The proposed idea is implemented for the problem of binary classification of hand-written digits. By choosing a 1-dimensional model class (with one real-valued free parameter), we can solve the optimization problems through an (approximate) exhaustive search over (a discrete version of) the parameter space. Our empirical results suggest that conformal predictors trained by minimizing their observed fuzziness perform better than conformal predictors trained in the traditional way by minimizing the prediction error of the corresponding point classifier. They also have reasonable performance in terms of their prediction error on the test set.

KW - classification

KW - criterion of efficiency

KW - inductive conformal prediction

KW - observed fuzziness

M3 - Chapter

VL - 128

SP - 55

EP - 64

BT - Proceedings of Machine Learning Research

A2 - Gammerman, Alex

A2 - Vovk, Vladimir

A2 - Luo, Zhiyuan

A2 - Smirnov, Evgueni

A2 - Cherubin, Giovanni

ER -