Reliable Confidence Measures and Well-Calibrated Probabilistic Outputs in Classification Algorithms. / Lamprou, Antonis.

2016. 169 p.

Research output: ThesisDoctoral Thesis

Unpublished

Standard

Harvard

APA

Vancouver

Author

BibTeX

@phdthesis{a626a2ae087a4fb7bdbd18c9373e108f,
title = "Reliable Confidence Measures and Well-Calibrated Probabilistic Outputs in Classification Algorithms",
abstract = "The Machine Learning research area is widely used in several predictive systems,where observations from the past can be used to create predictions aboutfuture events. Machine Learning can be applied to any area where classificationor regression is used. Nonetheless, most Machine Learning algorithmsdo not provide any measures of valid confidence. Conformal Prediction (CP)is a framework which uses underlying Machine Learning algorithms, and canprovide valid measures of confidence for predictions. Additionally, the VennPrediction (VP) framework, which is an extension to the CP framework, provideswell-calibrated probabilistic outputs. This thesis explores and providesnew methods for valid measures of confidence and probabilistic outputs, basedon the Conformal and Venn Prediction frameworks. We introduce a new ConformalPredictor based on Genetic Algorithms and compare our approach withother methods. Additionally, the CP framework is extended for multi-label applicationswhere predictions can contain more than one possible classifications.Furthermore, a new Venn Predictor based on Inductive CP is introduced, whichgreatly improves the computational eciency of VP. We conduct experimentson our methods and examine their performance and validity. Finally, we examinethe applications of osteoporosis risk assessment, the diagnosis of childhoodabdominal pain, and the evaluation of the risk of stroke based on ultrasoundimages of atherosclerotic carotid plaques. Our experimental results on all ourmethods demonstrate the reliability and usefulness of our conndence andprobabilistic outputs.",
keywords = "confidence measures, conformal prediction, venn machines, machine learning",
author = "Antonis Lamprou",
year = "2016",
language = "English",
school = "Royal Holloway, University of London",

}

RIS

TY - THES

T1 - Reliable Confidence Measures and Well-Calibrated Probabilistic Outputs in Classification Algorithms

AU - Lamprou, Antonis

PY - 2016

Y1 - 2016

N2 - The Machine Learning research area is widely used in several predictive systems,where observations from the past can be used to create predictions aboutfuture events. Machine Learning can be applied to any area where classificationor regression is used. Nonetheless, most Machine Learning algorithmsdo not provide any measures of valid confidence. Conformal Prediction (CP)is a framework which uses underlying Machine Learning algorithms, and canprovide valid measures of confidence for predictions. Additionally, the VennPrediction (VP) framework, which is an extension to the CP framework, provideswell-calibrated probabilistic outputs. This thesis explores and providesnew methods for valid measures of confidence and probabilistic outputs, basedon the Conformal and Venn Prediction frameworks. We introduce a new ConformalPredictor based on Genetic Algorithms and compare our approach withother methods. Additionally, the CP framework is extended for multi-label applicationswhere predictions can contain more than one possible classifications.Furthermore, a new Venn Predictor based on Inductive CP is introduced, whichgreatly improves the computational eciency of VP. We conduct experimentson our methods and examine their performance and validity. Finally, we examinethe applications of osteoporosis risk assessment, the diagnosis of childhoodabdominal pain, and the evaluation of the risk of stroke based on ultrasoundimages of atherosclerotic carotid plaques. Our experimental results on all ourmethods demonstrate the reliability and usefulness of our conndence andprobabilistic outputs.

AB - The Machine Learning research area is widely used in several predictive systems,where observations from the past can be used to create predictions aboutfuture events. Machine Learning can be applied to any area where classificationor regression is used. Nonetheless, most Machine Learning algorithmsdo not provide any measures of valid confidence. Conformal Prediction (CP)is a framework which uses underlying Machine Learning algorithms, and canprovide valid measures of confidence for predictions. Additionally, the VennPrediction (VP) framework, which is an extension to the CP framework, provideswell-calibrated probabilistic outputs. This thesis explores and providesnew methods for valid measures of confidence and probabilistic outputs, basedon the Conformal and Venn Prediction frameworks. We introduce a new ConformalPredictor based on Genetic Algorithms and compare our approach withother methods. Additionally, the CP framework is extended for multi-label applicationswhere predictions can contain more than one possible classifications.Furthermore, a new Venn Predictor based on Inductive CP is introduced, whichgreatly improves the computational eciency of VP. We conduct experimentson our methods and examine their performance and validity. Finally, we examinethe applications of osteoporosis risk assessment, the diagnosis of childhoodabdominal pain, and the evaluation of the risk of stroke based on ultrasoundimages of atherosclerotic carotid plaques. Our experimental results on all ourmethods demonstrate the reliability and usefulness of our conndence andprobabilistic outputs.

KW - confidence measures

KW - conformal prediction

KW - venn machines

KW - machine learning

M3 - Doctoral Thesis

ER -