Transformer-based conformal predictors for paraphrase detection

Research output: Contribution to conferencePaperpeer-review

Abstract

Transformer architectures have established themselves as the state-of-the-art in many areas of natural language processing (NLP), including paraphrase detection (PD). However, they do not include a confidence estimation for each prediction and, in many cases, the applied models are poorly calibrated. These features are essential for numerous real-world applications. For example, in those cases when PD is used for sensitive tasks, like plagiarism detection, hate speech recognition or in medical NLP, mistakes might be very costly. In this work we build several variants of transformer- based conformal predictors and study their behaviour on a standard PD dataset. We show that our models are able to produce \emph{valid} predictions while retaining the accuracy of the original transformer-based models. The proposed technique can be extended to many more NLP problems that are currently being investigated.
Original languageEnglish
Pages243-265
Number of pages23
Publication statusPublished - 2021
Event10th Symposium on Conformal and Probabilistic Prediction with Applications, COPA 2021 - Online
Duration: 8 Sept 202110 Sept 2021
Conference number: 10th

Conference

Conference10th Symposium on Conformal and Probabilistic Prediction with Applications, COPA 2021
Abbreviated titleCOPA 2021
Period8/09/2110/09/21

Cite this