Enabling Fair ML Evaluations for Security. / Pendlebury, Feargus; Pierazzi, Fabio; Jordaney, Roberto; Kinder, Johannes; Cavallaro, Lorenzo.

2018. 2264-2266 Poster session presented at ACM Conference on Computer and Communications Security, Toronto, Canada.

Research output: Contribution to conferencePoster

Published

Documents

Abstract

Machine learning is widely used in security research to classify malicious activity, ranging from malware to malicious URLs and network traffic. However, published performance numbers often seem to leave little room for improvement and, due to a wide range of datasets and configurations, cannot be used to directly compare alternative approaches; moreover, most evaluations have been found to suffer from experimental bias which positively inflates results. In this manuscript we discuss the implementation of Tesseract, an open-source tool to evaluate the performance of machine learning classifiers in a security setting mimicking a deployment with typical data feeds over an extended period of time. In particular, Tesseract allows for a fair comparison of different classifiers in a realistic scenario, without disadvantaging any given classifier. Tesseract is available as open-source to provide the academic community with a way to report sound and comparable performance results, but also to help practitioners decide which system to deploy under specific budget constraints.
Original languageEnglish
Pages2264-2266
Number of pages3
DOIs
Publication statusPublished - 8 Oct 2018
EventACM Conference on Computer and Communications Security - Beanfield Centre, Toronto, Canada
Duration: 15 Oct 201819 Oct 2018
https://www.sigsac.org/ccs/CCS2018/

Conference

ConferenceACM Conference on Computer and Communications Security
Abbreviated titleCCS '18
CountryCanada
CityToronto
Period15/10/1819/10/18
Internet address
This open access research output is licenced under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

ID: 32698559