Dos and Don'ts of Machine Learning in Computer Security. / Arp, Daniel; Quiring, Erwin; Pendlebury, Feargus; Warnecke, Alexander; Pierazzi, Fabio; Wressnegger, Christian; Cavallaro, Lorenzo; Rieck, Konrad.

31st USENIX Security Symposium (USENIX Sec). 2021.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Forthcoming

Standard

Dos and Don'ts of Machine Learning in Computer Security. / Arp, Daniel; Quiring, Erwin; Pendlebury, Feargus; Warnecke, Alexander; Pierazzi, Fabio; Wressnegger, Christian; Cavallaro, Lorenzo; Rieck, Konrad.

31st USENIX Security Symposium (USENIX Sec). 2021.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Harvard

Arp, D, Quiring, E, Pendlebury, F, Warnecke, A, Pierazzi, F, Wressnegger, C, Cavallaro, L & Rieck, K 2021, Dos and Don'ts of Machine Learning in Computer Security. in 31st USENIX Security Symposium (USENIX Sec). <https://arxiv.org/pdf/2010.09470.pdf>

APA

Arp, D., Quiring, E., Pendlebury, F., Warnecke, A., Pierazzi, F., Wressnegger, C., Cavallaro, L., & Rieck, K. (Accepted/In press). Dos and Don'ts of Machine Learning in Computer Security. In 31st USENIX Security Symposium (USENIX Sec) https://arxiv.org/pdf/2010.09470.pdf

Vancouver

Arp D, Quiring E, Pendlebury F, Warnecke A, Pierazzi F, Wressnegger C et al. Dos and Don'ts of Machine Learning in Computer Security. In 31st USENIX Security Symposium (USENIX Sec). 2021

Author

Arp, Daniel ; Quiring, Erwin ; Pendlebury, Feargus ; Warnecke, Alexander ; Pierazzi, Fabio ; Wressnegger, Christian ; Cavallaro, Lorenzo ; Rieck, Konrad. / Dos and Don'ts of Machine Learning in Computer Security. 31st USENIX Security Symposium (USENIX Sec). 2021.

BibTeX

@inproceedings{bc06c45a87f94217893a0f01748eb5d2,
title = "Dos and Don'ts of Machine Learning in Computer Security",
abstract = "With the growing processing power of computing systems and the increasing availability of massive datasets, machine learning algorithms have led to major breakthroughs in many different areas. This development has influenced computer security, spawning a series of work on learning-based security systems, such as for malware detection, vulnerability discovery, and binary code analysis. Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance and render learning-based systems potentially unsuitable for security tasks and practical deployment.In this paper, we look at this problem with critical eyes. First, we identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We conduct a study of 30 papers from top-tier security conferences within the past 10 years, confirming that these pitfalls are widespread in the current security literature. In an empirical analysis, we further demonstrate how individual pitfalls can lead to unrealistic performance and interpretations, obstructing the understanding of the security problem at hand. As a remedy, we propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible. Furthermore, we identify open problems when applying machine learning in security and provide directions for further research.",
author = "Daniel Arp and Erwin Quiring and Feargus Pendlebury and Alexander Warnecke and Fabio Pierazzi and Christian Wressnegger and Lorenzo Cavallaro and Konrad Rieck",
year = "2021",
month = jul,
day = "16",
language = "English",
booktitle = "31st USENIX Security Symposium (USENIX Sec)",

}

RIS

TY - GEN

T1 - Dos and Don'ts of Machine Learning in Computer Security

AU - Arp, Daniel

AU - Quiring, Erwin

AU - Pendlebury, Feargus

AU - Warnecke, Alexander

AU - Pierazzi, Fabio

AU - Wressnegger, Christian

AU - Cavallaro, Lorenzo

AU - Rieck, Konrad

PY - 2021/7/16

Y1 - 2021/7/16

N2 - With the growing processing power of computing systems and the increasing availability of massive datasets, machine learning algorithms have led to major breakthroughs in many different areas. This development has influenced computer security, spawning a series of work on learning-based security systems, such as for malware detection, vulnerability discovery, and binary code analysis. Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance and render learning-based systems potentially unsuitable for security tasks and practical deployment.In this paper, we look at this problem with critical eyes. First, we identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We conduct a study of 30 papers from top-tier security conferences within the past 10 years, confirming that these pitfalls are widespread in the current security literature. In an empirical analysis, we further demonstrate how individual pitfalls can lead to unrealistic performance and interpretations, obstructing the understanding of the security problem at hand. As a remedy, we propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible. Furthermore, we identify open problems when applying machine learning in security and provide directions for further research.

AB - With the growing processing power of computing systems and the increasing availability of massive datasets, machine learning algorithms have led to major breakthroughs in many different areas. This development has influenced computer security, spawning a series of work on learning-based security systems, such as for malware detection, vulnerability discovery, and binary code analysis. Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance and render learning-based systems potentially unsuitable for security tasks and practical deployment.In this paper, we look at this problem with critical eyes. First, we identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We conduct a study of 30 papers from top-tier security conferences within the past 10 years, confirming that these pitfalls are widespread in the current security literature. In an empirical analysis, we further demonstrate how individual pitfalls can lead to unrealistic performance and interpretations, obstructing the understanding of the security problem at hand. As a remedy, we propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible. Furthermore, we identify open problems when applying machine learning in security and provide directions for further research.

M3 - Conference contribution

BT - 31st USENIX Security Symposium (USENIX Sec)

ER -