On the Development of Next Generation Memory Exploits

Research output: ThesisDoctoral Thesis

336 Downloads (Pure)


Memory vulnerabilities can be dangerous. To counter their effects, software and hardware support is being developed for Control-Flow Integrity (CFI): a technique to stop classical exploits from working. Unfortunately, there remains an underexplored residual attack surface. We refer to exploits targeting this attack surface as Next Generation Memory Exploits (NGMEs). This dissertation focuses on attacks and defences of NGMEs through an exploitation model of three phases: vulnerability, control and payload. We discuss vulnerabilities as a whole and propose new, measurable properties for vulnerabilities. These properties form the foundation of a vulnerability taxonomy called GEN, as it GENerically describes and classifies vulnerabilities. GEN defines what a vulnerability is through a technical definition. Moreover, GEN identifies what vulnerabilities are relevant for NGMEs and how to find them.
Typically, memory vulnerabilities expose limited capabilities. Control techniques are necessary to exploit such vulnerabilities. Heap Layout Manipulation is one technique to overwrite useful data structures. We propose a toolchain that can generate puzzles from real-world applications, to then be solved in a puzzle game called Hack the Heap. This way, we can solve the Heap Layout Manipulation problem through gamification while remaining heap manager agnostic and explainable. With control gained over a vulnerable application, the next step is to deliver the payload: what the exploit wants to achieve. To halt this payload, we propose System Call Argument Integrity: automatic data-flow protection tailored towards security-sensitive system calls. It protects against data-only attacks while incurring overhead only when handling security-sensitive data-flows. Concluding, we propose a novel framework for characterising NGMEs (and vulnerabilities more broadly), in addition to techniques for assessing and mitigating their impact during different phases of the exploitation lifecycle. Considering the increasing risk of real-world NGMEs, we hope this dissertation fosters further research into both NGME threats and mitigations.
Original languageEnglish
Awarding Institution
  • Royal Holloway, University of London
  • O'Keeffe, Daniel, Supervisor
  • Blasco Alis, Jorge, Advisor
  • Cavallaro, Lorenzo, Advisor
  • Sgandurra, Daniele, Supervisor
  • Kinder, Johannes, Advisor
Thesis sponsors
Award date1 Mar 2023
Publication statusUnpublished - 8 Feb 2023


  • memory vulnerabilities
  • memory
  • heap
  • CFI
  • control-flow integrity
  • data-only attacks
  • DOP
  • ROP
  • exploitation
  • exploit development

Cite this