Learning to Identify Bugs in Video Games

The use of intelligent software agents promises to revolutionise video game testing. While agents automate the time-consuming task of repeatedly playing a game in search of issues, humans can spend their time on the more creative aspects of game development. Despite the substantial advancements in game-playing that have made this possible, agents are reliant on humans, or hand-crafted guards, to determine whether there are issues with the game's design or functioning.

This thesis aimed to develop testing agents that can identify issues with a game's function or bugs with minimal human involvement by learning from their prior experiences. The problem is framed as one of anomaly detection, where bugs correspond to abnormality or novelty in an agent's experience. A series of approaches based on Self-Supervised Learning and Causal Inference have been developed to enable an agent to measure abnormality or otherwise model the game to subsequently identify bugs. The focus was on laying the foundations for testing agents that operate over the same input/output modalities as human testers. The approaches were evaluated by testing a diverse collection of purpose-built video games, where they successfully identified bugs from a broad class.

This thesis is among the first work to investigate the use of machine learning in the context of video game bug identification. It presents an exposition of the problem of learning intended behaviour, and then endeavours to develop solutions that demonstrate the benefits of using agents with learning capabilities for testing. Namely, ease of reuse across projects (reusability) and in identifying bugs that would otherwise require human involvement to be found (capability). The use of agents equipped with sophisticated game-playing algorithms and the identification tools outlined in this thesis offers a new framework for video game testing.