Public defence in Computer Science, M.Sc. Shaghayegh Roohi

Title of the doctoral thesis: Advances in AI-assisted Game Testing

Opponent: Associate Professor Julian Togelius, Tandon School of Engineering, New York University, USA
Custos: Associate Professor Perttu Hämäläinen, Aalto University School of Science, Department of Computer Science

The thesis is publicly displayed 10 days before the defence in the publication archive Aaltodoc of Aalto University.

Electronic thesis

Public defence announcement:

In this dissertation, we have studied the application of machine learning and artificial intelligence in game testing, to provide new tools for games user researchers and game developers. We propose and evaluate new ways of utilizing deep learning and reinforcement learning to reduce the time and resources needed for game testing. Here we focus on the player experience aspect of game testing.

Emotion is an important part of player experience and rests at the center of the first two publications included in the dissertation. We introduce Affect Gradient for measuring emotional facial expression changes at game events, which can be exploited for investigating and summarizing the playtest videos. However, we remain cautious about the direct application of automatic facial expression analysis in guiding game design, since players might express confusing emotional cues, e.g., frowning in concentration which an AI system may interpret as anger. We also highlight how widely available game streams could be a valuable resource for game testing. In the second paper, we present a game stream dataset and apply multimodal emotion recognition to it. The results indicate that machine learning has difficulty recognizing subtle and fine-grained emotions. However, it could accurately highlight emotionally salient game events.

In the other two publications, we study simulation-based approaches for modeling player experience and behavior. Particularly, the relation between game level pass and churn rates, indicators of level difficulty and engagement, is modeled over game levels. Deep Reinforcement Learning (DRL) agents are employed to play a match-3 game and based on their performance the game levels’ difficulties are estimated. Then, the rule-based simulation of player population changes over game levels predicts the game levels’ pass and churn rates. Moreover, the last publication presents technical improvements of the method.

In summary, we demonstrate that machine learning facilitates the game testing process in two ways. First, it might accelerate the analysis of human playtester data by highlighting the important parts of playtest videos. Second, it can augment high-cost human data with synthetic data generated by AI agents. The synthetic data could help detect problematic game levels in the early stages of game development. Furthermore, this work calls for more research on game agents equipped with computational models of emotions.

Contact details of the doctoral student: [email protected], +358414986176