Public defence in Signal Processing and Data Analytics, M.Sc.(Tech.) Petteri Pulkkinen

The title of the thesis: Model-based reinforcement learning for integrated radar and communications systems

Thesis defender: Petteri Pulkkinen
Opponents: Prof. Rick S. Blum, Lehigh University, US and Prof. Christos Masouros, University College, London, UK
Custos: Prof. Visa Koivunen, Aalto University School of Electrical Engineering

Traditionally, radar and communications systems have been distinct and operated on dedicated hardware, radio frequencies, and waveforms. However, there is a convergence between radar and communications technologies that partly facilitates combining these functionalities into unified systems, referred to as integrated sensing and communications (ISAC) systems. Such ISAC systems are an integral part of the emerging 6G wireless networks, enabling new applications utilizing radar-based environmental sensing. However, this integration introduces new challenges for radio waveform design and resource management. 

Optimization of waveforms and resources for ISAC systems has been extensively studied in the literature. Most existing methods rely on structured optimization, which cannot learn from previous experiences and requires precise knowledge of models and their parameters. In practice, these models may be inaccurate, and their parameters can be difficult to estimate, especially in dynamic radio environments. Therefore, the dissertation focuses on data-driven reinforcement learning methods that enable ISAC systems to learn from their experiences in real-time and to avoid modelling deficiencies. 

Applying reinforcement learning in the context of ISAC systems is not straightforward. These methods typically require large amounts of data from interactions between the system and its operational environment, which is not desirable in practice. Furthermore, the learnt decision policies may not be interpretable. Therefore, the methods in the dissertation are based on model-based reinforcement learning, which utilizes learning only in those parts where it provides added value, and otherwise, the rich domain knowledge is used. This allows the methods to be more sample-efficient and interpretable. 

The dissertation develops new theories and performance bounds for model-based reinforcement learning in the context of ISAC systems. Additionally, the methods are evaluated through practical examples of multicarrier and multiantenna ISAC systems operating in dynamic radio interference and radar target environments. The results indicate that model-based reinforcement learning can be highly data-efficient and significantly improve ISAC key performance indicators.

Thesis available for public display 7 days prior to the defence at Aaltodoc. 

Contact:
https://www.linkedin.com/in/petteri-pulkkinen-12a900146/
petteri.pulkkinen@aalto.fi