Department of Computer Science: MSc Thesis Presentation

AIR: Aerial Inspection RetinaNet for Land Search and Rescue Missions

Author: Pasi Pyrrö
Supervisor: Alex Jung

Date: Thursday 14 October 2021
Time: 13:00-14:00
Zoom: https://aalto.zoom.us/j/62447155959 (passcode: 857622)

Abstract: Search and rescue (SAR) missions have been carried out for centuries to aid those who are lost or in distress, typically in some remote areas, such as wilderness. With recent advances in technology, small unmanned aerial vehicles or drones have been used during SAR missions for years in many countries. The reason is that drones enable rapid aerial photographing of large areas with potentially difficult-to-reach terrain, which can even replace several land search parties in efficiency. However, there remains the issue of inspecting a vast amount of aerial drone images for tiny clues about the missing person location, which is currently a manual task done by humans in most cases. It turns out this inspection process is very slow, tedious and error-prone for most people, and can significantly delay the entire aerial drone searches operation.

In this thesis, we propose a deep learning based object detection approach to automate this drone footage inspection task. As such, we use the HERIDAL data set of aerial imagery from Mediterranean landscape to train our detector, and the goal is to outperform existing object detection methods on the HERIDAL test data. Consequently, we perform hyperparameter tuning, model architecture selection, online data augmentation, image tiling, confidence score threshold calibration and various other tricks to improve the test performance of our method. Finally, we present Aerial Inspection RetinaNet (AIR) as the outcome of these various experiments, which is our solution to this aerial person detection (APD) problem. Moreover, we demonstrate state-of-the-art results for the AIR detector on the HERIDAL test data in terms of both speed and precision.

In addition, we formalize the SAR-APD problem of the HERIDAL data set, and define a modified evaluation scheme, which ranks detectors in terms of real-world SAR localization requirements, which are much looser than in typical object detection tasks. Moreover, we devise an estimator for average human detection performance via a meta-analysis study, which can be used as baseline for comparing APD methods. Finally, we propose a novel bounding box aggregation method, the merging of overlapping bounding boxes (MOB) algorithm for robust, approximate object localization.