Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection



: Fahimeh Farahnakian, Jukka Heikkonen

Publisher: MDPI

: 2020

: Remote Sensing

: REMOTE SENSING

: REMOTE SENS-BASEL

: ARTN 2509

: 12

: 6

: 17

: 2072-4292

DOI: https://doi.org/10.3390/rs12162509

: https://research.utu.fi/converis/portal/detail/Publication/50340179


Object detection is a fundamental computer vision task for many real-world applications. In the maritime environment, this task is challenging due to varying light, view distances, weather conditions, and sea waves. In addition, light reflection, camera motion and illumination changes may cause to false detections. To address this challenge, we present three fusion architectures to fuse two imaging modalities: visible and infrared. These architectures can provide complementary information from two modalities in different levels: pixel-level, feature-level, and decision-level. They employed deep learning for performing fusion and detection. We investigate the performance of the proposed architectures conducting a real marine image dataset, which is captured by color and infrared cameras on-board a vessel in the Finnish archipelago. The cameras are employed for developing autonomous ships, and collect data in a range of operation and climatic conditions. Experiments show that feature-level fusion architecture outperforms the state-of-the-art other fusion level architectures.

Last updated on 2024-26-11 at 20:19