A1 Refereed original research article in a scientific journal
Deep Learning Based Multi-Modal Fusion Architectures for Maritime Vessel Detection
Authors: Fahimeh Farahnakian, Jukka Heikkonen
Publisher: MDPI
Publication year: 2020
Journal: Remote Sensing
Journal name in source: REMOTE SENSING
Journal acronym: REMOTE SENS-BASEL
Article number: ARTN 2509
Volume: 12
Issue: 6
Number of pages: 17
eISSN: 2072-4292
DOI: https://doi.org/10.3390/rs12162509
Self-archived copy’s web address: https://research.utu.fi/converis/portal/detail/Publication/50340179
Abstract
Object detection is a fundamental computer vision task for many real-world applications. In the maritime environment, this task is challenging due to varying light, view distances, weather conditions, and sea waves. In addition, light reflection, camera motion and illumination changes may cause to false detections. To address this challenge, we present three fusion architectures to fuse two imaging modalities: visible and infrared. These architectures can provide complementary information from two modalities in different levels: pixel-level, feature-level, and decision-level. They employed deep learning for performing fusion and detection. We investigate the performance of the proposed architectures conducting a real marine image dataset, which is captured by color and infrared cameras on-board a vessel in the Finnish archipelago. The cameras are employed for developing autonomous ships, and collect data in a range of operation and climatic conditions. Experiments show that feature-level fusion architecture outperforms the state-of-the-art other fusion level architectures.
Object detection is a fundamental computer vision task for many real-world applications. In the maritime environment, this task is challenging due to varying light, view distances, weather conditions, and sea waves. In addition, light reflection, camera motion and illumination changes may cause to false detections. To address this challenge, we present three fusion architectures to fuse two imaging modalities: visible and infrared. These architectures can provide complementary information from two modalities in different levels: pixel-level, feature-level, and decision-level. They employed deep learning for performing fusion and detection. We investigate the performance of the proposed architectures conducting a real marine image dataset, which is captured by color and infrared cameras on-board a vessel in the Finnish archipelago. The cameras are employed for developing autonomous ships, and collect data in a range of operation and climatic conditions. Experiments show that feature-level fusion architecture outperforms the state-of-the-art other fusion level architectures.
Downloadable publication This is an electronic reprint of the original article. |  |
UTU Research Portal