Maritime object tracking from a commercial vessel using radars, electro-optical cameras and navigational instruments

The aim of this work was to develop partial maritime situational awareness based on a tracking system using two radars, 12 cameras and common navigational instruments mounted on a commercial vessel operating on the coast of Norway. Objects on the sea surface are tracked based on detections extracted from sensor images which are segmented by convolutional neural networks. The tracker structure consists of a JIPDA filter where marginalized association probabilities are approximated using loopy belief propagation and track initialization is performed using intensities from a PHD filter. The novelties of the proposed tracker are: a) the application to data from a large commercial vessel in operation, b) the use of heterogeneous sensors (multiple radars and optical cameras), and c) the introduction of a visibility state per sensor type enabling the tracker to maintain tracks for objects that are invisible to one of the sensor types. Although no ground truth was available, the results indicate that the implementation efficiently confirms true tracks, discards false tracks, finds and maintains tracks on objects that are (temporarily) invisible to one of the sensor types, and provides increased accuracy compared to a single sensor setup. Video of tracking results available at: https://vimeo.com/534840034 .