Probabilistic 3D Multi-Object Cooperative Tracking for Autonomous Driving via Differentiable Multi-Sensor Kalman Filter

Hsu-kuang Chiu1, Chien-Yi Wang2, Min-Hung Chen2, Stephen F. Smith1
1Carnegie Mellon University, 2NVIDIA Research
Paper arXiv Video Code

IEEE International Conference on Robotics and Automation (ICRA), 2024



DMSTrack: Differentiable Multi-Sensor Kalman Filter for 3D Multi-Object Cooperative Tracking

Overview

We propose a novel algorithm: Differentiable Multi-Sensor Kalman Filter for 3D Multi-Object Cooperative Tracking (DMSTrack). Our algorithm is designed to be capable of estimating observation noise covariance of each detection from different Connected Autonomous Vehicles (CAVs) to better take advantage of the Kalman Filter’s theoretical optimality property: minimizing the expected error of state estimation.

To train our Differentiable Multi-Sensor Kalman Filter neural network model, we use each detection's local point cloud BEV feature and the positional feature as input. We calculate the regression loss by measureing the difference between the tracking result and the ground-truth.

The experiment results show that our algorithm improves the tracking accuracy by 17% with only 0.037x communication costs compared with the state-of-the-art method in V2V4Real. For more details, please refer to our paper at arxiv.

Tracking Results

The following videos show our DMSTrack's tracking results of different cooperative driving scenatios in the validation set. Different colors represent different tracking IDs.










BibTeX

@ARTICLE{chiu2024probabilistic,
  title={Probabilistic 3D Multi-Object Cooperative Tracking for Autonomous Driving via Differentiable Multi-Sensor Kalman Filter},
  author={Chiu, Hsu-kuang and Wang, Chien-Yi and Chen, Min-Hung and Smith, Stephen F.},
  journal={IEEE International Conference on Robotics and Automation (ICRA)},
  year={2024}
}