JSTSP 2008 – Target detection and tracking with heterogeneous sensors

Huiyu Zhou, Murtaza Taj, Andrea Cavallaro
IEEE Journal of Selected Topics in Signal Processing, 2008

Abstract

We present a multimodal detection and tracking algorithm for sensors composed of a camera mounted between two microphones. Target localization is performed on color-based change detection in the video modality and on Time Difference of Arrival (TDOA) estimation between the two microphones in the audio modality. The TDOA is computed by multi-band Generalized Cross Correlation (GCC) analysis. The estimated directions of arrival are then post-processed using a Riccati Kalman filter. The visual and audio estimates are finally integrated, at the likelihood level, into a particle filter (PF) that uses a zero-order motion model, and a Weighted Probabilistic Data Association (WPDA) scheme. We demonstrate that the Kalman filtering (KF) improves the accuracy of the audio source localization and that the WPDA helps to enhance the tracking performance of sensor fusion in reverberant scenarios. The combination of multi-band GCC, KF and WPDA within the particle filtering framework improves the performance of the algorithm in noisy scenarios. We also show how the proposed audiovisual tracker summarizes the observed scene by generating metadata that can be transmitted to other network nodes instead of transmitting the raw images and can be used for very low bit rate communication. Moreover, the generated metadata can also be used to detect and monitor events of interest.

Resources

Download PDF
(Distributed here for timely dissemination of scholarly work. Copyright retained by copyright holders. May not be posted without permission from copyright holders.)

Demo: Youtube

Text Reference:

H. Zhou, M. Taj and A. Cavallaro,
"Target detection and tracking with heterogeneous sensors",
in IEEE Journal of Selected Topics in Signal Processing, vol. 2, no. 4, pp. 503-513, August 2008

Bibtex Reference:

@ARTICLE{Maggio08TCSVT,
 title={Efficient multi-target visual tracking using Random Finite Sets},
 author={E. Maggio and M. Taj and A. Cavallaro},
 journal=IEEE_CSVT,
 year={2008},
 month={August},
 volume={18},
 number={8},
 pages={1016-1027},
}