Detecting Carried Objects in Short Video Sequences

Dima Damen, David Hogg

We have developed a new method for detecting objects carried by people. It detects visible objects, larger than a few centimeters, carried by people walking in any direction in a scene that is not too crowded. It works by comparing tracked individuals with an idealised shape model of a pedestrian not carrying any objects. Protrusions beyond the expected silhouette are potential carried objects.

Software Download

When using this code, kindly reference our TPAMI paper below.
MATLAB code of the software is now available with details of how to use it - download (3.0MB)
A short sequence from PETS2006 to test the code - download (3.9MB)

Demos and Videos

Demo of the approach AVI (4.8MB)
Short Presentation PDF (1.2MB)
Results on PETS2006 dataset AVI (11.8MB)
Other results MPG (1.4MB)

Publications

Damen, Dima and Hogg, David (2012). Detecting Carried Objects from Sequences of Walking Pedestrians. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) vol 34 (6) pp 1056-1067 pdf

Dima Damen and David Hogg (2008). Detecting Carried Objects in Short Video Sequences. European Conference on Computer Vision (ECCV) Springer-Verlag. 3,154-167 pdf, poster

Annotation of PETS2006 carried objects is available here (in VIPER and text file formats).

The Method


(a)	(b)	(c)	(d)	(e)

(a) One frame from a short sequence
(b) The motion template obtained from background subtraction
(c) Aligned motion template of a pedestrian walking in the same direction and carrying nothing
(d) A map of the protrusions that might correspond to carried objects
(e) Detected carried objects: The MAP solution for a Markov random field over this protrusion map, combining a prior on the location of carried objects with a spatial continuity assumption