Human and Luggage Analysis for Abandoned Object Detection

Main Article Content

Patinya Tantawiwat
Trairat Sabaichai
Datchakorn Tancharoen
Nattachai Watcharapinchai
Sitapa Rujikietgumjorn

Abstract

   This paper presents a method to analyze videos for Abandoned Object Detection (AOD) and ownership identification. Abandoned luggage is a risky event that can actually occur in many public areas such as train stations, airports, or other important places in the building. Deep learning is used for person and luggage detection. The dataset is trained for both people and luggage images including backpacks, handbags and suitcases for more than 12,000 images. In this work, we use the YOLOv3 model that can be processed in real time with 98% accuracy. For ownership identification and abandoned detection, we propose a method that evaluates the spatial relationships and trajectories of each person and luggage. Ownership identification yields 65.1% accuracy while abandoned detection provides 66.6% accuracy.

Article Details

Section
Research Article

References

E. Luna, J. C. San Miguel, D. Ortego, and J. M. Martínez, “Abandoned object detection in video-surveillance: survey and comparison,” Sensors, vol. 18, no. 12, pp. 1–32, Dec. 2018.

F. Porikli, “Detection of temporarily static regions by processing video at different frame rates,” in Proc. IEEE Int. Conf. Adv. Video Signal-based Surveillance, London, UK, Sep. 5–7, 2007, pp. 236–241.

F. Porikli, Y. Ivanov, and T. Haga, “Robust abandoned object detection using dual foregrounds,” EURASIP J. Adv. Signal Process., vol. 2008, Oct. 2007, Art. no. 197875 (2007).

J. Redmon, S. Divvala, R. Girshick and A. Farhadi, “You only look once: Unified, real-time object detection,” 29th IEEE Conf. Comput. Vision Pattern Recognit., Las Vegas, NV, USA, Jun. 27–30, 2016, pp. 779–788.

R. Girshick, J. Donahue, T. Darrell and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in IEEE Conf. Comput. Vision Pattern Recognit., Columbus, OH, USA, Jun. 23–28, 2014, pp. 580–587.

T. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan and S. Belongie, “Feature pyramid networks for object detection,” in IEEE Conf. Comput. Vision Pattern Recognit., Honolulu, HI, USA, Jul. 21–26, 2017, pp. 936–944, doi: 10.1109/CVPR.2017.106.

T. Bouwmans, F. Porikli, B. Höferlin, and A. Vacavant, Background Modeling and Foreground Detection for Video Surveillance, Boca Raton, FL, USA: CRC Press, 2014.

A. Borji, M. Cheng, H. Jiang and J. Li, “Salient object detection: A benchmark,” IEEE Trans. Image Process., vol. 24, no. 12, pp. 5706–5722, Dec. 2015.

F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Sorkine-Hornung, “A Benchmark dataset and evaluation methodology for video object segmentation,” in 29th IEEE Conf. Comput. Vision Pattern Recognit., Las Vegas, NV, USA, Jun. 27–30, 2016, pp. 724–732.

A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient convolutional neural networks for mobile vision applications,” 2017. [Online]. Available: arXiv:1704.04861.

S. Smeureanu and R. T. Ionescu, “Real-time deep learning method for abandoned luggage detection in video,” in Proc. 26th Eur. Signal Process. Conf. EUSIPCO, Rome, Italy, Sep. 3–7, 2018, pp. 1775–1779.

X. Xie, C. Wang, S. Chen, G. Shi, and Z. Zhao, “Real-time illegal parking detection system based on deep learning,” in Int. Conf. Deep Learn. Technol., Chengdu, China, Jun. 2–4, 2017, pp. 23–27.

C.-Y. Lin, K. Muchtar, and C.-H. Yeh, “Robust techniques for abandoned and removed object detection based on Markov random field,” J. Vis. Commun. Image Represent., vol. 39, pp. 181–195, Aug. 2016.

I. Dahi, M. C. E. Mezouar, N. Taleb, and M. Elbahri, “An edge-based method for effective abandoned luggage detection in complex surveillance videos,” Comput. Vis. Image Underst., vol. 158, pp. 141–151, May 2017.

J. Kim and D. Kim, “Accurate abandoned and removed object classification using hierarchical finite state machine,” Image Vis. Comput., vol. 44, pp. 1–14, Dec. 2015.

T. Lin et al., “Microsoft COCO: Common Objects in Context,” 2014. [Online]. Available: arXiv:1405.0312.

A. Kuznetsova et al., “The Open Images Dataset V4 Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale,” Int. J. Comput. Vis., vol. 128, pp. 1956–1981, Mar. 2020.

i-Lids dataset for AVSS 2007, Sep. 2007. [Online]. Available: http://www.eecs.qmul.ac.uk/~andrea/avss2007_d.html

P. L. Venetianer, Z. Zhang, W. Yin, and A. J. Lipton, “Stationary target detection using the objectvideo surveillance system,” in IEEE Conf. Adv. Video Signal-based Surveillance, London, UK, Sep. 5–7, 2007, pp. 242–247.