Self-checkout system using vision-based object detection and subcategory recognition

Abstract

The purpose of this research was to study and develop a conceptual framework for self-checkout systems to use only cameras at the checkout station to detect goods from customers. This framework is designed to use deep learning models and computer vision to classify similar products or identify sub-category products with similar appearances but different sizes. The framework is flexible to use with no retraining whenever a new product or package is introduced by using feature point extraction and feature point matching. The researcher uses the YOLO model to detect and identify the shape of the products and also defines the bounding boxes in the detected image, and then defines the feature point of the product using feature point algorithms (e.g. SIFT, ORB, and BRISK) and feature matching with reference images. The results from the experiment on the Thai products dataset that was collected by the researcher found that this framework can be applied to effectively detect and recognize retail products in a self-checkout system. The YOLO model can predict the shape and position of the product with an average mAP of 0.727, and the best feature point algorithm for product classification from this experiment is SIFT, which has an accuracy of 82.96%. In summary, SIFT can improve the accuracy of YOLO. Because of SIFT, the system can accurately predict the product class, although the YOLO bounding box does not cover an entire region occupied by a product. However, this experiment has a limitation because SIFT has low accuracy when predicting products in bottle shape. This is expected to be a result of the light and shadows reflected from the bottles. Therefore, the environment at the checkout area should be controlled. The product images should be clear, have no obstructions, and be free from disturbing light and shadows.