Optimizing pick-place operations: Leveraging k-means for visual object localization and decision-making in collaborative robots
Article Sidebar
Main Article Content
Abstract
This article presents an approach to object localization algorithms for pick-place operations in collaborative robots by utilizing conventional color segmentation in computer vision and k-means clustering. Adding the k-means clustering algorithm complements the color segmentation by distinguishing and grouping the sections of similar pixels; hence, object localization is more accurate. The order of pick-place operations of each cluster acquired from the proposed algorithm is prioritized based on norm. Integrating the proposed framework provides a well-structured depiction of the localized objects, which is fundamental for successful pick-place operations. The TCP/IP communication framework via socket communication is established to facilitate data transmission between the robot and the host computer. The objective is to ensure that the robot's end effector performs as directed by the host computer by obtaining information on the pick-and-place operation, including the localized coordinates, dimensions, the order of operations, and the pose of the objects of interest to the robot. In this experiment, a cobot arm is employed to autonomously pick and place objects with different shapes and colors in a workspace filled with diverse objects, requiring the robot to choose the closest objects to operate based on the data from the host computer. Our results demonstrate the effectiveness of this integration, showcasing the enhanced adaptability and efficiency of pick-place operations in collaborative robots. This study indicates 98% accuracy in pick-and-place operations with an average latency of 0.52 ± 0.1 s, indicating an improvement compared to the traditional algorithm without k-means clustering, which achieves an accuracy of 88%. Additional studies reveal that when incorporating pose estimation into the pick-place operations, the proposed algorithm's accuracy is 94%. The demonstration highlights the potential of leveraging machine learning algorithms and computer vision from the camera to perform flexible pick-place operations via socket communication.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Preuveneers D, Ilie-Zudor E. The intelligent industry of the future: A survey on emerging trends, research challenges and opportunities in Industry 4.0. J Amb Intel Smart En. 2017;9(3):287-98.
Kadir BA, Broberg O, Souza da Conceição C. Designing human-robot collaborations in industry 4.0: explorative case studies. In: DS 92: Proceedings of the DESIGN 2018 15th International Design Conference. 2018 May 21-24; Dubrovnik, Croatia. Dubrovnik: Design Society; 2018. p. 601-10.
Krüger M, Wiebel CB, Wersing H. From Tools Towards Cooperative Assistants. In: Proceedings of the 5th International Conference on Human Agent Interaction; 2017 Oct 17-20; Bielefeld, Germany. ACM; 2017. p. 287-94.
Li A, Gurocak H. Modified Bug Algorithm with Proximity Sensors to Reduce Human-Cobot Collisions. In: 2023 9th International Conference on Automation, Robotics and Applications (ICARA). 2023 Feb 10-12; Abu Dhabi, United Arab Emirates. Abu Dhabi: NYU Abu Dhabi; 2023. p. 80-5.
Syyrilä T. Development of a Collaborative Robot Manufacturing Workstation. [Master's Thesis]. Mechatronic System Design: Lahti University of Technology LUT; 2022.
Zhu Z, Hu H. Robot learning from demonstration in robotic assembly: A survey. Robotics. 2018;7(2):17.
Peternel L, Petrič T, Babič J. Robotic assembly solution by human-in-the-loop teaching method based on real-time stiffness modulation. Autonomous Robots. 2018;42:1-17.
Borboni A, Reddy KVV, Elamvazuthi I, AL-Quraishi MS, Natarajan E, Azhar Ali SS. The expanding role of artificial intelligence in collaborative robots for industrial applications: A systematic review of recent works. Machines. 2023;11:111.
Biggs G, MacDonald B. A survey of robot programming systems. In: Proceedings of the Australasian conference on robotics and automation. 2003. p. 1-3.
Hughes C, Hughes T. Robot programming: a guide to controlling autonomous robots. Vol. 1. Indianapolis, Indiana:Que; 2016.
Rossano GF, Martínez C, Hedelind M, Murphy S, Fuhlbrigge T. Easy robot programming concepts: An industrial perspective. In: 2013 IEEE International Conference on Automation Science and Engineering (CASE). 2013 Aug 17-20; Wisconsin, USA. IEEE; 2013. p. 1119-26.
Pedersen MR, Nalpantidis L, Andersen RS, Schou C, Bøgh S, Krüger V, et al. Robot skills for manufacturing: From concept to industrial deployment. Robotics and Computer-Integrated Manufacturing. 2016;37:282-91.
Manoharan M, Thilak KSS, Kumaraguru S. Collaborative motion synchronization for affordable open-source cobots. In: 2022 IEEE 6th Conference on Information and Communication Technology (CICT); 2022 Nov 18-20; Gwalior, India. IEEE; 2022. p.1-5.
Iriondo A, Lazkano E, Susperregi L, Urain J, Fernandez A, Molina J. Pick and place operations in logistics using a mobile manipulator controlled with deep reinforcement learning. Appl Sci. 2019;9(2):348.
Andhare P, Rawat S. Pick and place industrial robot controller with computer vision. In: 2016 International Conference on Computing Communication Control and automation (ICCUBEA). 2016 Aug 12-13; Pune, India. IEEE; 2016. p. 1-4.
Migimatsu T, Bohg J. Object-centric task and motion planning in dynamic environments. IEEE Robotics and Automation Letters. 2020;5:844-51.
Stulp F, Fedrizzi A, Beetz M. Learning and performing place-based mobile manipulation. In: 2009 IEEE 8th International Conference on Development and Learning; 2009 Jun 5-7; Shanghai, China. IEEE; 2009. p. 1-7.
Soltan S, Oleinikov A, Demirci MF, Shintemirov A. Deep learning-based object classification and position estimation pipeline for potential use in robotized pick-and-place operations. Robotics. 2020;9(3):63.
Kumar R, Kumar S, Lal S, Chand P. Object Detection and Recognition for a Pick and Place Robot. In: Asia-Pacific World Congress on Computer Science and Engineering; 2014 Nov 4-5; Nadi, Fiji. IEEE; 2015.
Lobbezoo A, Qian Y, Kwon HJ. Reinforcement learning for pick and place operations in robotics: A survey. Robotics. 2021;10(3):105.
Zhou W, Meng F, Li K. A cherry tomato classification-picking robot based on the K-means algorithm. J Phys: Conf Ser. 2020;1651:012126.
Anggi Maharani D, Fakhrurroja H, Riyanto R, Machbub C. Hand gesture recognition using K-means clustering and Support Vector Machine. In: 2018 IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE); 2018 Apr 28-29; Penang, Malaysia. IEEE; 2018. p. 1-6.
Wakayama T, Garcia Ricardez GA, Hafi LE, Takamatsu J. 6D-Pose Estimation for Manipulation in Retail Robotics using the Inference-embedded OAK-D Camera. In: 2022 IEEE/SICE International Symposium on System Integration (SII). 2022 Jan 9-12; Narvik, Norway. IEEE; 2022. p. 1046-51.
Abernathy A, Celebi ME. The incremental online k-means clustering algorithm and its application to color quantization. Expert Systems with Applications. 2022;207:117927.
Ezugwu AE, Ikotun AM, Oyelade OO, Abualigah L, Agushaka JO, Eke CI, et al. A comprehensive survey of clustering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and future research prospects. Engineering Applications of Artificial Intelligence. 2022;110:104743.
Hui-ping L, Dai-min C, Miao Y. Communication of multi-robot system on the TCP/IP. In: 2011 International Conference on Mechatronic Science, Electric Engineering and Computer (MEC). 2011 Aug 19-22; Jilin, China. IEEE; 2011. p. 1432-35.
Shin S, Yoon D, Song H, Kim B, Han J. Communication system of a segmented rescue robot utilizing socket programming and ROS. In: 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI). 2017 Jun 28-Jul 01; Jeju, Korea (South). IEEE; 2017. p. 565-9.
Jocher G, Chaurasia A, Qiu J. YOLO by Ultralytics (Version 8.0.0) [Computer software] [Internet]. GitHub, Inc, 2023. [cited 2023 Sep 10]. Available From: https://github.com/ultralytics/ultralytics/blob/main/CITATION.cff