Journal of Shanghai Jiao Tong University

Journal of Shanghai Jiaotong University >

2023 , Vol. 57 >Issue 3: 297 - 308

DOI: https://doi.org/10.16183/j.cnki.jsjtu.2021.301

Mechanical Engineering

A Pallet Recognition Method Based on Adaptive Color Fast Point Feature Histogram

Expand
  • 1. College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
    2. Noblelift Intelligent Equipment Co., Ltd., Huzhou 313199, Zhejiang, China
    3. Engineering Research Center of Intelligent Logistics Equipment of Zhejiang Province, Huzhou 313199, Zhejiang, China

Received date: 2021-08-12

  Accepted date: 2021-10-08

  Online published: 2022-08-23

Fold

Abstract

Pallet recognition is one of the critical technologies of cargo handling for unmanned industrial vehicles. A pallet recognition method based on adaptive color fast point feature histogram (ACFPFH) is proposed to solve the problems of current recognition methods such as low efficiency, time-consuming, poor robustness and random parameter selection. The Kinect V2 sensor is used to collect the point cloud data which represents the whole scene including pallet. Next, outliers are removed and the optimal neighborhood radius of each point is obtained based on the minimum criterion of neighborhood feature entropy function. Then, the key points are extracted from scene point clouds. The ACFPFH consisting of color feature and adaptive geometric feature is applied for similarity matching between the template and scene point clouds. Finally, wrong feature correspondences are rejected and the pallet in the scene point cloud is recognized. A comparison of the fast point feature histogram with the fixed neighbor radius of 0.012 m shows that the pallet recognition precision and efficiency of the method based on ACFPFH is improved by 83.74% and 35.55% respectively, which verifies the superiority of the proposed method.

Key words： fast point feature histogram; feature extraction; adaptive optimal neighborhood; pallet recognition

Cite this article

ZHAN Yan, CHEN Zhihui, ZHU Baochang, ZHU Tingting, SHAO Yiping, LU Jiansha . A Pallet Recognition Method Based on Adaptive Color Fast Point Feature Histogram[J]. Journal of Shanghai Jiaotong University, 2023 , 57(3) : 297 -308 . DOI: 10.16183/j.cnki.jsjtu.2021.301

References

[1] CASADO F, LAPIDO Y L, LOSADA D P, et al. Pose estimation and object tracking using 2D images[J]. Procedia Manufacturing, 2017, 11: 63-71.
[2] 王伟男, 杨朝红. 基于图像处理技术的目标识别方法综述[J]. 电脑与信息技术, 2019, 27(6): 9-15.
[2] WANG Weinan, YANG Chaohong. A survey of target recognition methods based on image processing technology[J]. Computer and Information Technology, 2019, 27(6): 9-15.
[3] 郝雯, 王映辉, 宁小娟, 等. 面向点云的三维物体识别方法综述[J]. 计算机科学, 2017, 44(9): 11-16.
[3] HAO Wen, WANG Yinghui, NING Xiaojuan, et al. Survey of 3D object recognition for point clouds[J]. Computer Science, 2017, 44(9): 11-16.
[4] GARCíA-PULIDO J A, PAJARES G, DORMIDO S, et al. Recognition of a landing platform for unmanned aerial vehicles by using computer vision-based techniques[J]. Expert Systems With Applications, 2017, 76: 152-165.
[5] CHEN J M, CHEN L P. Multi-dimensional color image recognition and mining based on feature mining algorithm[J]. Automatic Control and Computer Sciences, 2021, 55(2): 195-201.
[6] SEIDENARI L, SERRA G, BAGDANOV A D, et al. Local pyramidal descriptors for image recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 1033-1040.
[7] CHEN G, PENG R, WANG Z C, et al. Pallet recognition and localization method for vision guided forklift[C]// 2012 8th International Conference on Wireless Communications, Networking and Mobile Computing. Shanghai, China: IEEE, 2012: 1-4.
[8] SYU J L, LI H T, CHIANG J S, et al. A computer vision assisted system for autonomous forklift vehicles in real factory environment[J]. Multimedia Tools and Applications, 2017, 76(18): 18387-18407.
[9] LI T J, HUANG B, LI C, et al. Application of convolution neural network object detection algorithm in logistics warehouse[J]. The Journal of Engineering, 2019, 2019(23): 9053-9058.
[10] SHAO Y P, WANG K, DU S C, et al. High definition metrology enabled three dimensional discontinuous surface filtering by extended tetrolet transform[J]. Journal of Manufacturing Systems, 2018, 49: 75-92.
[11] SHAO Y P, DU S C, TANG H T. An extended bi-dimensional empirical wavelet transform based filtering approach for engineering surface separation using high definition metrology[J]. Measurement, 2021, 178: 109259.
[12] 武文汉, 杨明, 王冰, 等. 一种基于轮廓匹配的仓储机器人托盘检测方法[J]. 上海交通大学学报, 2019, 53(2): 197-202.
[12] WU Wenhan, YANG Ming, WANG Bing, et al. Pallet detection based on contour matching for warehouse robots[J]. Journal of Shanghai Jiao Tong University, 2019, 53(2): 197-202.
[13] XIAO J H, LU H M, ZHANG L L, et al. Pallet recognition and localization using an RGB-D camera[J]. International Journal of Advanced Robotic Systems, 2017, 14(6): 172988141773779.
[14] VARGA R, COSTEA A, NEDEVSCHI S. Improved autonomous load handling with stereo cameras[C]// 2015 IEEE International Conference on Intelligent Computer Communication and Processing. Cluj-Napoca, Romania: IEEE, 2015: 251-256.
[15] VARGA R, NEDEVSCHI S. Robust pallet detection for automated logistics operations[C]// Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. Rome, Italy: SCITEPRESS-Science and Technology Publications, 2016: 470-477.
[16] 吴登禄, 曹文希, 朱颖. 基于三维点云和图像边缘的托盘检测技术研究[J]. 自动化与信息工程, 2019, 40(3): 40-42.
[16] WU Denglu, CAO Wenxi, ZHU Ying. Research on pallet detection technology based on 3D point cloud and image edge features[J]. Automation & Information Engineering, 2019, 40(3): 40-42.
[17] 李洋洋, 史历程, 万卫兵, 等. 基于卷积神经网络的三维物体检测方法[J]. 上海交通大学学报, 2018, 52(1): 7-12.
[17] LI Yangyang, SHI Licheng, WAN Weibing, et al. A convolutional neural network-based method for 3D object detection[J]. Journal of Shanghai Jiao Tong University, 2018, 52(1): 7-12.
[18] TERABAYASHI K, TAKASHIMA I, SUZUKI Y, et al. Easy acquisition of range image dataset for object detection using retroreflective markers and a time-of-flight camera: An application to detection of forklift pallets[C]// Proceedings of the Seventh Asia International Symposium on Mechatronics. Hangzhou, China: Springer Singapore, 2020: 1001-1005.
[19] 郭裕兰, 鲁敏, 谭志国, 等. 距离图像局部特征提取方法综述[J]. 模式识别与人工智能, 2012, 25(5): 783-791.
[19] GUO Yulan, LU Min, TAN Zhiguo, et al. Survey of local feature extraction on range images[J]. Pattern Recognition and Artificial Intelligence, 2012, 25(5): 783-791.
[20] PRAKHYA S M, LIN J, CHANDRASEKHAR V, et al. 3DHoPD: A fast low-dimensional 3-D descriptor[J]. IEEE Robotics and Automation Letters, 2017, 2(3): 1472-1479.
[21] RUSU R B, BLODOW N, BEETZ M. Fast point feature histograms (FPFH) for 3D registration[C]// 2009 IEEE International Conference on Robotics and Automation. Kobe, Japan: IEEE, 2009: 3212-3217.
[22] GUO Y L, BENNAMOUN M, SOHEL F, et al. A comprehensive performance evaluation of 3D local feature descriptors[J]. International Journal of Computer Vision, 2016, 116(1): 66-89.
[23] HUANG J, YOU S Y. Detecting objects in scene point cloud: A combinational approach[C]// 2013 International Conference on 3D Vision-3DV 2013. Seattle, WA, USA: IEEE, 2013: 175-182.
[24] 王斐, 梁宸, 韩晓光, 等. 基于焊件识别与位姿估计的焊接机器人视觉引导[J]. 控制与决策, 2020, 35(8): 1873-1878.
[24] WANG Fei, LIANG Chen, HAN Xiaoguang, et al. Visual guidance of welding robot based on weldment recognition and pose estimation[J]. Control and Decision, 2020, 35(8): 1873-1878.
[25] LIU J, BAI D, CHEN L. 3-D point cloud registration algorithm based on greedy projection triangulation[J]. Applied Sciences, 2018, 8(10): 1776.
[26] LI P, WANG J, ZHAO Y D, et al. Improved algorithm for point cloud registration based on fast point feature histograms[J]. Journal of Applied Remote Sensing, 2016, 10: 045024.
[27] NAPOLI A, GLASS S, WARD C, et al. Performance analysis of a generalized motion capture system using microsoft kinect 2.0[J]. Biomedical Signal Processing and Control, 2017, 38: 265-280.
[28] DEMANTKE J, MALLET C, DAVID N, et al. Dimensionality based scale selection in 3D LIDAR point clouds[C]// ISPRS Workshop Laser Scanning. Calgary, Canada: Copernicus Gesellschaft Mbh, 2011: 97-102.
[29] WEINMANN M, JUTZI B, MALLET C. Semantic 3D scene interpretation: A framework combining optimal neighborhood size selection with relevant features[J]. Photogrammetry, Remote Sensing and Spatial Information Sciences, 2014, 2(3): 181-188.
[30] 王红雨, 尹午荣, 汪梁, 等. 基于HSV颜色空间的快速边缘提取算法[J]. 上海交通大学学报, 2019, 53(7): 765-772.
[30] WANG Hongyu, YIN Wurong, WANG Liang, et al. Fast edge extraction algorithm based on HSV color space[J]. Journal of Shanghai Jiao Tong University, 2019, 53(7): 765-772
[31] 熊风光, 蔡晋茹, 况立群, 等. 三维点云模型中特征点描述子及其匹配算法研究[J]. 小型微型计算机系统, 2017, 38(3): 640-644.
[31] XIONG Fengguang, CAI Jinru, KUANG Liqun, et al. Study on descriptor and matching algorithm of feature point in 3D point cloud[J]. Journal of Chinese Computer Systems, 2017, 38(3): 640-644.
[32] 唐敏杰, 赵欢, 丁汉. 二进制点云局部特征描述子研究[J]. 机械工程学报, 2021, 57(2): 219-229.
[32] TANG Minjie, ZHAO Huan, DING Han. Research on binarized local feature descriptors of point clouds[J]. Journal of Mechanical Engineering, 2021, 57(2): 219-229.
Options
Outlines

/

〈 〉