A Pallet Recognition Method Based on Adaptive Color Fast Point Feature Histogram

doi:10.16183/j.cnki.jsjtu.2021.301

Abstract

Abstract:

Pallet recognition is one of the critical technologies of cargo handling for unmanned industrial vehicles. A pallet recognition method based on adaptive color fast point feature histogram (ACFPFH) is proposed to solve the problems of current recognition methods such as low efficiency, time-consuming, poor robustness and random parameter selection. The Kinect V2 sensor is used to collect the point cloud data which represents the whole scene including pallet. Next, outliers are removed and the optimal neighborhood radius of each point is obtained based on the minimum criterion of neighborhood feature entropy function. Then, the key points are extracted from scene point clouds. The ACFPFH consisting of color feature and adaptive geometric feature is applied for similarity matching between the template and scene point clouds. Finally, wrong feature correspondences are rejected and the pallet in the scene point cloud is recognized. A comparison of the fast point feature histogram with the fixed neighbor radius of 0.012 m shows that the pallet recognition precision and efficiency of the method based on ACFPFH is improved by 83.74% and 35.55% respectively, which verifies the superiority of the proposed method.

Key words: fast point feature histogram, feature extraction, adaptive optimal neighborhood, pallet recognition

CLC Number:

TH166

ZHAN Yan, CHEN Zhihui, ZHU Baochang, ZHU Tingting, SHAO Yiping, LU Jiansha. A Pallet Recognition Method Based on Adaptive Color Fast Point Feature Histogram[J]. Journal of Shanghai Jiao Tong University, 2023, 57(3): 297-308.

Figures/Tables 23

Fig.1

Fig.2

Fig.3

Tab.1

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Fig.18

Tab.2

Fig.19

Tab.3

Tab.4

References 32

[1]	CASADO F, LAPIDO Y L, LOSADA D P, et al. Pose estimation and object tracking using 2D images[J]. Procedia Manufacturing, 2017, 11: 63-71. doi: 10.1016/j.promfg.2017.07.134 URL
[2]	王伟男, 杨朝红. 基于图像处理技术的目标识别方法综述[J]. 电脑与信息技术, 2019, 27(6): 9-15.
	WANG Weinan, YANG Chaohong. A survey of target recognition methods based on image processing technology[J]. Computer and Information Technology, 2019, 27(6): 9-15.
[3]	郝雯, 王映辉, 宁小娟, 等. 面向点云的三维物体识别方法综述[J]. 计算机科学, 2017, 44(9): 11-16. doi: 10.11896/j.issn.1002-137X.2017.09.002
	HAO Wen, WANG Yinghui, NING Xiaojuan, et al. Survey of 3D object recognition for point clouds[J]. Computer Science, 2017, 44(9): 11-16. doi: 10.11896/j.issn.1002-137X.2017.09.002
[4]	GARCÍA-PULIDO J A, PAJARES G, DORMIDO S, et al. Recognition of a landing platform for unmanned aerial vehicles by using computer vision-based techniques[J]. Expert Systems With Applications, 2017, 76: 152-165. doi: 10.1016/j.eswa.2017.01.017 URL
[5]	CHEN J M, CHEN L P. Multi-dimensional color image recognition and mining based on feature mining algorithm[J]. Automatic Control and Computer Sciences, 2021, 55(2): 195-201. doi: 10.3103/S0146411621020048
[6]	SEIDENARI L, SERRA G, BAGDANOV A D, et al. Local pyramidal descriptors for image recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 1033-1040. doi: 10.1109/TPAMI.2013.232 pmid: 26353235
[7]	CHEN G, PENG R, WANG Z C, et al. Pallet recognition and localization method for vision guided forklift[C]// 2012 8th International Conference on Wireless Communications, Networking and Mobile Computing. Shanghai, China: IEEE, 2012: 1-4.
[8]	SYU J L, LI H T, CHIANG J S, et al. A computer vision assisted system for autonomous forklift vehicles in real factory environment[J]. Multimedia Tools and Applications, 2017, 76(18): 18387-18407. doi: 10.1007/s11042-016-4123-6 URL
[9]	LI T J, HUANG B, LI C, et al. Application of convolution neural network object detection algorithm in logistics warehouse[J]. The Journal of Engineering, 2019, 2019(23): 9053-9058. doi: 10.1049/tje2.v2019.23 URL
[10]	SHAO Y P, WANG K, DU S C, et al. High definition metrology enabled three dimensional discontinuous surface filtering by extended tetrolet transform[J]. Journal of Manufacturing Systems, 2018, 49: 75-92. doi: 10.1016/j.jmsy.2018.09.002 URL
[11]	SHAO Y P, DU S C, TANG H T. An extended bi-dimensional empirical wavelet transform based filtering approach for engineering surface separation using high definition metrology[J]. Measurement, 2021, 178: 109259.
[12]	武文汉, 杨明, 王冰, 等. 一种基于轮廓匹配的仓储机器人托盘检测方法[J]. 上海交通大学学报, 2019, 53(2): 197-202.
	WU Wenhan, YANG Ming, WANG Bing, et al. Pallet detection based on contour matching for warehouse robots[J]. Journal of Shanghai Jiao Tong University, 2019, 53(2): 197-202.
[13]	XIAO J H, LU H M, ZHANG L L, et al. Pallet recognition and localization using an RGB-D camera[J]. International Journal of Advanced Robotic Systems, 2017, 14(6): 172988141773779.
[14]	VARGA R, COSTEA A, NEDEVSCHI S. Improved autonomous load handling with stereo cameras[C]// 2015 IEEE International Conference on Intelligent Computer Communication and Processing. Cluj-Napoca, Romania: IEEE, 2015: 251-256.
[15]	VARGA R, NEDEVSCHI S. Robust pallet detection for automated logistics operations[C]// Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. Rome, Italy: SCITEPRESS-Science and Technology Publications, 2016: 470-477.
[16]	吴登禄, 曹文希, 朱颖. 基于三维点云和图像边缘的托盘检测技术研究[J]. 自动化与信息工程, 2019, 40(3): 40-42.
	WU Denglu, CAO Wenxi, ZHU Ying. Research on pallet detection technology based on 3D point cloud and image edge features[J]. Automation & Information Engineering, 2019, 40(3): 40-42.
[17]	李洋洋, 史历程, 万卫兵, 等. 基于卷积神经网络的三维物体检测方法[J]. 上海交通大学学报, 2018, 52(1): 7-12.
	LI Yangyang, SHI Licheng, WAN Weibing, et al. A convolutional neural network-based method for 3D object detection[J]. Journal of Shanghai Jiao Tong University, 2018, 52(1): 7-12.
[18]	TERABAYASHI K, TAKASHIMA I, SUZUKI Y, et al. Easy acquisition of range image dataset for object detection using retroreflective markers and a time-of-flight camera: An application to detection of forklift pallets[C]// Proceedings of the Seventh Asia International Symposium on Mechatronics. Hangzhou, China: Springer Singapore, 2020: 1001-1005.
[19]	郭裕兰, 鲁敏, 谭志国, 等. 距离图像局部特征提取方法综述[J]. 模式识别与人工智能, 2012, 25(5): 783-791.
	GUO Yulan, LU Min, TAN Zhiguo, et al. Survey of local feature extraction on range images[J]. Pattern Recognition and Artificial Intelligence, 2012, 25(5): 783-791.
[20]	PRAKHYA S M, LIN J, CHANDRASEKHAR V, et al. 3DHoPD: A fast low-dimensional 3-D descriptor[J]. IEEE Robotics and Automation Letters, 2017, 2(3): 1472-1479. doi: 10.1109/LRA.2017.2667721 URL
[21]	RUSU R B, BLODOW N, BEETZ M. Fast point feature histograms (FPFH) for 3D registration[C]// 2009 IEEE International Conference on Robotics and Automation. Kobe, Japan: IEEE, 2009: 3212-3217.
[22]	GUO Y L, BENNAMOUN M, SOHEL F, et al. A comprehensive performance evaluation of 3D local feature descriptors[J]. International Journal of Computer Vision, 2016, 116(1): 66-89. doi: 10.1007/s11263-015-0824-y URL
[23]	HUANG J, YOU S Y. Detecting objects in scene point cloud: A combinational approach[C]// 2013 International Conference on 3D Vision-3DV 2013. Seattle, WA, USA: IEEE, 2013: 175-182.
[24]	王斐, 梁宸, 韩晓光, 等. 基于焊件识别与位姿估计的焊接机器人视觉引导[J]. 控制与决策, 2020, 35(8): 1873-1878.
	WANG Fei, LIANG Chen, HAN Xiaoguang, et al. Visual guidance of welding robot based on weldment recognition and pose estimation[J]. Control and Decision, 2020, 35(8): 1873-1878.
[25]	LIU J, BAI D, CHEN L. 3-D point cloud registration algorithm based on greedy projection triangulation[J]. Applied Sciences, 2018, 8(10): 1776. doi: 10.3390/app8101776 URL
[26]	LI P, WANG J, ZHAO Y D, et al. Improved algorithm for point cloud registration based on fast point feature histograms[J]. Journal of Applied Remote Sensing, 2016, 10: 045024.
[27]	NAPOLI A, GLASS S, WARD C, et al. Performance analysis of a generalized motion capture system using microsoft kinect 2.0[J]. Biomedical Signal Processing and Control, 2017, 38: 265-280. doi: 10.1016/j.bspc.2017.06.006 URL
[28]	DEMANTKE J, MALLET C, DAVID N, et al. Dimensionality based scale selection in 3D LIDAR point clouds[C]// ISPRS Workshop Laser Scanning. Calgary, Canada: Copernicus Gesellschaft Mbh, 2011: 97-102.
[29]	WEINMANN M, JUTZI B, MALLET C. Semantic 3D scene interpretation: A framework combining optimal neighborhood size selection with relevant features[J]. Photogrammetry, Remote Sensing and Spatial Information Sciences, 2014, 2(3): 181-188.
[30]	王红雨, 尹午荣, 汪梁, 等. 基于HSV颜色空间的快速边缘提取算法[J]. 上海交通大学学报, 2019, 53(7): 765-772.
	WANG Hongyu, YIN Wurong, WANG Liang, et al. Fast edge extraction algorithm based on HSV color space[J]. Journal of Shanghai Jiao Tong University, 2019, 53(7): 765-772
[31]	熊风光, 蔡晋茹, 况立群, 等. 三维点云模型中特征点描述子及其匹配算法研究[J]. 小型微型计算机系统, 2017, 38(3): 640-644.
	XIONG Fengguang, CAI Jinru, KUANG Liqun, et al. Study on descriptor and matching algorithm of feature point in 3D point cloud[J]. Journal of Chinese Computer Systems, 2017, 38(3): 640-644.
[32]	唐敏杰, 赵欢, 丁汉. 二进制点云局部特征描述子研究[J]. 机械工程学报, 2021, 57(2): 219-229. doi: 10.3901/JME.2021.02.219
	TANG Minjie, ZHAO Huan, DING Han. Research on binarized local feature descriptors of point clouds[J]. Journal of Mechanical Engineering, 2021, 57(2): 219-229. doi: 10.3901/JME.2021.02.219

特征值关系式	点云局部邻域维度特性
λ₁?λ₂, λ₂≈λ₃	一维线状特征
λ₁≈λ₂, λ₂?λ₃	二维面状特征
λ₁≈λ₂, λ₂≈λ₃	三维散乱点特征

特征描述符		正确匹配点数量	正确匹配点对占初始匹配点对比率/%	正确匹配点间的平均距离间隔/m
名称	r/m	正确匹配点数量	正确匹配点对占初始匹配点对比率/%	正确匹配点间的平均距离间隔/m
FPFH	0.015	183	8.31	0.054
CFPFH	0.015	332	9.87	0.039
ACFPFH	自适应	759	14.35	0.025

名称	特征描述符特征维度	r/m	d_th = 0.75		ACFPFH与P_FD 对比/%
名称	特征描述符特征维度	r/m	R_c	P_c	ACFPFH与P_FD 对比/%
SHOT	352	0.012	0.009 7	0.079 4	44.67
		0.014	0.012 5	0.110 8	22.79
FPFH	33	0.012	0.012 2	0.078 1	83.74
		0.014	0.009 6	0.093 6	53.31
CFPFH	36	0.012	0.010 2	0.097 5	47.17
		0.014	0.010 6	0.120 8	18.79
ACFPFH	36	自适应最优	0.013 8	0.143 5

名称	特征描述符特征维度	r/m	场景点云特征提取用时/s	ACFPFH与t_FD 对比/%
SHOT	352	0.012	142.256	28.26
		0.014	407.324	74.95
FPFH	33	0.012	158.339	35.55
		0.014	383.760	73.41
CFPFH	36	0.012	180.602	43.49
		0.014	416.313	75.49
ACFPFH	36	自适应最优	102.053