基于自适应颜色快速点特征直方图的托盘识别方法

doi:10.16183/j.cnki.jsjtu.2021.301

上海交通大学学报 ›› 2023, Vol. 57 ›› Issue (3): 297-308.doi: 10.16183/j.cnki.jsjtu.2021.301

所属专题：《上海交通大学学报》2023年“机械与动力工程”专题

基于自适应颜色快速点特征直方图的托盘识别方法

詹燕¹, 陈志慧¹, 朱宝昌²^,³, 朱婷婷², 邵益平¹^,²(), 鲁建厦¹

1.浙江工业大学机械工程学院, 杭州 310023
2.诺力智能装备股份有限公司, 浙江湖州 313199
3.浙江省智能物流装备工程技术研究中心, 浙江湖州 313199

收稿日期:2021-08-12 接受日期:2021-10-08 出版日期:2023-03-28 发布日期:2023-03-30
通讯作者: 邵益平,讲师;E-mail:syp123 gh@zjut.edu.cn.
作者简介:詹燕(1976-),博士,副教授,研究方向为智能物流、系统优化.
基金资助:
浙江省自然科学基金项目(LQ22E050017);中国博士后科学基金项目(2021M702894);浙江省博士后科研择优资助项目(ZJ2021119);浙江省重点研发计划资助项目(2018C01003)

A Pallet Recognition Method Based on Adaptive Color Fast Point Feature Histogram

ZHAN Yan¹, CHEN Zhihui¹, ZHU Baochang²^,³, ZHU Tingting², SHAO Yiping¹^,²(), LU Jiansha¹

1. College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
2. Noblelift Intelligent Equipment Co., Ltd., Huzhou 313199, Zhejiang, China
3. Engineering Research Center of Intelligent Logistics Equipment of Zhejiang Province, Huzhou 313199, Zhejiang, China

Received:2021-08-12 Accepted:2021-10-08 Online:2023-03-28 Published:2023-03-30

摘要/Abstract

摘要：

托盘识别是无人驾驶工业车辆进行货物搬运的关键技术之一.针对现有托盘识别方法低效耗时、鲁棒性差、参数选择随意的缺点,提出了一种基于自适应颜色快速点特征直方图的托盘识别方法.该方法使用Kinect V2传感器采集包含托盘的场景点云数据,点云经离群点剔除后,基于邻域特征熵函数最小准则获取每个点的最优邻域半径.提取点云关键点,计算关键点的颜色特征和自适应邻域快速点特征直方图,融合成自适应颜色快速点特征直方图,进行特征匹配与误匹配点对剔除,从而实现托盘识别.与固定邻域半径为0.012 m的快速点特征直方图对比,实验结果表明:基于自适应颜色快速点特征直方图的托盘识别精度提高了83.74%,特征提取用时减少了35.55%,验证了方法的优越性.

关键词: 快速点特征直方图, 特征提取, 自适应最优邻域, 托盘识别

Abstract:

Pallet recognition is one of the critical technologies of cargo handling for unmanned industrial vehicles. A pallet recognition method based on adaptive color fast point feature histogram (ACFPFH) is proposed to solve the problems of current recognition methods such as low efficiency, time-consuming, poor robustness and random parameter selection. The Kinect V2 sensor is used to collect the point cloud data which represents the whole scene including pallet. Next, outliers are removed and the optimal neighborhood radius of each point is obtained based on the minimum criterion of neighborhood feature entropy function. Then, the key points are extracted from scene point clouds. The ACFPFH consisting of color feature and adaptive geometric feature is applied for similarity matching between the template and scene point clouds. Finally, wrong feature correspondences are rejected and the pallet in the scene point cloud is recognized. A comparison of the fast point feature histogram with the fixed neighbor radius of 0.012 m shows that the pallet recognition precision and efficiency of the method based on ACFPFH is improved by 83.74% and 35.55% respectively, which verifies the superiority of the proposed method.

Key words: fast point feature histogram, feature extraction, adaptive optimal neighborhood, pallet recognition

中图分类号:

TH166

詹燕, 陈志慧, 朱宝昌, 朱婷婷, 邵益平, 鲁建厦. 基于自适应颜色快速点特征直方图的托盘识别方法[J]. 上海交通大学学报, 2023, 57(3): 297-308.

ZHAN Yan, CHEN Zhihui, ZHU Baochang, ZHU Tingting, SHAO Yiping, LU Jiansha. A Pallet Recognition Method Based on Adaptive Color Fast Point Feature Histogram[J]. Journal of Shanghai Jiao Tong University, 2023, 57(3): 297-308.

图/表 23

图1

图2

图3

表1

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

图14

图15

图16

图17

图18

表2

图19

表3

表4

参考文献 32

[1]	CASADO F, LAPIDO Y L, LOSADA D P, et al. Pose estimation and object tracking using 2D images[J]. Procedia Manufacturing, 2017, 11: 63-71. doi: 10.1016/j.promfg.2017.07.134 URL
[2]	王伟男, 杨朝红. 基于图像处理技术的目标识别方法综述[J]. 电脑与信息技术, 2019, 27(6): 9-15.
	WANG Weinan, YANG Chaohong. A survey of target recognition methods based on image processing technology[J]. Computer and Information Technology, 2019, 27(6): 9-15.
[3]	郝雯, 王映辉, 宁小娟, 等. 面向点云的三维物体识别方法综述[J]. 计算机科学, 2017, 44(9): 11-16. doi: 10.11896/j.issn.1002-137X.2017.09.002
	HAO Wen, WANG Yinghui, NING Xiaojuan, et al. Survey of 3D object recognition for point clouds[J]. Computer Science, 2017, 44(9): 11-16. doi: 10.11896/j.issn.1002-137X.2017.09.002
[4]	GARCÍA-PULIDO J A, PAJARES G, DORMIDO S, et al. Recognition of a landing platform for unmanned aerial vehicles by using computer vision-based techniques[J]. Expert Systems With Applications, 2017, 76: 152-165. doi: 10.1016/j.eswa.2017.01.017 URL
[5]	CHEN J M, CHEN L P. Multi-dimensional color image recognition and mining based on feature mining algorithm[J]. Automatic Control and Computer Sciences, 2021, 55(2): 195-201. doi: 10.3103/S0146411621020048
[6]	SEIDENARI L, SERRA G, BAGDANOV A D, et al. Local pyramidal descriptors for image recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 1033-1040. doi: 10.1109/TPAMI.2013.232 pmid: 26353235
[7]	CHEN G, PENG R, WANG Z C, et al. Pallet recognition and localization method for vision guided forklift[C]// 2012 8th International Conference on Wireless Communications, Networking and Mobile Computing. Shanghai, China: IEEE, 2012: 1-4.
[8]	SYU J L, LI H T, CHIANG J S, et al. A computer vision assisted system for autonomous forklift vehicles in real factory environment[J]. Multimedia Tools and Applications, 2017, 76(18): 18387-18407. doi: 10.1007/s11042-016-4123-6 URL
[9]	LI T J, HUANG B, LI C, et al. Application of convolution neural network object detection algorithm in logistics warehouse[J]. The Journal of Engineering, 2019, 2019(23): 9053-9058. doi: 10.1049/tje2.v2019.23 URL
[10]	SHAO Y P, WANG K, DU S C, et al. High definition metrology enabled three dimensional discontinuous surface filtering by extended tetrolet transform[J]. Journal of Manufacturing Systems, 2018, 49: 75-92. doi: 10.1016/j.jmsy.2018.09.002 URL
[11]	SHAO Y P, DU S C, TANG H T. An extended bi-dimensional empirical wavelet transform based filtering approach for engineering surface separation using high definition metrology[J]. Measurement, 2021, 178: 109259.
[12]	武文汉, 杨明, 王冰, 等. 一种基于轮廓匹配的仓储机器人托盘检测方法[J]. 上海交通大学学报, 2019, 53(2): 197-202.
	WU Wenhan, YANG Ming, WANG Bing, et al. Pallet detection based on contour matching for warehouse robots[J]. Journal of Shanghai Jiao Tong University, 2019, 53(2): 197-202.
[13]	XIAO J H, LU H M, ZHANG L L, et al. Pallet recognition and localization using an RGB-D camera[J]. International Journal of Advanced Robotic Systems, 2017, 14(6): 172988141773779.
[14]	VARGA R, COSTEA A, NEDEVSCHI S. Improved autonomous load handling with stereo cameras[C]// 2015 IEEE International Conference on Intelligent Computer Communication and Processing. Cluj-Napoca, Romania: IEEE, 2015: 251-256.
[15]	VARGA R, NEDEVSCHI S. Robust pallet detection for automated logistics operations[C]// Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. Rome, Italy: SCITEPRESS-Science and Technology Publications, 2016: 470-477.
[16]	吴登禄, 曹文希, 朱颖. 基于三维点云和图像边缘的托盘检测技术研究[J]. 自动化与信息工程, 2019, 40(3): 40-42.
	WU Denglu, CAO Wenxi, ZHU Ying. Research on pallet detection technology based on 3D point cloud and image edge features[J]. Automation & Information Engineering, 2019, 40(3): 40-42.
[17]	李洋洋, 史历程, 万卫兵, 等. 基于卷积神经网络的三维物体检测方法[J]. 上海交通大学学报, 2018, 52(1): 7-12.
	LI Yangyang, SHI Licheng, WAN Weibing, et al. A convolutional neural network-based method for 3D object detection[J]. Journal of Shanghai Jiao Tong University, 2018, 52(1): 7-12.
[18]	TERABAYASHI K, TAKASHIMA I, SUZUKI Y, et al. Easy acquisition of range image dataset for object detection using retroreflective markers and a time-of-flight camera: An application to detection of forklift pallets[C]// Proceedings of the Seventh Asia International Symposium on Mechatronics. Hangzhou, China: Springer Singapore, 2020: 1001-1005.
[19]	郭裕兰, 鲁敏, 谭志国, 等. 距离图像局部特征提取方法综述[J]. 模式识别与人工智能, 2012, 25(5): 783-791.
	GUO Yulan, LU Min, TAN Zhiguo, et al. Survey of local feature extraction on range images[J]. Pattern Recognition and Artificial Intelligence, 2012, 25(5): 783-791.
[20]	PRAKHYA S M, LIN J, CHANDRASEKHAR V, et al. 3DHoPD: A fast low-dimensional 3-D descriptor[J]. IEEE Robotics and Automation Letters, 2017, 2(3): 1472-1479. doi: 10.1109/LRA.2017.2667721 URL
[21]	RUSU R B, BLODOW N, BEETZ M. Fast point feature histograms (FPFH) for 3D registration[C]// 2009 IEEE International Conference on Robotics and Automation. Kobe, Japan: IEEE, 2009: 3212-3217.
[22]	GUO Y L, BENNAMOUN M, SOHEL F, et al. A comprehensive performance evaluation of 3D local feature descriptors[J]. International Journal of Computer Vision, 2016, 116(1): 66-89. doi: 10.1007/s11263-015-0824-y URL
[23]	HUANG J, YOU S Y. Detecting objects in scene point cloud: A combinational approach[C]// 2013 International Conference on 3D Vision-3DV 2013. Seattle, WA, USA: IEEE, 2013: 175-182.
[24]	王斐, 梁宸, 韩晓光, 等. 基于焊件识别与位姿估计的焊接机器人视觉引导[J]. 控制与决策, 2020, 35(8): 1873-1878.
	WANG Fei, LIANG Chen, HAN Xiaoguang, et al. Visual guidance of welding robot based on weldment recognition and pose estimation[J]. Control and Decision, 2020, 35(8): 1873-1878.
[25]	LIU J, BAI D, CHEN L. 3-D point cloud registration algorithm based on greedy projection triangulation[J]. Applied Sciences, 2018, 8(10): 1776. doi: 10.3390/app8101776 URL
[26]	LI P, WANG J, ZHAO Y D, et al. Improved algorithm for point cloud registration based on fast point feature histograms[J]. Journal of Applied Remote Sensing, 2016, 10: 045024.
[27]	NAPOLI A, GLASS S, WARD C, et al. Performance analysis of a generalized motion capture system using microsoft kinect 2.0[J]. Biomedical Signal Processing and Control, 2017, 38: 265-280. doi: 10.1016/j.bspc.2017.06.006 URL
[28]	DEMANTKE J, MALLET C, DAVID N, et al. Dimensionality based scale selection in 3D LIDAR point clouds[C]// ISPRS Workshop Laser Scanning. Calgary, Canada: Copernicus Gesellschaft Mbh, 2011: 97-102.
[29]	WEINMANN M, JUTZI B, MALLET C. Semantic 3D scene interpretation: A framework combining optimal neighborhood size selection with relevant features[J]. Photogrammetry, Remote Sensing and Spatial Information Sciences, 2014, 2(3): 181-188.
[30]	王红雨, 尹午荣, 汪梁, 等. 基于HSV颜色空间的快速边缘提取算法[J]. 上海交通大学学报, 2019, 53(7): 765-772.
	WANG Hongyu, YIN Wurong, WANG Liang, et al. Fast edge extraction algorithm based on HSV color space[J]. Journal of Shanghai Jiao Tong University, 2019, 53(7): 765-772
[31]	熊风光, 蔡晋茹, 况立群, 等. 三维点云模型中特征点描述子及其匹配算法研究[J]. 小型微型计算机系统, 2017, 38(3): 640-644.
	XIONG Fengguang, CAI Jinru, KUANG Liqun, et al. Study on descriptor and matching algorithm of feature point in 3D point cloud[J]. Journal of Chinese Computer Systems, 2017, 38(3): 640-644.
[32]	唐敏杰, 赵欢, 丁汉. 二进制点云局部特征描述子研究[J]. 机械工程学报, 2021, 57(2): 219-229. doi: 10.3901/JME.2021.02.219
	TANG Minjie, ZHAO Huan, DING Han. Research on binarized local feature descriptors of point clouds[J]. Journal of Mechanical Engineering, 2021, 57(2): 219-229. doi: 10.3901/JME.2021.02.219

特征值关系式	点云局部邻域维度特性
λ₁?λ₂, λ₂≈λ₃	一维线状特征
λ₁≈λ₂, λ₂?λ₃	二维面状特征
λ₁≈λ₂, λ₂≈λ₃	三维散乱点特征

特征描述符		正确匹配点数量	正确匹配点对占初始匹配点对比率/%	正确匹配点间的平均距离间隔/m
名称	r/m	正确匹配点数量	正确匹配点对占初始匹配点对比率/%	正确匹配点间的平均距离间隔/m
FPFH	0.015	183	8.31	0.054
CFPFH	0.015	332	9.87	0.039
ACFPFH	自适应	759	14.35	0.025

名称	特征描述符特征维度	r/m	d_th = 0.75		ACFPFH与P_FD 对比/%
名称	特征描述符特征维度	r/m	R_c	P_c	ACFPFH与P_FD 对比/%
SHOT	352	0.012	0.009 7	0.079 4	44.67
		0.014	0.012 5	0.110 8	22.79
FPFH	33	0.012	0.012 2	0.078 1	83.74
		0.014	0.009 6	0.093 6	53.31
CFPFH	36	0.012	0.010 2	0.097 5	47.17
		0.014	0.010 6	0.120 8	18.79
ACFPFH	36	自适应最优	0.013 8	0.143 5

名称	特征描述符特征维度	r/m	场景点云特征提取用时/s	ACFPFH与t_FD 对比/%
SHOT	352	0.012	142.256	28.26
		0.014	407.324	74.95
FPFH	33	0.012	158.339	35.55
		0.014	383.760	73.41
CFPFH	36	0.012	180.602	43.49
		0.014	416.313	75.49
ACFPFH	36	自适应最优	102.053

基于自适应颜色快速点特征直方图的托盘识别方法

A Pallet Recognition Method Based on Adaptive Color Fast Point Feature Histogram

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 23

参考文献 32

相关文章 13

编辑推荐

Metrics

本文评价

[1]	曾志贤，曹建军，翁年凤，袁震，余旭. 基于细粒度联合注意力机制的图像-文本跨模态实体分辨[J]. J Shanghai Jiaotong Univ Sci, 2023, 28(6): 728-737.
[2]	张晏合, 臧月进, 陈渤, 徐铭晟. 基于解耦表征变分自编码机的雷达目标识别算法[J]. 空天防御, 2022, 5(2): 87-93.
[3]	王微, 王冰, 胡雄, 孙德建. 基于在线改进符号序列熵与逻辑回归模型的岸桥起升减速箱在线退化评估[J]. 上海交通大学学报, 2021, 55(10): 1272-1280.
[4]	王俊, 王赛, 任俞明, 陈德红, 崔闪, 魏少明. 结合深度学习去噪和超分辨的SAR检测识别[J]. 空天防御, 2020, 3(3): 24-30.
[5]	颜波，张磊，褚学宁. 基于卷积神经网络的用户感知评估建模[J]. 上海交通大学学报, 2019, 53(7): 844-851.
[6]	商诺诺, 梁艳, 翟恒峰, 原浩娟. 一种高效的雷达信号全脉冲特征提取方法[J]. 空天防御, 2018, 1(4): 37-43.
[7]	吴倩红，韩蓓，冯琳，李国杰，江秀臣. “人工智能+”时代下的智能电网预测分析[J]. 上海交通大学学报（自然版）, 2018, 52(10): 1206-1219.
[8]	王瑞，刘宾，周天润，杨羽. 基于协同表示的声振传感器网络车辆分类识别[J]. 上海交通大学学报（自然版）, 2018, 52(1): 103-110.
[9]	王嗣阳，许黎明，赖小平. 砂轮磨削过载判据及其快速诊断[J]. 上海交通大学学报（自然版）, 2015, 49(09): 1346-1352.
[10]	王佳婧1，2，张树生1，何卫平1. 基于多信息融合的失效DPM码识别[J]. 上海交通大学学报（自然版）, 2014, 48(12): 1675-1680.
[11]	成谢锋,傅女婷. 心音身份识别综述[J]. 上海交通大学学报（自然版）, 2014, 48(12): 1745-1750.
[12]	余焕伟，叶震，张志芬，陈华斌，陈善本. Al-Mg合金钨极气体保护焊接电弧的光谱特征提取方法[J]. 上海交通大学学报（自然版）, 2013, 47(11): 1655-1660.
[13]	郝圣桥，许黎明，沈伟，王建楼. 电液伺服阀状态在线特征提取和异常检测方法 [J]. 上海交通大学学报（自然版）, 2010, 44(12): 1747-1752.