基于自适应颜色快速点特征直方图的托盘识别方法

图1 目标识别算法流程图

Fig.1 Flow chart of object recognition algorithm

(1) 点云预处理.通过Kinect V2传感器采集托盘模板点云与场景点云,利用统计滤波剔除场景点云中的离群点,采用随机采样一致性(RANSAC)算法对场景点云中的地面及墙面进行分割.

(2) 关键点检测.利用内部形状签名(ISS)算法对托盘模板点云及平面分割后的场景点云提取关键点,计算点云邻域协方差矩阵及对应特征值,将特征值满足特定关系的点定义为关键点.

(3) ACFPFH特征提取.引入邻域特征熵函数,确定自适应最优邻域半径,计算托盘模板点云与场景点云关键点的ACFPFH特征描述符.

(4) 特征匹配与误匹配点对剔除.将模板点云与场景点云中的ACFPFH特征描述符进行匹配,建立模板点云与场景点云关键点之间的对应关系,得到初始匹配点对;基于RANSAC算法,剔除不满足变换关系的匹配点对,保留正确匹配点对,完成托盘识别.

2 点云处理与关键点检测

2.1 点云离群点剔除

由于Kinect V2传感器的硬件设计、外界环境干扰等因素的影响,采集到的原始点云分布不均匀,会导致托盘识别精度大大降低,所以需要剔除原始场景点云Q_SO中的离群点,如图2所示.点云中任意点 $P_{i} 到其邻域点 P_{i k} (k = 1, 2, \dots, m)$ 的距离近似服从高斯分布,邻域平均距离的概率密度函数可以表示为

$\begin{matrix} f (d_{i}) = \frac{1}{\sqrt{2 π} σ} e x p (- \frac{(d_{i} {- μ)}^{2}}{2 σ^{2}}) \end{matrix}$

(1)

式中: $d_{i}$ 为任意点 $P_{i}$ 的邻域平均距离;μ和σ分别为 $d_{i}$ 的均值和标准差.本文计算点 $P_{i}$ 到其m个邻域点 $P_{i k}$ 的平均距离 $d_{i}$ ,若 $d_{i}$ 超出距离范围μ±σ,则认为点 $P_{i}$ 为离群点并剔除.

图2

图2 原始场景点云及离群点剔除示意图

Fig.2 Point cloud of original scene and schematic diagram of outlier elimination

2.2 点云平面分割

在仓储环境中,Kinect V2传感器采集到的场景点云包含大量地面及墙面上的冗余信息,会降低计算效率.因此,需要进行平面分割剔除场景点云中的墙面及地面,如图3所示.具体分割过程如下:①在点云中随机选取3点,构建初始平面模型 $A x + B y + C z + D = 0 (A, B, C, D 为常数);$ ②计算点P_i到初始平面的距离(D_i)以及P_i点坐标与初始平面法向量之间的角度( $β_{i}$ ),若距离 $D_{i}$ 小于距离阈值( $D_{ε}$ )且角度 $β_{i}$ 小于角度阈值( $β_{ε}$ ),则认为点 $P_{i}$ 为平面内点;③不断进行迭代直到平面内点数量达到数量阈值t,得到最终的拟合平面模型进行剔除.

图3

图3 场景点云平面分割示意图

Fig.3 Schematic diagram of plane segmentation of scene point cloud

2.3 点云关键点检测

托盘模板点云(Q_M)和预处理后的场景点云(Q_S)数据量大,会降低特征提取及匹配效率,因此需要提取点云关键点,即保留特征明显的点,减少点云数量.内部形状描述子(ISS)算法具有较好可重复性,效率较高,因此本文采用ISS算法进行关键点检测,具体步骤如下.

(1) 对点云Q_M、Q_S中的任意一点 $P_{i} 进行半径搜索, 得到邻域点 P_{i k} (k = 1, 2, \dots, m), 计算邻域点到 P_{i}$ 的距离并取倒数作为权重

$\begin{matrix} ω_{i k} = \frac{1}{‖ P_{i k L} - P_{i L} ‖} \end{matrix}$

(2)

(2) 计算每个点 $P_{i}$ 的邻域协方差矩阵

$\begin{matrix} C_{i} = \frac{\overset{m}{\sum_{k = 1}} ω_{i k} (P_{i k L} - {\bar{P}}_{i L}) (P_{i k L} - {\bar{P}}_{i L})^{T}}{\overset{m}{\sum_{k = 1}} ω_{i k}} \end{matrix}$

(3)

式中: $P_{i L}$ 为 $P_{i}$ 的矢量坐标; $P_{i k L}$ 为 $P_{i k}$ 的矢量坐标; ${\bar{P}}_{i L}$ 为 $P_{i}$ 邻域点的重心,且 ${\bar{P}}_{i L} = \frac{1}{m} \overset{m}{\sum_{k = 1}} P_{i k L} .$

(3) 对协方差矩阵 $C_{i}$ 进行特征值分解,得到3个特征值 ${λ_{1}^{i}, λ_{2}^{i}, λ_{3}^{i}}$ ,并对特征值进行降序排列.

(4) 设置阈值κ₁和κ₂,将满足下式的点定义为关键点:

$\begin{matrix} \frac{λ_{2}}{λ_{1}} < κ_{1} ⋂ \frac{λ_{3}}{λ_{2}} < κ_{2} \end{matrix}$

(4)

式中:κ₁和κ₂为0~1之间的常数,最后得到托盘模板点云关键点集Q_MK和场景点云关键点集Q_SK.

3 ACFPFH特征提取及匹配

3.1 ACFPFH特征提取算法

3.1.1 自适应最优邻域估计

在对关键点进行特征提取之前,首先需要计算特征提取的自适应最优邻域半径(r_opt),具体步骤如下:①设置邻域搜索的半径范围[r_min, r_max]以及变化间隔Δr;②计算不同邻域半径对应的协方差矩阵C_i及特征值λ₁,λ₂,λ₃;③计算不同邻域半径对应的邻域特征熵函数(E_e);④基于邻域特征熵函数E_e最小准则计算得到最优邻域半径r_opt.

根据特征值 $λ_{1}, λ_{2}, λ_{3},$ 可以判断出点云局部邻域的维度特性^[27],如表1所示.

表1 点云局部邻域的维度特性判断表

Tab.1 Dimensional characteristics judgment of local neighborhood of point cloud

特征值关系式	点云局部邻域维度特性
λ₁≫λ₂, λ₂≈λ₃	一维线状特征
λ₁≈λ₂, λ₂≫λ₃	二维面状特征
λ₁≈λ₂, λ₂≈λ₃	三维散乱点特征

构建点 $P_{i}$ 局部邻域的一维线性维度特征、二维平面维度特征和三维曲面维度特征如下:

$\begin{matrix} \begin{array}{l} L_{λ} = \frac{λ_{1} - λ_{2}}{λ_{1}} \\ P_{λ} = \frac{λ_{2} - λ_{3}}{λ_{1}} \\ S_{λ} = λ_{3} / λ_{1} \end{array}\} \end{matrix}$

(5)

式中: $L_{λ} + P_{λ} + S_{λ} = 1$ ,因此可将 $L_{λ}$ , $P_{λ}$ , $S_{λ}$ 分别视为点 $P_{i}$ 属于3个维度特征的概率.根据信息熵理论建立局部邻域熵函数^[28],即

$\begin{matrix} E_{n} = - L_{λ} l n (L_{λ}) - P_{λ} l n (P_{λ}) - S_{λ} l n (S_{λ}) \end{matrix}$

(6)

根据变量不确定性越小、信息熵越小的香农熵理论^[29],可以得出,局部邻域信息熵值E_n越小,点P_i维度特征的不确定性越小,即P_i属于某种维度特征的概率越大,该邻域半径下的局部数据点的空间分布特性越相近,邻域半径越趋于最优.因此,可以根据邻域熵函数最小准则获取点云自适应最优邻域半径:

$\begin{matrix} r_{e - o p t} = a r g m i n (E_{n}) \end{matrix}$

(7)

式中:r_e-opt是E_n取最小值的变量值.然而,依据式(6)和式(7)得到的最优邻域半径r_e-opt是基于点云某种维度特征明显的假设,对于点P_i属于某个特征维度的概率略大于其他两个维度的情况,此时估计出的邻域不一定最优.由表1可知,点云数据点邻域协方差矩阵的特征值直接反映了该点邻域范围内邻域点的维度分布特性,为了避免对点云进行假设,提高最优邻域的估计精度,本文直接根据特征值构建邻域特征熵函数,即

$\begin{matrix} E_{e} = - e_{1} l n (e_{1}) - e_{2} l n (e_{2}) - e_{3} l n (e_{3}) \end{matrix}$

(8)

式中: $e_{j} = \frac{λ_{j}}{\sum λ_{j}}$ , $λ_{j}$ 为邻域协方差矩阵分解得到的特征值.基于邻域特征熵函数 $E_{e}$ 最小准则计算得到最优邻域半径为

$\begin{matrix} r_{o p t} = a r g m i n (E_{e}) \end{matrix}$

(9)

因此,对于不同的邻域半径,当E_e取最小值时,对应的邻域半径为最优邻域半径.

3.1.2 ACFPFH特征描述符构建

首先是颜色特征.Kinect V2传感器采集得到的点云数据包含待测目标的坐标及颜色信息.三维点云的RGB颜色空间是一种不均匀的颜色空间,两种颜色之间的知觉差异(色差)不能表示为该颜色空间中两点间的距离,因此不适用于特征相似度的检测.HSV颜色空间是一种基于感知的颜色模型,相比RGB空间,具有更强的识别能力,更符合人类的视觉特征^[30].本文选用HSV颜色空间来进行特征提取,其中H表示点云的色调,即所处的光谱颜色的位置,S表示点云的饱和度,V表示点云色彩的明度.提取HSV颜色空间的3个颜色分量来表示点云中每个关键点P_F_i的颜色特征,RGB空间到HSV空间的转换关系如下式所示:

$\begin{array}{l} V = m a x {R, G, B} \\ S = \{\begin{array}{l} 0, & V = 0 \\ \frac{m a x {R, G, B} - m i n {R, G, B}}{m a x {R, G, B}}, & 其他 \end{array} \\ \begin{matrix} H = \{\begin{array}{l} 0, S = 0 \\ 60 (G - B) / (S V), \\ S \neq 0 且 V = R \\ 60 \times [2 + (B - R) / (S V)], \\ S \neq 0 且 V = G \\ 60 \times [4 + (R - G) / (S V)], \\ 其他 \end{array} \end{matrix} \end{array}$

式中:红色、绿色、蓝色分别对应的强度值R, G, B的取值范围均为[0255];H的取值范围为[0360];S的取值范围为[01];V的取值范围为[0255].

其次是几何特征.FPFH通过统计点云查询点与邻域点之间的法线关系形成直方图,从而描述点云的几何特征,计算步骤如下.

(1) 基于自适应最优邻域半径r_opt,寻找关键点P_F_i(即查询点P_q)的邻域点P_q_j.

(2) 基于关键点 $P_{F i} 与任一邻域点 P_{q j},$ 构建邻域点对p_s和p_t,并建立局部坐标系uvw,示意图如4所示,定义为

$\begin{matrix} \begin{array}{l} u = n_{s} \\ v = (p_{t} - p_{s}) \times u \\ w = u \times v \end{array}\} \end{matrix}$

(10)

式中:n_s为点p_s的法向量.

(3) 计算关键点 $P_{F i}$ 与每个邻域点 $P_{q j}$ 法向量间的相对偏差(α,φ,θ)(见图4),得到关键点 $P_{F i}$ 简化的点特征直方图(SPFH),表示为

$\begin{matrix} \begin{array}{l} α = v \cdot n_{t} \\ φ = u \cdot (p_{t} - p_{s}) / d \\ θ = a r c t a n (w \cdot n_{t}, u \cdot n_{t}) \end{array}\} \end{matrix}$

(11)

式中:n_t为点p_t的法向量.

图4

图4 局部坐标系uvw示意图

Fig.4 Schematic diagram of local coordinate system

(4) 基于r_opt重新确定每个邻域点 $P_{q j} 的最优邻域, 计算邻域点 P_{q j}$ 的SPFH值 $V_{S P F H (P_{q j})}$ ,最后加权得到关键点 $P_{F i}$ 的FPFH值 $V_{F P F H (P_{F i})}$ ,表示为

$V_{F P F H (P_{F i})} = V_{S P F H (P_{F i})} + \frac{1}{k} \overset{k}{\sum_{i = 1}} \frac{1}{ω} V_{S P F H (P_{q j})}$

(12)

式中: $V_{S P F H (P_{F i})}$ 为关键点 $P_{F i}$ 的SPFH值.

FPFH计算过程的邻域影响范围如图5所示.

图5

图5 FPFH邻域影响范围示意图

Fig.5 Neighborhood influence range of FPFH

(5) 将(α, φ, θ)每个维度分成11个区间,在每个维度上统计落在这11个区间上点的个数,将3个维度进行叠加,得到33维几何特征向量.

最后是ACFPFH特征描述符.将3维的HSV颜色特征与33维的FPFH几何特征如下式所示进行叠加,得到36维的ACFPFH特征描述符,具体如图6所示.

$\begin{matrix} V_{A C F P F H (P_{F i})} = V_{H S V (P_{F i})} + V_{F P F H (P_{F i})} \end{matrix}$

(13)

式中: $V_{A C F P F H (P_{F i})}$ 为关键点 $P_{F i}$ 的ACFPFH特征值; $V_{H S V (P_{F i})} 为关键点 P_{F i}$ 的HSV特征值.

图6

图6 $V_{A C F P F H (P_{F i})}$ 示意图

Fig.6 Schematic of $V_{A C F P F H (P_{F i})}$

3.2 特征匹配与误匹配点对剔除

3.2.1 特征匹配

对于托盘模板点云Q_M与场景点云Q_S中关键点的ACFPFH特征描述符集合 F_M={ ${f^{i}}_{M}$ }和F_S={ ${f^{i}}_{S}$ },如果f_M和f_S满足最近邻距离比的匹配规则,即

$\begin{matrix} \frac{‖ f_{S} - f_{M} ‖}{‖ f_{S} - f_{M}'‖} < d_{t h} \end{matrix}$

(14)

式中:f_S为F_S中的任意特征描述符;f_M和f'_M为F_M中与f_S最近及次近的特征描述符;d_th为最近与次近距离比的阈值,是0~1之间的常数.则其对应的点对属于匹配点对集合E={Q_MP, Q_SP},其中,Q_MP与Q_SP分别为托盘模板点云及场景点云中对应的特征匹配点.基于最近邻距离比的特征匹配,F_M中的一个特征描述符在F_S中最多只有一个与之对应的特征描述符.

3.2.2 误匹配点对剔除

由于匹配点对中存在错误匹配点对,会造成识别错误,所以利用RANSAC算法剔除误匹配点对,具体过程如下:从匹配点对集合E中随机选取3对对应的关键点点对,求解托盘模板点云及场景点云中对应的特征匹配点Q_MP与Q_SP的旋转矩阵(R)和平移矩阵(T);计算特征匹配点对之间的欧氏距离,即

$\begin{matrix} D (R, T) = ‖ L_{Q S P} - (R \times L_{Q M P} + T) ‖ \end{matrix}$

(15)

式中: L_QSP与L_QMP分别为Q_SP与Q_MP的三维坐标向量.

设定阈值 $ε, 如果 D (R, T) < ε,$ 则认为该匹配点对为正确匹配点对,统计正确匹配点对数量;设定迭代次数,不断重复以上步骤,最后将最大正确匹配点对数量对应的旋转矩阵和平移矩阵作为正确变换关系;基于正确变换关系与阈值,得到最终的正确匹配点对,并将场景点云的正确匹配点作为最终的识别结果,如图7所示.

图7

图7 特征匹配与误匹配点对剔除

Fig.7 Feature matching and elimination of mismatching point pairs

4 案例分析

4.1 实验过程

为验证基于ACFPFH的托盘识别方法的有效性,采用基于飞行时间(TOF)原理的Kinect V2传感器采集点云数据进行结果对比分析.Kinect V2是一款可同时获得彩色图像和深度图像的3D传感器,彩色图像分辨率为 1 920 像素×1 080 像素,深度图像分辨率为512像素×424像素.Kinect V2共有3个摄像头,从左至右依次为RGB彩色摄像头、红外摄像头和红外投影机,其中红外摄像头和红外投影机共同构成深度传感器,深度传感器采用飞行时间差原理进行测距.Kinect V2结构示意图如图8所示,其水平视场为70°,垂直视场为60°,有效测距范围为0.5~4.5 m.

图8

图8 Kinect V2结构示意图

Fig.8 Schematic diagram of structure of Kinect V2

将Kinect V2传感器安装在叉车货叉架的顶部(见图9),传感器随着货叉一起上下移动,为了能拍摄到地面或货架上的托盘,将Kinect V2传感器的摄像头向下微微倾斜10°.叉车的货叉长度一般为 1 150 mm,根据工厂实际操作情况,将货叉顶端与托盘前端面的距离设置为500 mm以便叉车进行位置调整,保证货叉与托盘前端面垂直且传感器中心与托盘中心在一条线上,托盘具体放置情况如图10所示.白天正常环境光照射条件下,将托盘放置在空旷的平整地面上,在PC端采集点云数据,采用平面分割算法分割并剔除地面及墙面点云,将剩余点云作为托盘模板点云.在托盘上放置纸箱,Kinect V2传感器保持同样的距离采集包含托盘的场景点云数据.

图9

图9 Kinect V2传感器安装位置示意图

Fig.9 Diagram of Kinect V2 sensor installation position

图10

图10 点云数据采集距离示意图(mm)

Fig.10 Schematic diagram of acquisition of point cloud data distance (mm)

采用Kinect V2传感器获取场景的彩色图像,如图11所示,采集到地托盘模板点云与场景点云如图12所示,托盘为蓝色川字塑料托盘,尺寸为 1 200 mm×1 000 mm×150 mm.对场景点云进行预处理,根据先验知识可得场景点云地面法向量为[0 1 0],墙面法向量为[0 0 1],设置距离阈值D_ε=0.02 m,角度阈值β_ε=5°,对预处理后的场景点云进行平面分割,结果如图13所示.对托盘模板点云以及平面分割后的场景点云进行半径为 0.008 5 m 的半径搜索,考虑到识别精度与效率的平衡,将关键点数控制在 10 000 左右,设置ISS阈值κ₁=0.7,κ₂=0.5,完成关键点检测,托盘模板点云点的数量从 14 554 减少至 7 479,场景点云点的数量从 17 948 减少至 9 665,结果如图14所示.其中,红色点为关键点.

图11

图11 原始场景的彩色图像

Fig.11 Color image of original scene

图12

图12 托盘模板点云及原始场景点云

Fig.12 Pallet template point cloud and original scene point cloud

图13

图13 场景点云预处理

Fig.13 Preprocessing of scene point cloud

图14

图14 托盘模板点云及场景点云关键点

Fig.14 Key points of pallet template point cloud and original scene point cloud

为了计算关键点的ACFPFH特征值,首先需要求解点云中每个点的自适应邻域半径,Kinect V2传感器采集到的点云数据两个采样点之间的间隔为7 mm,因此设置半径范围r_min=0.008 5 m, r_max=0.018 m, Δr=0.000 5 m,基于邻域特征熵函数最小准则,得到每个点的自适应最优邻域半径.托盘模板点云与场景点云自适应最优邻域半径分布情况如图15所示.图中:N为r_opt取不同值时对应的点数.由图可见,点的最优邻域半径集中于给定的最小邻域半径,有利于提高托盘识别效率.提取托盘模板点云与场景点云关键点的HSV颜色分量,基于自适应最优邻域半径计算几何特征,二者叠加得到关键点的ACFPFH特征值.设置最近邻距离比率阈值d_th=0.75,完成托盘模板点云与场景点云的特征匹配,得到初始匹配点对,匹配结果如图16所示,托盘模板点云与场景点云中的对应点对用绿色线段相连,托盘上的点与纸箱上的点之间的连线代表错误匹配点对,利用RANSAC算法进行误匹配点对剔除,设置迭代次数为 1 000 次.将场景点云中的正确匹配点作为托盘识别结果,如图17所示,红色点为场景点云的正确匹配点,代表从场景点云中识别出的托盘.

图15

图15 点云r_opt分布情况

Fig.15 Distribution of r_opt of point cloud

图16

图16 特征匹配结果

Fig.16 Result of feature matching

图17

图17 场景点云中的托盘识别结果

Fig.17 Pallet recognition result in scene point cloud

4.2 实验结果分析

图18为固定半径的FPFH、CFPFH的托盘识别结果(红色点),结合图17及表2可以看出,基于ACFPFH的托盘识别得到的正确匹配点数量更多,正确匹配点之间的平均距离间隔更小,点云分布更稠密,且正确匹配点对数量达到初始匹配点对数量的10%以上,能正确表示完整托盘^[31].表中:r为邻域半径.匹配点间的平均距离间隔可近似理解为所有k个邻域点间的平均距离的平均值,以k=5为例.

图18

图18 不同特征描述符在场景点云中的托盘识别结果

Fig.18 Pallet recognition results of different feature descriptors in scene point cloud

表2 不同特征描述符对应的匹配点数量及间隔对比

Tab.2 Comparison of number and interval of matching points of different feature descriptors

特征描述符		正确匹配点数量	正确匹配点对占初始匹配点对比率/%	正确匹配点间的平均距离间隔/m
名称	r/m	正确匹配点数量	正确匹配点对占初始匹配点对比率/%	正确匹配点间的平均距离间隔/m
FPFH	0.015	183	8.31	0.054
CFPFH	0.015	332	9.87	0.039
ACFPFH	自适应	759	14.35	0.025

针对本文提出的ACFPFH特征描述符,采用召回率精度曲线及特征提取时间验证其性能优劣,召回率(R_c)及精度(P_c)定义为

$\begin{matrix} \begin{array}{l} P_{c} = Q_{C M} / Q_{P F} \\ R_{c} = Q_{C M} / Q_{S M P} \end{array}\} \end{matrix}$

(16)

式中: Q_CM为正确匹配点对数量,即误匹配点对剔除后得到的匹配点对数量;Q_PF为模板点云关键点数量;Q_SMP为特征匹配之后得到的初始特征匹配点对数量.通过改变特征匹配阶段的阈值d_th可获得多个召回率及对应的精度,从而得到特征描述子的PR曲线.方向直方图签名(SHOT)特征描述符是目前最常用且性能较优的特征描述符^[32].因此,将ACFPFH特征描述符与固定半径的FPFH、CFPFH以及SHOT特征描述符做对比,选取的特征匹配阶段的阈值集合为d_th={0.2, 0.4, 0.6, 0.75, 0.85, 0.925, 0.95, 0.975, 1.0},得到不同特征描述符对应的托盘识别P-R曲线,如图19所示.取d_th=0.75对不同特征描述符的精度进行比较,如表3所示.表中:P_FD为其他特征描述符的精度.进一步分析场景点云的特征提取所需时间,比较特征描述符的性能,如表4所示.表中:t_FD为其他特征描述符所用的运行时间.

图19

图19 不同特征描述符的P-R曲线

Fig.19 Curves of P-R of different feature descriptors

表3 不同特征描述符精度对比

Tab.3 Precision comparison of different feature descriptors

名称	特征描述符特征维度	r/m	d_th = 0.75		ACFPFH与P_FD 对比/%
名称	特征描述符特征维度	r/m	R_c	P_c	ACFPFH与P_FD 对比/%
SHOT	352	0.012	0.009 7	0.079 4	44.67
		0.014	0.012 5	0.110 8	22.79
FPFH	33	0.012	0.012 2	0.078 1	83.74
		0.014	0.009 6	0.093 6	53.31
CFPFH	36	0.012	0.010 2	0.097 5	47.17
		0.014	0.010 6	0.120 8	18.79
ACFPFH	36	自适应最优	0.013 8	0.143 5

表4 不同特征描述符特征提取时间对比

Tab.4 Comparison of feature extraction time of different feature descriptors

名称	特征描述符特征维度	r/m	场景点云特征提取用时/s	ACFPFH与t_FD 对比/%
SHOT	352	0.012	142.256	28.26
		0.014	407.324	74.95
FPFH	33	0.012	158.339	35.55
		0.014	383.760	73.41
CFPFH	36	0.012	180.602	43.49
		0.014	416.313	75.49
ACFPFH	36	自适应最优	102.053

DOI:10.1016/j.promfg.2017.07.134 URL [本文引用: 1]

传统的SHOT、FPFH等特征描述符只描述了托盘的几何特征,忽略了托盘的颜色信息,因此托盘的识别精度较低;CFPFH特征描述符计算了托盘的HSV颜色特征,提高了托盘的识别精度,但其邻域半径要依靠复杂低效的人工调试方法获得,且获得的邻域半径并不适用于所有的点云关键点,较大的邻域半径导致关键点的邻域点过多,降低特征提取速度.ACFPFH特征描述符不仅增加了颜色信息,且根据邻域特征熵方法为每个关键点自适应选择最优邻域半径,使得点云关键点的特征描述符在具有较高识别精度的同时,特征提取用时也减少了.

在P-R曲线图中,曲线越靠近右上方,特征描述符的性能越好.由图19可见,与固定半径的SHOT、FPFH、CFPFH特征描述符相比,ACFPFH特征描述符具有最优性能.ACFPFH特征描述符在特征提取时自适应确定最优邻域半径,由表3和表4可知,d_th=0.75时,与邻域半径为0.012 m的SHOT特征描述符相比,托盘识别精度提高了44.67%,特征提取用时减少了28.26%;与邻域半径为0.012 m的FPFH特征描述符相比,托盘识别精度提高了83.74%,特征提取用时减少了35.55%;与邻域半径为0.012 m的CFPFH特征描述符相比,托盘识别精度提高了47.17%,特征提取用时减少了43.49%.由于Kinect V2传感器成本较低,所以其在仓储环境中的应用较为广泛,但Kinect V2传感器采集的点云密度不高,故本文基于SHOT、FPFH、CFPFH、ACFPFH特征描述符得到的托盘识别精度普遍在0.05-0.5范围内,考虑到托盘体积大、叉取灵活,基于ACFPFH特征描述符的托盘识别方法得到的精度符合仓储环境中对托盘的识别要求.

与传统的通过手动多次调试以取得相对较好邻域半径的特征提取方法相比,本文提出的ACFPFH方法能够根据点云分布情况自适应地选择邻域范围的大小,克服了邻域选择随意、低效的问题,有效提升了特征描述符的性能,加快了运算速度,并能够运用到托盘识别当中,对实际的仓储作业具有指导作用.

5 结语

为了提高仓储环境中托盘识别算法的精度与计算经济性,本文提出了一种基于ACFPFH特征描述符的托盘识别方法,包括点云预处理、关键点检测、ACFPFH特征提取、特征匹配及误匹配点对剔除等步骤.该方法克服了现有托盘识别方法低效耗时、鲁棒性差、特征提取时邻域半径选择随意的缺点,通过与固定半径的SHOT、FPFH、CFPFH特征描述符作对比,验证了ACFPFH特征描述符的优越性.进一步获取识别到的托盘的位姿信息,将其反馈给无人驾驶工业车辆的运动系统,可以实现托盘的自动高效叉取,有助于构建智能化工厂.未来将探索在不同光照、包含货架及多种托盘的场景中提高托盘识别算法的精度及速度的方法.

参考文献

原文顺序

文献年度倒序

文中引用次数倒序

被引期刊影响因子

[1]

CASADO

, LAPIDO

Y L

, LOSADA

D P

, et al.

Pose estimation and object tracking using 2D images

[J]. Procedia Manufacturing, 2017, 11: 63-71.

[2]

王伟男, 杨朝红.

基于图像处理技术的目标识别方法综述

[J]. 电脑与信息技术, 2019, 27(6): 9-15.

WANG

Weinan

, YANG

Chaohong

A survey of target recognition methods based on image processing technology

[J]. Computer and Information Technology, 2019, 27(6): 9-15.

DOI:10.11896/j.issn.1002-137X.2017.09.002 [本文引用: 1]

[3]

郝雯, 王映辉, 宁小娟, 等.

面向点云的三维物体识别方法综述

[J]. 计算机科学, 2017, 44(9): 11-16.

随着三维扫描技术的快速发展,获取各类场景的点云数据已经非常简单快捷；加之点云数据具备不受光照、阴影、纹理的影响等优势,基于点云的三维物体识别已成为计算机视觉领域的研究热点。首先,对近年来面向点云数据的三维物体识别方法进行归纳和总结；然后,对已有方法的优势及缺点进行分析；最后,指出点云物体识别中所面临的挑战及进一步的研究方向。

HAO

Wen

, WANG

Yinghui

, NING

Xiaojuan

, et al.

Survey of 3D object recognition for point clouds

[J]. Computer Science, 2017, 44(9): 11-16.

DOI:10.11896/j.issn.1002-137X.2017.09.002 [本文引用: 1]

With the rapid development of 3D scanning technology,it is convenient to obtain point clouds of different scenes.Since point clouds are not influenced by light,shadows and textures,recognizing 3D object from scene point clouds has become a research hotspot of computer vision.This paper first summarized the 3D object recognition methods from point clouds in recent years.Then the advantages and disadvantages of the existing methods were discussed.Finally,the challenges and further research directions of object recognition were pointed out.

[4]

GARCÍA-PULIDO

J A

, PAJARES

, DORMIDO

, et al.

Recognition of a landing platform for unmanned aerial vehicles by using computer vision-based techniques

[J]. Expert Systems With Applications, 2017, 76: 152-165.

DOI:10.1016/j.eswa.2017.01.017 URL [本文引用: 1]

[5]

CHEN

J M

, CHEN

L P

Multi-dimensional color image recognition and mining based on feature mining algorithm

[J]. Automatic Control and Computer Sciences, 2021, 55(2): 195-201.

DOI:10.3103/S0146411621020048 [本文引用: 1]

[6]

SEIDENARI

, SERRA

, BAGDANOV

A D

, et al.

Local pyramidal descriptors for image recognition

[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 1033-1040.

DOI:10.1109/TPAMI.2013.232 PMID:26353235 [本文引用: 1]

In this paper, we present a novel method to improve the flexibility of descriptor matching for image recognition by using local multiresolution pyramids in feature space. We propose that image patches be represented at multiple levels of descriptor detail and that these levels be defined in terms of local spatial pooling resolution. Preserving multiple levels of detail in local descriptors is a way of hedging one's bets on which levels will most relevant for matching during learning and recognition. We introduce the Pyramid SIFT (P-SIFT) descriptor and show that its use in four state-of-the-art image recognition pipelines improves accuracy and yields state-of-the-art results. Our technique is applicable independently of spatial pyramid matching and we show that spatial pyramids can be combined with local pyramids to obtain further improvement. We achieve state-of-the-art results on Caltech-101 (80.1%) and Caltech-256 (52.6%) when compared to other approaches based on SIFT features over intensity images. Our technique is efficient and is extremely easy to integrate into image recognition pipelines.

[7]

CHEN

, PENG

, WANG

Z C

, et al.

Pallet recognition and localization method for vision guided forklift

[C]// 2012 8th International Conference on Wireless Communications, Networking and Mobile Computing. Shanghai, China: IEEE, 2012: 1-4.

DOI:10.1007/s11042-016-4123-6 URL [本文引用: 1]

[8]

SYU

J L

, LI

H T

, CHIANG

J S

, et al.

A computer vision assisted system for autonomous forklift vehicles in real factory environment

[J]. Multimedia Tools and Applications, 2017, 76(18): 18387-18407.

[9]

T J

, HUANG

, LI

, et al.

Application of convolution neural network object detection algorithm in logistics warehouse

[J]. The Journal of Engineering, 2019, 2019(23): 9053-9058.

DOI:10.1049/tje2.v2019.23 URL [本文引用: 1]

[10]

SHAO

Y P

, WANG

, DU

S C

, et al.

High definition metrology enabled three dimensional discontinuous surface filtering by extended tetrolet transform

[J]. Journal of Manufacturing Systems, 2018, 49: 75-92.

DOI:10.1016/j.jmsy.2018.09.002 URL [本文引用: 1]

[11]

SHAO

Y P

, DU

S C

, TANG

H T

An extended bi-dimensional empirical wavelet transform based filtering approach for engineering surface separation using high definition metrology

[J]. Measurement, 2021, 178: 109259.

[12]

武文汉, 杨明, 王冰, 等.

一种基于轮廓匹配的仓储机器人托盘检测方法

[J]. 上海交通大学学报, 2019, 53(2): 197-202.

Wenhan

, YANG

Ming

, WANG

Bing

, et al.

Pallet detection based on contour matching for warehouse robots

[J]. Journal of Shanghai Jiao Tong University, 2019, 53(2): 197-202.

[13]

XIAO

J H

, LU

H M

, ZHANG

L L

, et al.

Pallet recognition and localization using an RGB-D camera

[J]. International Journal of Advanced Robotic Systems, 2017, 14(6): 172988141773779.

[14]

VARGA

, COSTEA

, NEDEVSCHI

Improved autonomous load handling with stereo cameras

[C]// 2015 IEEE International Conference on Intelligent Computer Communication and Processing. Cluj-Napoca, Romania: IEEE, 2015: 251-256.

[15]

VARGA

, NEDEVSCHI

Robust pallet detection for automated logistics operations

[C]// Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. Rome, Italy: SCITEPRESS-Science and Technology Publications, 2016: 470-477.

[16]

吴登禄, 曹文希, 朱颖.

基于三维点云和图像边缘的托盘检测技术研究

[J]. 自动化与信息工程, 2019, 40(3): 40-42.

Denglu

, CAO

Wenxi

, ZHU

Ying

Research on pallet detection technology based on 3D point cloud and image edge features

[J]. Automation & Information Engineering, 2019, 40(3): 40-42.

[17]

李洋洋, 史历程, 万卫兵, 等.

基于卷积神经网络的三维物体检测方法

[J]. 上海交通大学学报, 2018, 52(1): 7-12.

Yangyang

, SHI

Licheng

, WAN

Weibing

, et al.

A convolutional neural network-based method for 3D object detection

[J]. Journal of Shanghai Jiao Tong University, 2018, 52(1): 7-12.

[18]

TERABAYASHI

, TAKASHIMA

, SUZUKI

, et al.

Easy acquisition of range image dataset for object detection using retroreflective markers and a time-of-flight camera: An application to detection of forklift pallets

[C]// Proceedings of the Seventh Asia International Symposium on Mechatronics. Hangzhou, China: Springer Singapore, 2020: 1001-1005.

[19]

郭裕兰, 鲁敏, 谭志国, 等.

距离图像局部特征提取方法综述

[J]. 模式识别与人工智能, 2012, 25(5): 783-791.

基于距离图像的三维目标识别是计算机视觉领域的研究热点，而局部特征提取则是实现遮挡和复杂场景下三维目标识别的关键。文中首先介绍距离图像及其表示形式，详细分析法向量、曲率和形状索引等微分几何属性。进而将局部特征检测方法分类为固定尺度和自适应尺度方法，将局部特征描述方法分类为基于深度信息、基于点云空间分布和基于几何属性分布的方法，并对各种具体算法进行阐述、分析和定性评价。最后对现有方法进行归纳总结，并指出所面临的挑战及进一步研究的方向。

GUO

Yulan

, LU

Min

, TAN

Zhiguo

, et al.

Survey of local feature extraction on range images

[J]. Pattern Recognition and Artificial Intelligence, 2012, 25(5): 783-791.

DOI:10.1109/LRA.2017.2667721 URL [本文引用: 1]

Three dimensional (3D) object recognition is a hot research topic in computer vision. Local feature extraction is a key stage for 3D object recognition with the presence of occlusion and clutter. Firstly, range images and their representations are described. The differential geometric attributes are introduced, including the surface normal, the curvature and the shape index. Then, the local feature detection methods are classified into fixed scale method and adaptive scale method. And the local feature description methods are classified into depth value based, point spatial distribution based and geometric attributes distribution based methods. These methods with their merits and demerits are described. Finally, the existing methods are summarized and several challenges and future research directions are pointed out.

[20]

PRAKHYA

S M

, LIN

, CHANDRASEKHAR

, et al.

3DHoPD: A fast low-dimensional 3-D descriptor

[J]. IEEE Robotics and Automation Letters, 2017, 2(3): 1472-1479.

[21]

RUSU

R B

, BLODOW

, BEETZ

Fast point feature histograms (FPFH) for 3D registration

[C]// 2009 IEEE International Conference on Robotics and Automation. Kobe, Japan: IEEE, 2009: 3212-3217.

DOI:10.1007/s11263-015-0824-y URL [本文引用: 1]

[22]

GUO

Y L

, BENNAMOUN

, SOHEL

, et al.

A comprehensive performance evaluation of 3D local feature descriptors

[J]. International Journal of Computer Vision, 2016, 116(1): 66-89.

[23]

HUANG

, YOU

S Y

Detecting objects in scene point cloud: A combinational approach

[C]// 2013 International Conference on 3D Vision-3DV 2013. Seattle, WA, USA: IEEE, 2013: 175-182.

[24]

王斐, 梁宸, 韩晓光, 等.

基于焊件识别与位姿估计的焊接机器人视觉引导

[J]. 控制与决策, 2020, 35(8): 1873-1878.

WANG

Fei

, LIANG

Chen

, HAN

Xiaoguang

, et al.

Visual guidance of welding robot based on weldment recognition and pose estimation

[J]. Control and Decision, 2020, 35(8): 1873-1878.

DOI:10.3390/app8101776 URL [本文引用: 1]

[25]

LIU

, BAI

, CHEN

3-D point cloud registration algorithm based on greedy projection triangulation

[J]. Applied Sciences, 2018, 8(10): 1776.

To address the registration problem in current machine vision, a new three-dimensional (3-D) point cloud registration algorithm that combines fast point feature histograms (FPFH) and greedy projection triangulation is proposed. First, the feature information is comprehensively described using FPFH feature description and the local correlation of the feature information is established using greedy projection triangulation. Thereafter, the sample consensus initial alignment method is applied for initial transformation to implement initial registration. By adjusting the initial attitude between the two cloud points, the improved initial registration values can be obtained. Finally, the iterative closest point method is used to obtain a precise conversion relationship; thus, accurate registration is completed. Specific registration experiments on simple target objects and complex target objects have been performed. The registration speed increased by 1.1% and the registration accuracy increased by 27.3% to 50% in the experiment on target object. The experimental results show that the accuracy and speed of registration have been improved and the efficient registration of the target object has successfully been performed using the greedy projection triangulation, which significantly improves the efficiency of matching feature points in machine vision.

[26]

, WANG

, ZHAO

Y D

, et al.

Improved algorithm for point cloud registration based on fast point feature histograms

[J]. Journal of Applied Remote Sensing, 2016, 10: 045024.

DOI:10.1016/j.bspc.2017.06.006 URL [本文引用: 1]

[27]

NAPOLI

, GLASS

, WARD

, et al.

Performance analysis of a generalized motion capture system using microsoft kinect 2.0

[J]. Biomedical Signal Processing and Control, 2017, 38: 265-280.

[28]

DEMANTKE

, MALLET

, DAVID

, et al.

Dimensionality based scale selection in 3D LIDAR point clouds

[C]// ISPRS Workshop Laser Scanning. Calgary, Canada: Copernicus Gesellschaft Mbh, 2011: 97-102.

[29]

WEINMANN

, JUTZI

, MALLET

Semantic 3D scene interpretation: A framework combining optimal neighborhood size selection with relevant features

[J]. Photogrammetry, Remote Sensing and Spatial Information Sciences, 2014, 2(3): 181-188.

[30]

王红雨, 尹午荣, 汪梁, 等.

基于HSV颜色空间的快速边缘提取算法

[J]. 上海交通大学学报, 2019, 53(7): 765-772.

WANG

Hongyu

, YIN

Wurong

, WANG

Liang

, et al.

Fast edge extraction algorithm based on HSV color space

[J]. Journal of Shanghai Jiao Tong University, 2019, 53(7): 765-772

[31]

熊风光, 蔡晋茹, 况立群, 等.

三维点云模型中特征点描述子及其匹配算法研究

[J]. 小型微型计算机系统, 2017, 38(3): 640-644.

XIONG

Fengguang

, CAI

Jinru

, KUANG

Liqun

, et al.

Study on descriptor and matching algorithm of feature point in 3D point cloud

[J]. Journal of Chinese Computer Systems, 2017, 38(3): 640-644.