基于模型的增强现实无标识三维注册追踪方法

doi:10.16183/j.cnki.jsjtu.2018.01.013

摘要/Abstract

摘要： 为了克服物体表面缺少足够的纹理特征且算法搜索空间太大的缺陷，提出了一种基于模型与自然特征点相融合的三维注册追踪方法.采用保持旋转和尺度不变性的线性并行多模态(LINE-MOD)模板匹配方法快速识别目标物体、获取与当前视角接近的参考视图而完成相机位姿的粗略估计，并缩小算法的搜索空间；采用基于自然特征点的方法完成相机位姿的精确计算；为了避免因特征点较少而引起的位姿抖动或扰动，引入了有效的非迭代透视n点问题(RPnP)算法以提高注册追踪的精度和速度.结果表明，所提出的注册追踪方法能够进行快速三维注册，具有良好的实时性和鲁棒性，其运算速度可达30帧/s.

关键词: 增强现实；三维注册追踪；无标识

Abstract: To tackle the huge searching space and few texture feature points on the object surface problem, we propose a novel method that combines model-based tracking and natural feature tracking for fast and robust camera pose calculation. The proposed method first adapts the state-of-the-art template matching method LINE-MOD into a scale invariant descriptor to recognize the target object and obtain the key-frame that is similar to the current viewpoint. Therefore, it greatly reduces the search space for pose refinement method and improves the searching speed. After that, the keyframe obtained from template matching stage is exploited to refine the tracking accuracy. In order to avoid tracking jitter and improve tracking accuracy and speed, RPnP algorithm is incorporated into our system. The experimental results and performance evaluations demonstrate that our method is fast, accurate and robust for 3D registration and tracking even under partial occlusion, and the tracking speed can reach 30 frame/s.

Key words: augmented reality; 3D registration and tracking; marker-less

中图分类号:

TP 391.4

王月，张树生，何卫平，白晓亮. 基于模型的增强现实无标识三维注册追踪方法[J]. 上海交通大学学报, 2018, 52(1): 83-89.

WANG Yue,ZHANG Shusheng,HE Weiping,BAI Xiaoliang. Model-Based Marker-Less 3D Tracking Approach for Augmented Reality[J]. Journal of Shanghai Jiao Tong University, 2018, 52(1): 83-89.

参考文献

［1］WESTERFIELD G, MITROVIC A, BILLINGHURST M. Intelligent augmented reality training for motherboard assembly［J］. International Journal of Artificial Intelligence in Education, 2015, 25(1): 157-172. ［2］FIORENTION M, UVA A E, GATTULLO M, et al. Augmented reality on large screen for interactive maintenance instructions［J］. Computers in Industry, 2014, 65(2): 270-278. ［3］CRIVELLARO A, RAD M, VERDIE Y, et al. A novel representation of parts for accurate 3D object detection and tracking in monocular images［C］∥Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015: 4391-4399. ［4］MOTTAGHI R, YU X, SILVIO S. A coarse-to-fine model for 3D pose estimation and sub-category recognition ［C］∥IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE, 2015, 418-426. ［5］ENGEL J, THOMAS S, DANIEL C. LSD-SLAM: Large-scale direct monocular SLAM［C］∥European Conference on Computer Vision (ECCV). Zurich, Switzerland: Springer, 2014: 834-849. ［6］PAUWELS K, RUBIO L, DIAZ J, et al. Real-time model-based rigid object pose estimation and tracking combining dense and sparse visual cues［C］∥Conference on Computer Vision and Pattern Recognition (CVPR). Portland, USA: IEEE, 2013: 2347-2354. ［7］KYRIAZIS N, ANTONIS A. Scalable 3D tracking of multiple interacting objects［C］∥IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus, USA: IEEE, 2014, 3430-3437. ［8］PAYET N, SINISA T. From contours to 3D object detection and pose estimation［C］∥International Conference on Computer Vision (ICCV). Barcelona, Spain: IEEE, 2011: 983-990. ［9］RADKOWSKI R. Object tracking with a range camera for augmented reality assembly assistance［J］. Journal of Computing and Information Science in Engineering, 2016, 16(1): 011004. ［10］GARRETT T, DEBERNARDIS S, OLIVER J, et al. Poisson mesh reconstruction for accurate object tracking with low-fidelity point clouds［J］. Journal of Computing and Information Science in Engineering, 2017, 17(1): 011003. ［11］WANG G, WANG B, ZHONG F, et al. Global optimal searching for textureless 3D object tracking［J］. The Visual Computer, 2015, 31(6-8): 979-988. ［12］NEWCOMBE R A, IZADI S, HILLIGES O, et al. KinectFusion: Real-time dense surface mapping and tracking［C］∥10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Basel, Switzerland: IEEE, 2011: 127-136. ［13］HINTERSTOISSER S, CAGNIART C, ILIC S, et al. Gradient response maps for real-time detection of textureless objects［J］. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2012, 34(5): 876-888. ［14］RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: An efficient alternative to SIFT or SURF［C］∥International Conference on Computer Vision (ICCV). Barcelona, Spain: IEEE, 2011: 2564-2571. ［15］KALAL Z, MATAS J, MIKOLAJCZYK K. P-N learning: Bootstrapping binary classifiers by structural constraints［C］∥Conference on Computer Vision and Pattern Recognition (CVPR). San Francisco, USA: IEEE, 2010, 49-56. ［16］周凯汀, 郑力新. 基于改进 ORB 特征的多姿态人脸识别［J］. 计算机辅助设计与图形学学报, 2015, 27(2): 287-295. ZHOU Kaiting, ZHENG Lixin. Multi-pose face recognization based on improved ORB features［J］. Journal of Computer-Aided Design and Computer Graphics, 2015, 27(2): 287-295. ［17］LU C P, HAGER G D, MJOLSNESS E. Fast and globally convergent pose estimation from video images［J］. Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(6): 610-622. ［18］LI S Q, XU C, XIE, M. A robust O(n) solution to the perspective-n-point problem［J］. Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(7): 1444-1450. ［19］ABAWI D F, JOACHIM B, RALF D. Accuracy in optical tracking with fiducial markers: An accuracy function for ARToolKit［C］∥Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality. Arlington, USA: IEEE, 2004: 260-261.

[1]	白雄飞, 龚水成, 李雪松, 许博, 杨晓力, 王明彦. 基于泊松融合数据增强的焊缝金相组织缺陷分类研究[J]. 上海交通大学学报, 2023, 57(10): 1316-1328.
[2]	高涛, 文渊博, 陈婷, 张静. 基于窗口自注意力网络的单图像去雨算法[J]. 上海交通大学学报, 2023, 57(5): 613-623.
[3]	叶仙，胡洁，田畔，戚进，车大钿，丁颖. 基于精细复合多尺度熵与支持向量机的睡眠分期[J]. 上海交通大学学报（自然版）, 2019, 53(3): 321-326.
[4]	刘宇，江宏毅，王仕亮，王伊冰，陈燕苹. 基于加速度时域特征的实时人体行为模式识别[J]. 上海交通大学学报（自然版）, 2015, 49(02): 169-172.
[5]	王佳婧1，2，张树生1，何卫平1. 基于多信息融合的失效DPM码识别[J]. 上海交通大学学报（自然版）, 2014, 48(12): 1675-1680.
[6]	赵洪伟，谢永芳，曹斌芳，蒋朝辉. 基于Gabor小波和LPP的浮选过程泡沫纹理特征提取及应用[J]. 上海交通大学学报（自然版）, 2014, 48(07): 942-947.
[7]	贾松敏1,王爽1,王丽佳1,2,李秀智1. CTF定位策略下基于多特征的智能机器人目标跟踪[J]. 上海交通大学学报（自然版）, 2014, 48(07): 1039-1045.
[8]	王佳婧1,2，张树生1，何卫平1. 金属零件上二维条码高光区域的恢复[J]. 上海交通大学学报（自然版）, 2013, 47(12): 1836-1841.
[9]	方挺，董冲，胡兴柳，王彦. 航拍图像中绝缘子串的轮廓提取和故障检测[J]. 上海交通大学学报（自然版）, 2013, 47(12): 1818-1822.
[10]	刘洪涛，梁振宁，胡文，莫锦秋，王石刚. 基于局部特征点检测与匹配的微悬臂梁变形受力测量方法[J]. 上海交通大学学报（自然版）, 2013, 47(12): 1842-1847.
[11]	王晓端1，刘书炘2，刘满华1. 基于模型分析的指纹奇异点检测[J]. 上海交通大学学报（自然版）, 2013, 47(08): 1222-1226.
[12]	胡文1, 刘洪涛1, 胡春2, 范明霞3, 莫锦秋1, 王石刚1. 基于伪Zernike矩不变量分析的视觉测力[J]. 上海交通大学学报（自然版）, 2013, 47(04): 589-593.
[13]	刘洪涛, 蒋如意, 胡文, 莫锦秋, 王石刚. 基于路面垂直轮廓B样条拟合的车辆运动估计偏移抑制[J]. 上海交通大学学报（自然版）, 2013, 47(04): 630-634.
[14]	李斌, 叶昊. 基于仿射几何不变量的鲁棒图像配准算法[J]. 上海交通大学学报（自然版）, 2012, 46(12): 1881-1884.
[15]	吕复强1, 王慧1, 刘泓2. 基于改进混合高斯模型的交通流背景检测算法[J]. 上海交通大学学报（自然版）, 2012, 46(12): 1920-1925.