A Comprehensive Method to Reject Detection Outliers by Combining Template Descriptor with Sparse 3D Point Clouds

doi:10.1007/s12204-017-1820-x

Abstract

Abstract: We are using a template descriptor on the image in order to try and find the object. However, we have a sparse 3D point clouds of the world that is not used at all when looking for the object in the images. Considering there are many false alarms during the detection, we are interested in exploring how to combine the detections on the image with the 3D point clouds in order to reject some detection outliers. In this experiment we use semi-direct-monocular visual odometry (SVO) to provide 3D points coordinates and camera poses to project 3D points to 2D image coordinates. By un-projecting points in the tracking on the selection tree (TST) detection box back to 3D space, we can use 3D Gaussian ellipsoid fitting to determine object scales. By ruling out different scales of detected objects, we can reject most of the detection outliers of the object.

Key words: semi-direct-monocular visual odometry (SVO)| tracking on the selection tree (TST)-recognizer| 3D point-clouds| Gaussian ellipsoid fitting

摘要： We are using a template descriptor on the image in order to try and find the object. However, we have a sparse 3D point clouds of the world that is not used at all when looking for the object in the images. Considering there are many false alarms during the detection, we are interested in exploring how to combine the detections on the image with the 3D point clouds in order to reject some detection outliers. In this experiment we use semi-direct-monocular visual odometry (SVO) to provide 3D points coordinates and camera poses to project 3D points to 2D image coordinates. By un-projecting points in the tracking on the selection tree (TST) detection box back to 3D space, we can use 3D Gaussian ellipsoid fitting to determine object scales. By ruling out different scales of detected objects, we can reject most of the detection outliers of the object.

关键词: semi-direct-monocular visual odometry (SVO)| tracking on the selection tree (TST)-recognizer| 3D point-clouds| Gaussian ellipsoid fitting

CLC Number:

TP 181

GUO Li (郭立). A Comprehensive Method to Reject Detection Outliers by Combining Template Descriptor with Sparse 3D Point Clouds[J]. Journal of shanghai Jiaotong University (Science), 2017, 22(2): 188-192.

References 11

[1]	LEE T, SOATTO S. Learning and matching multiscaletemplate descriptors for real-time detection, localizationand tracking [C]//In Proceedings of the IEEEConference on Computer Vision and Pattern Recognition(CVPR). Colorado, USA: Institute of Electricaland Electronic Engineers, 2011: 1457-1464.
[2]	BABENKO B, YANG M H, BELONGIE S. Visualtracking with online multiple instance learning [C]//InProceedings of the IEEE Conference on Computer Visionand Pattern Recognition (CVPR). San Fransico,USA: Institute of Electrical and Electronic Engineers,2009: 983-990.
[3]	HINTERSTOISSER S, LEPETIE V, ILIC S, et al.Dominant orientation templates for real-time detectionof texture-less objects [C]//In Proceedings of the IEEEConference on Computer Vision and Pattern Recognition(CVPR). Miami, USA: Institute of Electrical andElectronic Engineers, 2010: 2057-2069.
[4]	LEE T, SOATTO S. TST/BTD: An end-to-end visualrecognition system [R]. Los Angeles: UCLA TechnicalReport, 2010.
[5]	BL¨oSCH M, WEISS S, SCARAMUZZA D, et al.Vision based MAV navigation in unknown andunstructured environments [C]//Proceeding IEEEInternational Conference on Robotics and Automation.Alaska, USA: [s.n.], 2010: 21-28.
[6]	WEISEE S, ACHTELIK M W, LYNEN S, et al.Monocular vision for long-term micro aerial vehiclestate estimation: A Compendium [J]. Journal of FieldRobotics, 2013, 30(5): 803-831.
[7]	SCARAMYZZA D, ACHTELIK M, DOITSIDIS L, etal. Vision-controlled micro flying robots: From systemdesign to autonomous navigation and mapping inGPS-denied environments [J]. IEEE Robotics and AutomationMagazine, 2014, 21(3): 26-40.
[8]	FORSTER C, LYNEN S, KNEIP L, et al. Collaborativemonocular SLAM with multiple micro aerial vehicles[J]. Proceeding IEEE/RSJ International Conferenceon Intelligent Robots and Systems, 2013, 8215(2):3962-3970.
[9]	FORSTER C, PIZZOLI M, SCARAMUZZA D.Air-ground localization and map augmentation usingmonocular dense reconstruction [C]//ProceedingIEEE/RJ International Conference on IntelligentRobots and Systems. Tokyo, Japan: IEEE, 2013: 592-625.
[10]	FORSTER C, PIZZOLI M, SCARAMUZZA D. SVO:fast semi-direct monocular visual odometry [C]//IEEEInternational Conference on Robotics and Automation(ICRA). Hong Kong, China: IEEE, 2014: 624-675.
[11]	DEVERNAY F, FAUGERAS O. Straight lines haveto be straight: Automatic calibration and removal ofdistortion from scenes of structured environments [J].Machine Vision and Applications, 2001, 13(1): 14-24.

[1]	LIANG Liang1 (梁良), SHI Ying1∗ (石英), MOU Junmin2∗ (牟军敏). Submarine Multi-Model Switching Control Under Full Working Condition Based on Machine Learning [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(3): 402-410.
[2]	JIA Dengqiang* (贾灯强), LUO Xinzhe (罗鑫喆), DING Wangbin (丁王斌),HUANG Liqin (黄立勤), ZHUANG Xiahai (庄吓海). SeRN: A Two-Stage Framework of Registration for Semi-Supervised Learning for Medical Images [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(2): 176-189.
[3]	YU Qing (余青), MA Yi (马祎), LI Yongfu∗ (李永福). Enhancing Speech Recognition for Parkinson’s Disease Patient Using Transfer Learning Technique [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 90-98.
[4]	XIA Ming (夏明), XU Tianyi (徐天意), JIANG Hong∗ (姜虹). Progress and Perspective of Artificial Intelligence and Machine Learning of Prediction in Anesthesiology [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 112-120.
[5]	LIU Mingguang, LIAO Yaxuan, LI Xiangshun . Data-Driven Fault Detection of Three-Tank System Applying MWAT-ICA [J]. J Shanghai Jiaotong Univ Sci, 2020, 25(5): 659-664.
[6]	WANG Shengsheng, ZHANG Hang, CHEN Juan . Dual Sum-Product Networks Autoencoder for Multi-Label Classification [J]. Journal of Shanghai Jiao Tong University(Science), 2020, 25(5): 665-673.
[7]	WANG Yinglin . Fine-Grained Opinion Mining on Chinese Car Reviews with Conditional Random Field [J]. Journal of Shanghai Jiao Tong University (Science), 2020, 25(3): 325-332.
[8]	ZHONG Haowen (钟昊文), WANG Chao (王超), TUO Hongya (庹红娅), HU Jian (胡健), QIAO Lingfeng (乔凌峰), JING Zhongliang (敬忠良). Transfer Learning Based on Joint Feature Matching and Adversarial Networks [J]. Journal of Shanghai Jiao Tong University (Science), 2019, 24(6): 699-705.
[9]	ZHENG Bo (郑波), LI Yanfeng (李彦锋), FU Guozhong (付国忠). Discretization Algorithm Based on Particle Swarm Optimization and Its Application in Attributes Reduction for Fault Data [J]. Journal of Shanghai Jiao Tong University (Science), 2018, 23(5): 691-695.
[10]	ZHANG Xiaoping1,2* (张晓平), RUAN Xiaogang1 (阮晓钢), XIAO Yao1 (肖尧), HUANG Jing1 (黄静). Sensorimotor Self-Learning Model Based on Operant Conditioning for Two-Wheeled Robot [J]. Journal of shanghai Jiaotong University (Science), 2017, 22(2): 148-155.
[11]	WANG Yinglin (王英林). Stock Market Forecasting with Financial Micro-Blog Based on Sentiment and Time Series Analysis [J]. Journal of shanghai Jiaotong University (Science), 2017, 22(2): 173-179.
[12]	HUANG Jiaxuan (黄嘉烜), JIN Xiaogang* (金小刚). Continuous Multiplicative Attribute Graph Model [J]. Journal of shanghai Jiaotong University (Science), 2017, 22(1): 87-091.
[13]	TU Shi-tao* (涂世涛), ZHU Lan-juan (朱兰娟). A Bandit Method Using Probabilistic Matrix Factorization in Recommendation [J]. Journal of shanghai Jiaotong University (Science), 2015, 20(5): 535-539.
[14]	LIU Xi-juan1* (刘溪涓), WANG Ying-lin2 (王英林). Semantic-Based Knowledge Categorization and Organization for Product Design Enterprises [J]. Journal of shanghai Jiaotong University (Science), 2015, 20(1): 106-112.
[15]	WANG Ying-lin1,2 (王英林), XU He-ming2* (徐鹤鸣). Multiobjective Particle Swarm Optimization Without the Personal Best [J]. Journal of shanghai Jiaotong University (Science), 2014, 19(2): 155-159.