J Shanghai Jiaotong Univ Sci ›› 2023, Vol. 28 ›› Issue (6): 717-727.doi: 10.1007/s12204-022-2471-0
武 星1, 2*,张庆丰1, 王健嘉1, 2, 姚骏峰3, 郭毅可4,5
接受日期:
2021-06-20
出版日期:
2023-11-28
发布日期:
2023-12-04
WU Xing1,2*(武星),ZHANG Qingfeng1(张庆丰),WANG Jianjia1,2(王健嘉), YAO Junfeng3(姚骏峰),Guo Yike4.5(郭毅可)
Accepted:
2021-06-20
Online:
2023-11-28
Published:
2023-12-04
摘要: 印刷电路板(PCB)是电子产品不可或缺的组成部分,它决定了这些产品的质量。随着制造技术的发展和进步,PCB的布局和结构变得越来越复杂。然而,有效准确的PCB缺陷检测方法却很少。在实际生产环境中,对PCB缺陷检测的准确性有很高的要求,因此我们提出了两种包括多模型融合的PCB缺陷检测框架,包括多模型投票方法(DDMV)和多模型学习方法(DDML)的缺陷检测。为了减少错误和漏检,DDMV和DDML将多个具有不同融合策略的缺陷检测网络进行整合。通过对两个开源PCB数据集进行大量实验,验证了所提出框架的有效性和准确性。实验结果表明,所提出的DDMV和DDML在F1分数方面优于其他任何单独的最先进的PCB缺陷检测模型,而DDML的曲线下面积值也高于其他任何单独的检测模型。此外,与DDMV相比,使用自动机器学习方法的DDML在PCB缺陷检测方面达到了最好的性能,在两个数据集上的F1分数分别可达到99.7%和95.6%。
中图分类号:
武 星, 张庆丰, 王健嘉, 姚骏峰, 郭毅可, . 基于多重检测模型融合框架的印刷电路板缺陷检测[J]. J Shanghai Jiaotong Univ Sci, 2023, 28(6): 717-727.
WU Xingl(武星), ZHANG Qingfeng(张庆丰), WANG Jianjia(王健嘉), YAO Junfeng(姚骏峰), Guo Yike.(郭毅可). Multiple Detection Model Fusion Framework for Printed Circuit Board Defect Detection[J]. J Shanghai Jiaotong Univ Sci, 2023, 28(6): 717-727.
[8] | LIN T Y, DOLL′ AR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 936-944. |
[1] | CHAUDHARY V, DAVE I R, UPLA K P. Automatic visual inspection of printed circuit board for defect detection and classification [C]//2017 International Conference on Wireless Communications, Signal Processing and Networking. Chennai: IEEE, 2017: 732-737. |
[9] | JIANG H Z, LEARNED-MILLER E. Face detection with the faster R-CNN [C]//2017 12th IEEE International Conference on Automatic Face & Gesture Recognition. Washington, DC, USA: IEEE, 2017: 650-657. |
[2] | ZHU J H, WU A, LIU X P. Printed circuit board defect visual detection based on wavelet denoising [J]. IOP Conference Series: Materials Science and Engineering, 2018, 392: 062055. |
[10] | WANG Y, LUO X B, DING L, et al. Detection based visual tracking with convolutional neural network [J]. Knowledge-Based Systems, 2019, 175: 62-71. |
[3] | KUO C F J, FANG T Y, LEE C L, et al. Automated optical inspection system for surface mount device light emitting diodes [J]. Journal of Intelligent Manufacturing, 2019, 30(2): 641-655. |
[11] | WEI H, YANG C Z, YU Q. Efficient graph-based search for object detection [J]. Information Sciences, 2017, 385/386: 395-414. |
[4] | KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM, 2017, 60(6): 84-90. |
[12] | BRIA A, MARROCCO C, MOLINARA M, et al. An effective learning strategy for cascaded object detection [J]. Information Sciences, 2016, 340/341: 17-26. |
[5] | WU X, ZHONG M Y, GUO Y K, et al. The assessment of small bowel motility with attentive deformable neural network [J]. Information Sciences, 2020, 508: 22-32. |
[13] | OLSON R S, MOORE J H. TPOT: A tree-based pipeline optimization tool for automating machine learning [M]//Automated machine learning. Cham: Springer, 2019: 151-160. |
[6] | LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector [M]//Computer vision -ECCV 2016. Cham: Springer, 2016: 21-37. |
[14] | HOSANG J, BENENSON R, SCHIELE B. Learning non-maximum suppression [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 6469-6477. |
[7] | REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE, 2016: 779- 788. |
[15] | DING R W, DAI L H, LI G P, et al. TDD-net: a tiny defect detection network for printed circuit boards [J].CAAI Transactions on Intelligence Technology, 2019, 4(2): 110-116. |
[8] | LIN T Y, DOLL′ AR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 936-944. |
[16] | TANG S L, HE F, HUANG X L, et al. Online PCB defect detector on a new PCB defect dataset [DB/OL]. (2019-02-17). https://arxiv.org/abs/1902.06197. |
[9] | JIANG H Z, LEARNED-MILLER E. Face detection with the faster R-CNN [C]//2017 12th IEEE International Conference on Automatic Face & Gesture Recognition. Washington, DC, USA: IEEE, 2017: 650-657. |
[17] | DALAL N, TRIGGS B. Histograms of oriented gradients for human detection [C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA: IEEE, 2005: 886-893. |
[10] | WANG Y, LUO X B, DING L, et al. Detection based visual tracking with convolutional neural network [J]. Knowledge-Based Systems, 2019, 175: 62-71. |
[18] | LOWE D G. Distinctive image features from scaleinvariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91-110. |
[11] | WEI H, YANG C Z, YU Q. Efficient graph-based search for object detection [J]. Information Sciences, 2017, 385/386: 395-414. |
[19] | BAY H, TUYTELAARS T, VAN GOOL L. SURF: speeded up robust features [M]//Computer vision - ECCV 2006. Berlin, Heidelberg: Springer, 2006: 404- 417. |
[12] | BRIA A, MARROCCO C, MOLINARA M, et al. An effective learning strategy for cascaded object detection [J]. Information Sciences, 2016, 340/341: 17-26. |
[20] | LU Z S, HE Q Q, XIANG X G, et al. Defect detection of PCB based on Bayes feature fusion [J]. The Journal of Engineering, 2018, 2018(16): 1741-1745. |
[13] | OLSON R S, MOORE J H. TPOT: A tree-based pipeline optimization tool for automating machine learning [M]//Automated machine learning. Cham: Springer, 2019: 151-160. |
[21] | BENEDEK C. Detection of soldering defects in printed circuit boards with hierarchical marked point processes [J]. Pattern Recognition Letters, 2011, 32(13): 1535- 1543. |
[14] | HOSANG J, BENENSON R, SCHIELE B. Learning non-maximum suppression [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 6469-6477. |
[22] | GAIDHANE V H, HOTE Y V, SINGH V. An efficient similarity measure approach for PCB surface defect detection [J]. Pattern Analysis and Applications, 2018, 21(1): 277-289. |
[15] | DING R W, DAI L H, LI G P, et al. TDD-net: a tiny defect detection network for printed circuit boards [J].CAAI Transactions on Intelligence Technology, 2019, 4(2): 110-116. |
[23] | ZHANG C, SHI W, LI X F, et al. Improved bare PCB defect detection approach based on deep feature learning [J]. The Journal of Engineering, 2018, 2018(16): 1415-1420. |
[16] | TANG S L, HE F, HUANG X L, et al. Online PCB defect detector on a new PCB defect dataset [DB/OL]. (2019-02-17). https://arxiv.org/abs/1902.06197. |
[24] | DAI W T, MUJEEB A, ERDT M, et al. Soldering defect detection in automatic optical inspection [J]. Advanced Engineering Informatics, 2020, 43: 101004. |
[17] | DALAL N, TRIGGS B. Histograms of oriented gradients for human detection [C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA: IEEE, 2005: 886-893. |
[25] | BENJDIRA B, KHURSHEED T, KOUBAA A, et al. Car detection using unmanned aerial vehicles: Comparison between faster R-CNN and YOLOv3 [C]//2019 1st International Conference on Unmanned Vehicle Systems-Oman. Muscat: IEEE, 2019: 1-6. |
[18] | LOWE D G. Distinctive image features from scaleinvariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91-110. |
[26] | LEI H W, WANG B, WU H H, et al. Defect detection for polymeric polarizer based on faster R-CNN [J]. Journal of Information Hiding and Multimedia Signal Processing, 2018, 9(6): 1414-1420. |
[19] | BAY H, TUYTELAARS T, VAN GOOL L. SURF: speeded up robust features [M]//Computer vision - ECCV 2006. Berlin, Heidelberg: Springer, 2006: 404- 417. |
[27] | LI Y T, HUANG H S, XIE Q S, et al. Research on a surface defect detection algorithm based on MobileNet-SSD [J]. Applied Sciences, 2018, 8(9): 1678. |
[20] | LU Z S, HE Q Q, XIANG X G, et al. Defect detection of PCB based on Bayes feature fusion [J]. The Journal of Engineering, 2018, 2018(16): 1741-1745. |
[28] | LI J Y, SU Z F, GENG J H, et al. Real-time detection of steel strip surface defects based on improved YOLO detection network [J]. IFAC-PapersOnLine, 2018, 51(21): 76-81. |
[21] | BENEDEK C. Detection of soldering defects in printed circuit boards with hierarchical marked point processes [J]. Pattern Recognition Letters, 2011, 32(13): 1535- 1543. |
[29] | HOU W, WEI Y, GUO J, et al. Automatic detection of welding defects using deep neural network [J]. Journal of Physics: Conference Serie, 2017, 933: 012006. |
[22] | GAIDHANE V H, HOTE Y V, SINGH V. An efficient similarity measure approach for PCB surface defect detection [J]. Pattern Analysis and Applications, 2018, 21(1): 277-289. |
[30] | LIN H, LI B, WANG X G, et al. Automated defect inspection of LED chip using deep convolutional neural network [J]. Journal of Intelligent Manufacturing, 2019, 30(6): 2525-2534. |
[23] | ZHANG C, SHI W, LI X F, et al. Improved bare PCB defect detection approach based on deep feature learning [J]. The Journal of Engineering, 2018, 2018(16): 1415-1420. |
[31] | LIN J H, YAO Y, MA L, et al. Detection of a casting defect tracked by deep convolution neural network [J]. The International Journal of Advanced Manufacturing Technology, 2018, 97(1/2/3/4): 573-581. |
[24] | DAI W T, MUJEEB A, ERDT M, et al. Soldering defect detection in automatic optical inspection [J]. Advanced Engineering Informatics, 2020, 43: 101004. |
[32] | NASROLLAHI M, BOLOURIAN N, HAMMAD A. Concrete surface defect detection using deep neural network based on lidar scanning [C]//CSCE Annual Conference. Laval: CSCE, 2019: CON032. |
[25] | BENJDIRA B, KHURSHEED T, KOUBAA A, et al. Car detection using unmanned aerial vehicles: Comparison between faster R-CNN and YOLOv3 [C]//2019 1st International Conference on Unmanned Vehicle Systems-Oman. Muscat: IEEE, 2019: 1-6. |
[33] | MEI S, WANG Y D, WEN G J. Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model [J]. Sensors, 2018, 18(4): 1064. |
[26] | LEI H W, WANG B, WU H H, et al. Defect detection for polymeric polarizer based on faster R-CNN [J]. Journal of Information Hiding and Multimedia Signal Processing, 2018, 9(6): 1414-1420. |
[34] | ADIBHATLA V A, CHIH H C, HSU C C, et al. Defect detection in printed circuit boards using You-onlylook-once convolutional neural networks [J]. Electronics, 2020, 9(9): 1547. |
[27] | LI Y T, HUANG H S, XIE Q S, et al. Research on a surface defect detection algorithm based on MobileNet-SSD [J]. Applied Sciences, 2018, 8(9): 1678. |
[35] | ZHANG X, YANG Y H, HAN Z G, et al. Object class detection [J]. ACM Computing Surveys, 2013, 46(1): 1-53. |
[28] | LI J Y, SU Z F, GENG J H, et al. Real-time detection of steel strip surface defects based on improved YOLO detection network [J]. IFAC-PapersOnLine, 2018, 51(21): 76-81. |
[36] | SENGUPTA A, YE Y T, WANG R, et al. Going deeper in spiking neural networks: VGG and residual architectures [J]. Frontiers in Neuroscience, 2019, 13: 95. |
[29] | HOU W, WEI Y, GUO J, et al. Automatic detection of welding defects using deep neural network [J]. Journal of Physics: Conference Serie, 2017, 933: 012006. |
[37] | HENDRY, CHEN R C. Automatic License Plate Recognition via sliding-window darknet-YOLO deep learning [J]. Image and Vision Computing, 2019, 87: 47-56. |
[30] | LIN H, LI B, WANG X G, et al. Automated defect inspection of LED chip using deep convolutional neural network [J]. Journal of Intelligent Manufacturing, 2019, 30(6): 2525-2534. |
[31] | LIN J H, YAO Y, MA L, et al. Detection of a casting defect tracked by deep convolution neural network [J]. The International Journal of Advanced Manufacturing Technology, 2018, 97(1/2/3/4): 573-581. |
[32] | NASROLLAHI M, BOLOURIAN N, HAMMAD A. Concrete surface defect detection using deep neural network based on lidar scanning [C]//CSCE Annual Conference. Laval: CSCE, 2019: CON032. |
[33] | MEI S, WANG Y D, WEN G J. Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model [J]. Sensors, 2018, 18(4): 1064. |
[34] | ADIBHATLA V A, CHIH H C, HSU C C, et al. Defect detection in printed circuit boards using You-onlylook-once convolutional neural networks [J]. Electronics, 2020, 9(9): 1547. |
[35] | ZHANG X, YANG Y H, HAN Z G, et al. Object class detection [J]. ACM Computing Surveys, 2013, 46(1): 1-53. |
[36] | SENGUPTA A, YE Y T, WANG R, et al. Going deeper in spiking neural networks: VGG and residual architectures [J]. Frontiers in Neuroscience, 2019, 13: 95. |
[37] | HENDRY, CHEN R C. Automatic License Plate Recognition via sliding-window darknet-YOLO deep learning [J]. Image and Vision Computing, 2019, 87: 47-56. |
[1] | 陈潇1,2,张瑞1,2,汤心溢1,2,钱娟3. 交叉特征深度编码网络预测儿童脓毒症[J]. J Shanghai Jiaotong Univ Sci, 2024, 29(1): 131-140. |
[2] | 张晟嘉, 林天成, 徐奕, . 利用软伪标签促进无监督领域自适应与课程学习[J]. J Shanghai Jiaotong Univ Sci, 2023, 28(6): 703-716. |
[3] | . [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(4): 498-504. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||