J Shanghai Jiaotong Univ Sci

Journal of Shanghai Jiaotong University(Science) >

2023 , Vol. 28 >Issue 6: 717 - 727

DOI: https://doi.org/10.1007/s12204-022-2471-0

Computing & Computer Technologies

Multiple Detection Model Fusion Framework for Printed Circuit Board Defect Detection

Expand
  • (1. School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China; 2. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai 200444, China; 3. Shanghai Marine High-End Equipment Functional Platform Co., Ltd., Shanghai 201306, China; 4. Hong Kong Baptist University, Hong Kong, China; 5. Data Science Institute, Imperial College London, London SW7 2AZ, UK)

Accepted date: 2021-06-20

  Online published: 2023-12-04

Fold

Abstract

The printed circuit board (PCB) is an indispensable component of electronic products, which determines the quality of these products. With the development and advancement of manufacturing technology, the layout and structure of PCB are getting complicated. However, there are few effective and accurate PCB defect detection methods. There are high requirements for the accuracy of PCB defect detection in the actual production environment, so we propose two PCB defect detection frameworks with multiple model fusion including the defect detection by multi-model voting method (DDMV) and the defect detection by multi-model learning method (DDML). With the purpose of reducing wrong and missing detection, the DDMV and DDML integrate multiple defect detection networks with different fusion strategies. The effectiveness and accuracy of the proposed framework are verified with extensive experiments on two open-source PCB datasets. The experimental results demonstrate that the proposed DDMV and DDML are better than any other individual state-of-the-art PCB defect detection model in F1-score, and the area under curve value of DDML is also higher than that of any other individual detection model. Furthermore, compared with DDMV, the DDML with an automatic machine learning method achieves the best performance in PCB defect detection, and the F1-score on the two datasets can reach 99.7% and 95.6% respectively.

Cite this article

WU Xingl(武星), ZHANG Qingfeng(张庆丰), WANG Jianjia(王健嘉), YAO Junfeng(姚骏峰), Guo Yike.(郭毅可) . Multiple Detection Model Fusion Framework for Printed Circuit Board Defect Detection[J]. Journal of Shanghai Jiaotong University(Science), 2023 , 28(6) : 717 -727 . DOI: 10.1007/s12204-022-2471-0

References

[1] CHAUDHARY V, DAVE I R, UPLA K P. Automatic visual inspection of printed circuit board for defect detection and classification [C]//2017 International Conference on Wireless Communications, Signal Processing and Networking. Chennai: IEEE, 2017: 732-737.
[2] ZHU J H, WU A, LIU X P. Printed circuit board defect visual detection based on wavelet denoising [J]. IOP Conference Series: Materials Science and Engineering, 2018, 392: 062055.
[3] KUO C F J, FANG T Y, LEE C L, et al. Automated optical inspection system for surface mount device light emitting diodes [J]. Journal of Intelligent Manufacturing, 2019, 30(2): 641-655.
[4] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM, 2017, 60(6): 84-90.
[5] WU X, ZHONG M Y, GUO Y K, et al. The assessment of small bowel motility with attentive deformable neural network [J]. Information Sciences, 2020, 508: 22-32.
[6] LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector [M]//Computer vision -ECCV 2016. Cham: Springer, 2016: 21-37.
[7] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE, 2016: 779- 788.
[8] LIN T Y, DOLL′ AR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 936-944.
[9] JIANG H Z, LEARNED-MILLER E. Face detection with the faster R-CNN [C]//2017 12th IEEE International Conference on Automatic Face & Gesture Recognition. Washington, DC, USA: IEEE, 2017: 650-657.
[10] WANG Y, LUO X B, DING L, et al. Detection based visual tracking with convolutional neural network [J]. Knowledge-Based Systems, 2019, 175: 62-71.
[11] WEI H, YANG C Z, YU Q. Efficient graph-based search for object detection [J]. Information Sciences, 2017, 385/386: 395-414.
[12] BRIA A, MARROCCO C, MOLINARA M, et al. An effective learning strategy for cascaded object detection [J]. Information Sciences, 2016, 340/341: 17-26.
[13] OLSON R S, MOORE J H. TPOT: A tree-based pipeline optimization tool for automating machine learning [M]//Automated machine learning. Cham: Springer, 2019: 151-160.
[14] HOSANG J, BENENSON R, SCHIELE B. Learning non-maximum suppression [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 6469-6477.
[15] DING R W, DAI L H, LI G P, et al. TDD-net: a tiny defect detection network for printed circuit boards [J].CAAI Transactions on Intelligence Technology, 2019, 4(2): 110-116.
[16] TANG S L, HE F, HUANG X L, et al. Online PCB defect detector on a new PCB defect dataset [DB/OL]. (2019-02-17). https://arxiv.org/abs/1902.06197.
[17] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection [C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA: IEEE, 2005: 886-893.
[18] LOWE D G. Distinctive image features from scaleinvariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[19] BAY H, TUYTELAARS T, VAN GOOL L. SURF: speeded up robust features [M]//Computer vision - ECCV 2006. Berlin, Heidelberg: Springer, 2006: 404- 417.
[20] LU Z S, HE Q Q, XIANG X G, et al. Defect detection of PCB based on Bayes feature fusion [J]. The Journal of Engineering, 2018, 2018(16): 1741-1745.
[21] BENEDEK C. Detection of soldering defects in printed circuit boards with hierarchical marked point processes [J]. Pattern Recognition Letters, 2011, 32(13): 1535- 1543.
[22] GAIDHANE V H, HOTE Y V, SINGH V. An efficient similarity measure approach for PCB surface defect detection [J]. Pattern Analysis and Applications, 2018, 21(1): 277-289.
[23] ZHANG C, SHI W, LI X F, et al. Improved bare PCB defect detection approach based on deep feature learning [J]. The Journal of Engineering, 2018, 2018(16): 1415-1420.
[24] DAI W T, MUJEEB A, ERDT M, et al. Soldering defect detection in automatic optical inspection [J]. Advanced Engineering Informatics, 2020, 43: 101004.
[25] BENJDIRA B, KHURSHEED T, KOUBAA A, et al. Car detection using unmanned aerial vehicles: Comparison between faster R-CNN and YOLOv3 [C]//2019 1st International Conference on Unmanned Vehicle Systems-Oman. Muscat: IEEE, 2019: 1-6.
[26] LEI H W, WANG B, WU H H, et al. Defect detection for polymeric polarizer based on faster R-CNN [J]. Journal of Information Hiding and Multimedia Signal Processing, 2018, 9(6): 1414-1420.
[27] LI Y T, HUANG H S, XIE Q S, et al. Research on a surface defect detection algorithm based on MobileNet-SSD [J]. Applied Sciences, 2018, 8(9): 1678.
[28] LI J Y, SU Z F, GENG J H, et al. Real-time detection of steel strip surface defects based on improved YOLO detection network [J]. IFAC-PapersOnLine, 2018, 51(21): 76-81.
[29] HOU W, WEI Y, GUO J, et al. Automatic detection of welding defects using deep neural network [J]. Journal of Physics: Conference Serie, 2017, 933: 012006.
[30] LIN H, LI B, WANG X G, et al. Automated defect inspection of LED chip using deep convolutional neural network [J]. Journal of Intelligent Manufacturing, 2019, 30(6): 2525-2534.
[31] LIN J H, YAO Y, MA L, et al. Detection of a casting defect tracked by deep convolution neural network [J]. The International Journal of Advanced Manufacturing Technology, 2018, 97(1/2/3/4): 573-581.
[32] NASROLLAHI M, BOLOURIAN N, HAMMAD A. Concrete surface defect detection using deep neural network based on lidar scanning [C]//CSCE Annual Conference. Laval: CSCE, 2019: CON032.
[33] MEI S, WANG Y D, WEN G J. Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model [J]. Sensors, 2018, 18(4): 1064.
[34] ADIBHATLA V A, CHIH H C, HSU C C, et al. Defect detection in printed circuit boards using You-onlylook-once convolutional neural networks [J]. Electronics, 2020, 9(9): 1547.
[35] ZHANG X, YANG Y H, HAN Z G, et al. Object class detection [J]. ACM Computing Surveys, 2013, 46(1): 1-53.
[36] SENGUPTA A, YE Y T, WANG R, et al. Going deeper in spiking neural networks: VGG and residual architectures [J]. Frontiers in Neuroscience, 2019, 13: 95.
[37] HENDRY, CHEN R C. Automatic License Plate Recognition via sliding-window darknet-YOLO deep learning [J]. Image and Vision Computing, 2019, 87: 47-56.
Options
Outlines

/

〈 〉