J Shanghai Jiaotong Univ Sci

Journal of Shanghai Jiaotong University(Science) >

2025 , Vol. 30 >Issue 2: 291 - 299

DOI: https://doi.org/10.1007/s12204-023-2608-9

Engieering and Technology

Weld Defect Monitoring Based on Two-Stage Convolutional Neural Network

Expand
  • 1. Key Laboratory of Nondestructive Testing Ministry of Education, Nanchang Hangkong University, Nanchang 330063, China; 2. Key Laboratory of Lightweight and High Strength Structural Materials of Jiangxi Province, School of Advanced Manufacturing, Nanchang University, Nanchang 330031, China; 3. School of Resources and Environment, Nanchang University, Nanchang 330031, China

Accepted date: 2022-04-28

  Online published: 2025-03-21

Fold

Abstract

Zn vapour is easily generated on the surface by fusion welding galvanized steel sheet, resulting in the formation of defects. Rapidly developing computer vision sensing technology collects weld images in the welding process, then obtains laser fringe information through digital image processing, identifies welding defects, and finally realizes online control of weld defects. The performance of a convolutional neural network is related to its structure and the quality of the input image. The acquired original images are labeled with LabelMe, and repeated attempts are made to determine the appropriate filtering and edge detection image preprocessing methods. Two-stage convolutional neural networks with different structures are built on the Tensorflow deep learning framework, different thresholds of intersection over union are set, and deep learning methods are used to evaluate the collected original images and the preprocessed images separately. Compared with the test results, the comprehensive performance of the improved feature pyramid networks algorithm based on the basic network VGG16 is lower than that of the basic network Resnet101. Edge detection of the image will significantly improve the accuracy of the model. Adding blur will reduce the accuracy of the model slightly; however, the overall performance of the improved algorithm is still relatively good, which proves the stability of the algorithm. The self-developed software inspection system can be used for image preprocessing and defect recognition, which can be used to record the number and location of typical defects in continuous welds.

Cite this article

Xiao Wenbo, Xiong Jiakai, Yu Lesheng, He Yinshui, Ma Guohong . Weld Defect Monitoring Based on Two-Stage Convolutional Neural Network[J]. Journal of Shanghai Jiaotong University(Science), 2025 , 30(2) : 291 -299 . DOI: 10.1007/s12204-023-2608-9

References

[1] ZHANG L, LIU S, HAN H, et al. Studies on the formation process and anti-corrosion performance of polypyrrole film deposited on the surface of Q235 steel by an electrochemical method [J]. Surface and Coatings Technology, 2018, 341: 95-102.
[2] SHREYAS P, PANDA B, KUMAR R. Mechanical properties and microstructure of 316L-galvanized steel weld [J]. Materials Today: Proceedings, 2020, 23: 600- 607.
[3] SHARMA A, LEE S J, CHOI D Y, et al. Effect of brazing current and speed on the bead characteristics, microstructure, and mechanical properties of the arc brazed galvanized steel sheets [J]. Journal of Materials Processing Technology, 2017, 249: 212-220.
[4] SATTARPANAH KARGANROUDI S, FEUJOFACK KEMDA V B, BARKA N. A novel method of identifying porosity during laser welding of galvanized steels using microhardness pattern matrix [J]. Manufacturing Letters, 2020, 25: 98-101.
[5] DELGADO M, GUERRERO M, GARZA R. Resistance spot welding of galvanized hsla steels [C]//9th International Conference on Zinc and Zinc Alloy Coated Steel Sheet & 2nd Asia-Pacific Galvanizing Conference. Beijing: Chinese Society for Metals, 2013: 233-236.
[6] MA G, WU C, YE J, et al. Effect of graphene on microstructure and mechanical properties of U-MIGwelded galvanized steel [J]. Journal of Materials Science: Materials in Electronics, 2020, 31(22): 20332- 20344.
[7] XU Y, FANG G, LV N, et al. Computer vision technology for seam tracking in robotic GTAW and GMAW [J]. Robotics and Computer-Integrated Manufacturing, 2015, 32: 25-36.
[8] MAHADEVAN R, JAGAN A, PAVITHRAN L, et al. Intelligent welding by using machine learning techniques [J]. Materials Today: Proceedings, 2021, 46: 7402-7410.
[9] DAVID E, RUMELHART, GEOFFREY E. Learning representations by back-propagating errors [J]. Nature, 1986, 323(6088): 533-536.
[10] SATHIYA P, PANNEERSELVAM K, ABDUL JALEEL M Y. Optimization of laser welding process parameters for super austenitic stainless steel using artificial neural networks and genetic algorithm [J]. Materials & Design (1980-2015 ), 2012, 36: 490-498.
[11] VALAVANIS I, KOSMOPOULOS D. Multiclass defect detection and classification in weld radiographic images using geometric and texture features [J]. Expert Systems With Applications, 2010, 37(12): 7606-7614.
[12] WU Y, GUO D, LIU H, et al. An end-to-end learning method for industrial defect detection [J]. Assembly Automation, 2020, 40(1): 31-39.
[13] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradientbased learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[14] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM, 2017, 60(6): 84-90.
[15] MIAO R, SHAN Z T, ZHOU Q Y, et al. Real-time defect identification of narrow overlap welds and application based on convolutional neural networks [J]. Journal of Manufacturing Systems, 2022, 62: 800-810.
[16] XIA C, PAN Z, FEI Z, et al. Vision based defects detection for Keyhole TIG welding using deep learning with visual explanation [J]. Journal of Manufacturing Processes, 2020, 56: 845-855.
[17] YANG L,WANG H, HUO B, et al. An automatic welding defect location algorithm based on deep learning [J]. NDT & E International, 2021, 120: 102435.
[18] MA G, YU L, YUAN H, et al. A vision-based method for lap weld defects monitoring of galvanized steel sheets using convolutional neural network [J]. Journal of Manufacturing Processes, 2021, 64: 130-139.
[19] CHEN H, CEN Z, WANG C, et al. Image restoration via improved Wiener filter applied to optical sparse aperture systems [J]. Optik, 2017, 147: 350-359.
[20] NNOLIMU A. Automated crack segmentation via saturation channel thresholding, area classification and fusion of modified level set segmentation with Canny edge detection [J]. Heliyon, 2020, 6(12): e05748.
[21] ZHENG Y, IWANA B K, UCHIDA S. Mining the displacement of max-pooling for text recognition [J]. Pattern Recognition, 2019, 93: 558-569.
[22] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation [C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 580-587.
[23] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.
[24] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[25] CHEN K, ZENG Z, YANG J. A deep region-based pyramid neural network for automatic detection and multi-classification of various surface defects of aluminum alloys [J]. Journal of Building Engineering, 2021, 43: 102523.
[26] DAI W, LI D, TANG D, et al. Deep learning assisted vision inspection of resistance spot welds [J]. Journal of Manufacturing Processes, 2021, 62: 262-274.
[27] NANDINI G S, SIVA KUMAR A P, CHIDANANDA K. Dropout technique for image classification based on extreme learning machine [J]. Global Transitions Proceedings, 2021, 2(1): 111-116.

Options
Outlines

/

〈 〉