Abstract
Deep learning-based methods have recently shown great promise in the defect detection task. However, current methods rely on large-scale annotated data and are unable to adapt a trained deep learning model to new samples that were not observed during training. To address this issue, we propose a new siamese defect-aware attention network (SDANet) with a template comparison detection strategy that improves the defect detection technique for matching new samples without rapidly collecting new data and retraining the model. In SDANet, the siamese feature pyramid network is used to extract multi-scale features from input and template images, the defect-aware attention module is proposed to obtain inconsistency between input and template features and use it to enhance abnormality in input image features, and the self-calibration module is developed to calibrate the alignment error between the input and template features. SDANet can be used as a plug-in module to enable most existing mainstream detection algorithms to detect defects using not only the features of defects, but also the inconsistency between features of the inspected image and the template image. Extensive experiments on two publicly available industrial defect detection benchmarks highlight the effectiveness of our method. SDANet can be seamlessly integrated into mainstream detection methods and improve the mAP of mainstream detection algorithms on unseen samples by 12% on average which outperforms current state-of-the-art method by 7.7%. It can also improve the performance in seen samples by 4.3% on average. SDANet can be used in general defect detection applications of industrial manufacturing.
Similar content being viewed by others
Notes
These datasets are available on the https://tianchi.aliyun.com/dataset/
References
Jia L, Chen C, Liang J, Hou Z (2017) Fabric defect inspection based on lattice segmentation and gabor filtering. Neurocomputing 238:84–102
Yapi D, Allili M S, Baaziz N (2017) Automatic fabric defect detection using learning-based local textural distributions in the contourlet domain. IEEE Trans Autom Sci Eng 15(3):1014–1026
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788
Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28:91–99
Cheng X, Yu J (2020) Retinanet with difference channel attention and adaptively spatial feature fusion for steel surface defect detection. IEEE Trans Instrum Meas 70:1–11
Cai Z, Vasconcelos N (2018) Cascade r-cnn: Delving into high quality object detection. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 6154–6162
Qiao S, Chen L-C, Yuille A (2021) Detectors: Detecting objects with recursive feature pyramid and switchable atrous convolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 10213–10224
Wu J, Le J, **ao Z, Zhang F, Geng L, Liu Y, Wang W (2021) Automatic fabric defect detection using a wide-and-light network. Appl Intell 51(7):4945–4961
Zheng X, Chen J, Wang H, Zheng S, Kong Y (2021) A deep learning-based approach for the automated surface inspection of copper clad laminate images. Appl Intell 51(3):1262–1279
Tian R, Shi H, Guo B, Zhu L (2021) Multi-scale object detection for high-speed railway clearance intrusion. Appl Intell, pp 1–16
Dai Z, Yi J, Zhang Y, Zhou B, He L (2020) Fast and accurate cable detection using cnn. Appl Intell 50(12):4688–4707
Li Y, Li J (2021) An end-to-end defect detection method for mobile phone light guide plate via multitask learning. IEEE Trans Instrum Meas 70:1–13
Czimmermann T, Ciuti G, Milazzo M, Chiurazzi M, Roccella S, Oddo C. M, Dario P (2020) Visual-based defect detection and classification approaches for industrial applications—a survey. Sensors 20(5):1459
Wang J, Ma Y, Zhang L, Gao R X, Wu D (2018) Deep learning for smart manufacturing: Methods and applications. J Manuf Syst 48:144–156
Božič J, Tabernik D, Skočaj D (2021) Mixed supervision for surface-defect detection: From weakly to fully supervised learning. Comput Ind 103459:129
Ren R, Hung T, Tan KC (2017) A generic deep-learning-based approach for automated surface inspection. IEEE Trans Cybern 48(3):929–940
Lin X, Wang X, Li L (2020) Intelligent detection of edge inconsistency for mechanical workpiece by machine vision with deep learning and variable geometry model. Appl Intell 50(7):2105–2119
Okaro IA, Jayasinghe S, Sutcliffe C, Black K, Paoletti P, Green PL (2019) Automatic fault detection for laser powder-bed fusion using semi-supervised machine learning. Addit Manuf 27:42–53
Qi G-J, Luo J (2020) Small data challenges in big data era: A survey of recent progress on unsupervised and semi-supervised methods. IEEE Transactions on Pattern Analysis and Machine Intelligence
Pal SK, Pramanik A, Maiti J, Mitra P (2021) Deep learning in multi-object detection and tracking: state of the art. Appl Intell 51(9):6400–6429
Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271
Farhadi A, Redmon J (2018) Yolov3: An incremental improvement. In: Computer vision and pattern recognition, vol 1804. Springer, Berlin/Heidelberg, Germany
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. In: European conference on computer vision, Springer, pp 21–37
Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988
Bochkovskiy A, Wang C-Y, Liao H-Y M (2020) Yolov4: Optimal speed and accuracy of object detection. ar**v:2004.10934
Tabernik D, Šela S, Skvarč J, Skočaj D (2020) Segmentation-based deep-learning approach for surface-defect detection. J Intell Manuf 31(3):759–776
Wang Y, Liu M, Zheng P, Yang H, Zou J (2020) A smart surface inspection system using faster r-cnn in cloud-edge computing environment. Adv Eng Inform 101037:43
Cui L, Jiang X, Xu M, Li W, Lv P, Zhou B (2021) Sddnet: A fast and accurate network for surface defect detection. IEEE Trans Instrum Meas 70:1–13
Luo J, Yang Z, Li S, Wu Y (2021) Fpcb surface defect detection: A decoupled two-stage object detection framework. IEEE Trans Instrum Meas 70:1–11
Li Y, Huang H, **e Q, Yao L, Chen Q (2018) Research on a surface defect detection algorithm based on mobilenet-ssd. Appl Sci 8(9):1678
Liu K, Li Y, Yang J, Liu Y, Yao Y (2020) Generative principal component thermography for enhanced defect detection and analysis. IEEE Trans Instrum Meas 69(10):8261–8269
Yin X, Chen Y, Bouferguene A, Zaman H, Al-Hussein M, Kurach L (2020) A deep learning-based framework for an automated defect detection system for sewer pipes. Autom Construct 109:102967
**g J, Zhuo D, Zhang H, Liang Y, Zheng M (2020) Fabric defect detection using the improved yolov3 model. J Eng Fibers Fabr 15:1558925020908268
Tulbure A-A, Tulbure A-A, Dulf E-H (2022) A review on modern defect detection models using dcnns–deep convolutional neural networks. J Adv Res 35:33–48
Dong H, Song K, He Y, Xu J, Yan Y, Meng Q (2019) Pga-net: Pyramid feature fusion and global context attention network for automated surface defect detection. IEEE Trans Industr Inform 16(12):7448–7458
Tu Y, Ling Z, Guo S, Wen H (2020) An accurate and real-time surface defects detection method for sawn lumber. IEEE Trans Instrum Meas 70:1–11
Li F, ** Q (2021) Defectnet: Toward fast and effective defect detection. IEEE Trans Instrum Meas 70:1–9
Li D, Li Y, **e Q, Wu Y, Yu Z, Wang J (2021) Tiny defect detection in high-resolution aero-engine blade images via a coarse-to-fine framework. IEEE Trans Instrum Meas 70:1–12
Huang Y, **g J, Wang Z (2021) Fabric defect segmentation method based on deep learning. IEEE Trans Instrum Meas 70:1–15
**ng J, Jia M (2021) A convolutional neural network-based method for workpiece surface defect detection. Measurement 176:109185
Xu Y, Li D, **e Q, Wu Q, Wang J (2021) Automatic defect detection and segmentation of tunnel surface using modified mask r-cnn. Measurement 178:109316
Zhang J, Su H, Zou W, Gong X, Zhang Z, Shen F (2021) Cadn: a weakly supervised learning-based category-aware object detection network for surface defect detection. Pattern Recogn 109:107571
Zhang G, Pan Y, Zhang L (2021) Semi-supervised learning with gan for automatic defect detection from images. Autom Constr 128:103764
Kang B, Liu Z, Wang X, Yu F, Feng J, Darrell T (2019) Few-shot object detection via feature reweighting. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 8420–8429
Wang X, Huang T, Gonzalez J, Darrell T, Yu F (2020) Frustratingly simple few-shot object detection. In: International conference on machine learning, pp 9919–9928. PMLR
Zheng Y, Wu J, Qin Y, Zhang F, Cui L (2021) Zero-shot instance segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2593–2602
Wang Q, Zhang X, Chen G, Dai F, Gong Y, Zhu K (2018) Change detection based on faster r-cnn for high-resolution remote sensing images. Remote Sens Lett 9(10):923–932
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125
Tianchi: Smart Diagnosis of Cloth Flaw Dataset (2020). https://tianchi.aliyun.com/dataset/dataDetail?dataId=79336. Accessed 2021-04
Tianchi: Smart Diagnosis of Tile Flaw Dataset (2020). https://tianchi.aliyun.com/dataset/dataDetail?dataId=110088. Accessed 2021-04
Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8026–8037
Acknowledgments
The authors would like to acknowledge financial support from the National Natural Science Foundation of China (NSFC) under Grant No. 61672498.
Funding
The National Natural Science Foundation of China (NSFC) under Grant No. 61672498.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest/ Competing interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zheng, Y., Cui, L. Defect detection on new samples with siamese defect-aware attention network. Appl Intell 53, 4563–4578 (2023). https://doi.org/10.1007/s10489-022-03595-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-022-03595-0