SpringerLink
Log in

Certifiable Prioritization for Deep Neural Networks via Movement Cost in Feature Space

  • Chapter
  • First Online:
Attacks, Defenses and Testing for Deep Learning

  • 82 Accesses

Abstract

Although deep neural networks (DNNs) have shown superior performance in different software systems, they also display malfunctioning and can even lead to irreversible catastrophes. Hence, it is significant to detect the misbehavior of DNN-based software and enhance the quality of DNNs. Test input prioritization is a highly effective approach to ensure the quality of DNNs. This method involves prioritizing test inputs in such a way that inputs that are more likely to reveal bugs or issues are identified early on, even with limited time and manual labeling efforts. Nevertheless, current prioritization methods still have limitations in three aspects: certifiability, effectiveness, and generalizability. To overcome the challenges, we propose a test input prioritization technique designed based on a movement cost perspective of test inputs in DNNs’ feature space. Our method differs from previous works in three key aspects: (1) certifiable—it provides a formal robustness guarantee for the movement cost; (2) effective—it leverages formally guaranteed movement costs to identify malicious bug-revealing inputs; and (3) generic—it can be applied to various tasks, data, models, and scenarios. Extensive evaluations across two tasks (i.e., classification and regression), six data forms, four model structures, and two scenarios (i.e., white box and black box) demonstrate our method’s superior performance. For instance, it significantly improves 53.97% prioritization effectiveness on average compared with baselines. Its robustness and generalizability are 1.41\(\sim \)2.00 times and 1.33\(\sim \)3.39 times that of baselines on average, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. Ma, L., Juefei-Xu, F., Zhang, F., Sun, J., Xue, M., Li, B., Chen, C., Su, T., Li, L., Liu, Y.: Deepgauge: Multi-granularity testing criteria for deep learning systems. In: 33rd ACM/IEEE International Conference on Automated Software Engineering, pp. 120–131. ACM, Montpellier (2018)

    Google Scholar 

  2. Pei, K., Cao, Y., Yang, J., Jana, S.: Deepxplore: automated whitebox testing of deep learning systems. Commun. ACM 62(11), 137–145 (2019)

    Article  Google Scholar 

  3. Wicker, M., Huang, X., Kwiatkowska, M.: Feature-guided black-box safety testing of deep neural networks. In: Tools and Algorithms for the Construction and Analysis of Systems - 24th International Conference. TACAS 2018, vol. 10805, pp. 408–426. Springer, Thessaloniki (2018)

    Google Scholar 

  4. Zhang, K., Zhang, Y., Zhang, L., Gao, H., Yan, R., Yan, J.: Neuron activation frequency based test case prioritization. In: International Symposium on Theoretical Aspects of Software Engineering, pp. 81–88. IEEE, Hangzhou (2020)

    Google Scholar 

  5. Byun, T., Sharma, V., Vijayakumar, A., Rayadurgam, S., Cofer, D.D.: Input prioritization for testing neural networks. In: IEEE International Conference On Artificial Intelligence Testing. AITest 2019, pp. 63–70. IEEE, Newark (2019)

    Google Scholar 

  6. Kim, J., Feldt, R., Yoo, S.: Guiding deep learning system testing using surprise adequacy. In: 41st IEEE/ACM International Conference on Software Engineering, pp. 1039–1049. IEEE/ACM, Montreal, QC, Canada (2019)

    Google Scholar 

  7. Feng, Y., Shi, Q., Gao, X., Wan, J., Fang, C., Chen, Z.: Deepgini: prioritizing massive tests to enhance the robustness of deep neural networks. In: 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 177–188. ACM, Virtual Event, USA (2020)

    Google Scholar 

  8. Shen, W., Li, Y., Chen, L., Han, Y., Zhou, Y., Xu, B.: Multiple-boundary clustering and prioritization to promote neural network retraining. In: 35th IEEE/ACM International Conference on Automated Software Engineering, pp. 410–422. IEEE, Melbourne, Australia (2020)

    Google Scholar 

  9. Zhang, L., Sun, X., Li, Y., Zhang, Z.: A noise-sensitivity-analysis-based test prioritization technique for deep neural networks. CoRR pp. 1–8 (2019)

    Google Scholar 

  10. Wang, Z., You, H., Chen, J., Zhang, Y., Dong, X., Zhang, W.: Prioritizing test inputs for deep neural networks via mutation analysis. In: 43rd IEEE/ACM International Conference on Software Engineering, pp. 397–409. IEEE, Madrid, Spain (2021)

    Google Scholar 

  11. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: 3rd International Conference on Learning Representations, pp. 1–11. OpenReview.net, San Diego, CA, USA (2014)

    Google Scholar 

  12. Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., Li, J.: Boosting adversarial attacks with momentum. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 9185–9193. Computer Vision Foundation/IEEE Computer Society, Salt Lake City, UT, USA (2018)

    Google Scholar 

  13. Chen, J., Zheng, H., **ong, H., Shen, S., Su, M.: Mag-gan: Massive attack generator via gan. Inf. Sci. 536(1), 67–90 (2020)

    Article  MathSciNet  Google Scholar 

  14. Moosavi-Dezfooli, S.M., Fawzi, A., Frossard, P.: Deepfool: a simple and accurate method to fool deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2574–2582. IEEE Computer Society, Las Vegas, NV, USA (2016)

    Google Scholar 

  15. Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019)

    Article  Google Scholar 

  16. Sun, B., Tsai, N.H., Liu, F., Yu, R., Su, H.: Adversarial defense by stratified convolutional sparse coding. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 11447–11456. Computer Vision Foundation/IEEE, Long Beach, CA, USA (2019)

    Google Scholar 

  17. Mustafa, A., Khan, S., Hayat, M., Goecke, R., Shen, J., Shao, L.: Adversarial defense by restricting the hidden space of deep neural networks. In: IEEE International Conference on Computer Vision, pp. 3384–3393. IEEE, Seoul, South Korea (2019)

    Google Scholar 

  18. Rakin, A.S., Zhezhi, H., Deliang, F.: Parametric noise injection: trainable randomness to improve deep neural network robustness against adversarial attack. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 588–597. Computer Vision Foundation/IEEE, Long Beach, CA, USA (2019)

    Google Scholar 

  19. Li, Y., Hua, J., Wang, H., Chen, C., Liu, Y.: Deeppayload: Black-box backdoor attack on deep learning models through neural payload injection. In: 43rd IEEE/ACM International Conference on Software Engineering (ICSE 2021), pp. 263–274. IEEE, Madrid, Spain (2021)

    Google Scholar 

  20. Alshemali, B., Kalita, J.: Improving the reliability of deep neural networks in nlp: a review. Knowl.-Based Syst. 191(1), 1–19 (2020)

    Google Scholar 

  21. Bellamy, R.K.E., Dey, K., Hind, M., Hoffman, S.C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., Nagar, S., Ramamurthy, K.N., Richards, J.T., Saha, D., Sattigeri, P., Singh, M., Varshney, K.R., Zhang, Y.: AI fairness 360: an extensible toolkit for detecting and mitigating algorithmic bias. IBM J. Res. Dev. 63(4/5), 1–15 (2019)

    Google Scholar 

  22. Zhang, Z., Cui, P., Zhu, W.: Deep learning on graphs: A survey. IEEE Trans. Knowl. Data Eng. 34(1), 249–270 (2020)

    Article  Google Scholar 

  23. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  24. Zheng, H., Chen, J., Du, H., Zhu, W., Ji, S., Zhang, X.: Grip-gan: An attack-free defense through general robust inverse perturbation. IEEE Trans. Depend. Secure Comput. 1–18 (2021)

    Google Scholar 

  25. Paulavičius, R., Žilinskas, J.: Analysis of different norms and corresponding lipschitz constants for global optimization. Technol. Econ. Dev. Econ. 12(4), 301–306 (2006)

    Article  Google Scholar 

  26. Gnedenko, B.: Sur la distribution limite du terme maximum d’une serie aleatoire. Ann. Math. 44(3), 423–453 (1943)

    Article  MathSciNet  Google Scholar 

  27. Ma, W., Papadakis, M., Tsakmalis, A., Cordy, M., Traon, Y.L.: Test selection for deep learning systems. ACM Trans. Softw. Engin. Methodol. (TOSEM) 30(2), 1–22 (2021)

    Article  Google Scholar 

  28. **e, X., Yin, P., Chen, S.: Boosting the revealing of detected violations in deep learning testing: a diversity-guided method. In: 37th IEEE/ACM International Conference on Automated Software Engineering, ASE 2022, Rochester, MI, USA, October 10–14, 2022, pp. 17:1–17:13. ACM (2022)

    Google Scholar 

  29. **e, X., Li, T., Wang, J., Ma, L., Guo, Q., Juefei-Xu, F., Liu, Y.: NPC: neuron path coverage via characterizing decision logic of deep neural networks. ACM Trans. Softw. Eng. Methodol. 31(3), 47:1–47:27 (2022)

    Google Scholar 

  30. Lou, Y., Hao, D., Zhang, L.: Mutation-based test-case prioritization in software evolution. In: 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), pp. 46–57. IEEE Computer Society, Gaithersbury, MD, USA (2015)

    Google Scholar 

  31. Shin, D., Yoo, S., Papadakis, M., Bae, D.H.: Empirical evaluation of mutation-based test case prioritization techniques. Softw. Testing, Verific. Reliab. 29(1–2), 1–2 (2019)

    Google Scholar 

  32. Denker, J.S., LeCun, Y.: Transforming neural-net output levels to probability distributions. In: Advances in Neural Information Processing Systems 3 (NIPS 1990), pp. 853–859. Morgan Kaufmann, Denver, Colorado, USA (1990)

    Google Scholar 

  33. Eckle, K., Schmidt-Hieber, J.: A comparison of deep networks with relu activation function and linear spline-type methods. Neural Netw. 110, 232–242 (2019)

    Article  Google Scholar 

  34. Weng, T., Zhang, H., Chen, P., Yi, J., Su, D., Gao, Y., Hsieh, C., Daniel, L.: Evaluating the robustness of neural networks: An extreme value theory approach. In: 6th International Conference on Learning Representations (ICLR 2018), pp. 1–18. OpenReview.net, Vancouver, BC, Canada (2018)

    Google Scholar 

  35. Chen, J., Zheng, H., Shangguan, W., Liu, L., Ji, S.: Act-detector: Adaptive channel transformation-based light-weighted detector for adversarial attacks. Inf. Sci. 564, 163–192 (2021)

    Article  MathSciNet  Google Scholar 

  36. LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)

    Article  Google Scholar 

  37. Van der Maaten, L., Hinton, G.: Visualizing data using t-sne. J. Mach. Learn. Res. 9(11), 2579–2605 (2008)

    Google Scholar 

  38. Amari, S.I.: Backpropagation and stochastic gradient descent method. Neurocomputing 5(4), 185–196 (1993)

    Article  Google Scholar 

  39. Chen, P., Zhang, H., Sharma, Y., Yi, J., Hsieh, C.: ZOO: zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, AISec@CCS 2017, pp. 15–26. ACM, Dallas, TX, USA (2017)

    Google Scholar 

  40. Krizhevsky, A.: Learning multiple layers of features from tiny images. University of Toronto, Technical report, Computer Science Department (2009)

    Google Scholar 

  41. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  42. Deng, Y., Zheng, J.X., Zhang, T., Chen, C., Lou, G., Kim, M.: An analysis of adversarial attacks and defenses on autonomous driving models. In: IEEE International Conference on Pervasive Computing and Communications (PerCom 2020), pp. 1–10. IEEE, Austin, TX, USA (2020)

    Google Scholar 

  43. **ao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. Ar**v Preprint pp. 1–6 (2017)

    Google Scholar 

  44. Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142–150. Association for Computational Linguistics, Portland, Oregon, USA (2011)

    Google Scholar 

  45. O’Shea, T.J., West, N.: Radio machine learning dataset generation with gnu radio. In: Proceedings of the GNU Radio Conference, pp. 1–10 (2016)

    Google Scholar 

  46. McCallum, A., Nigam, K., Rennie, J., Seymore, K.: Automating the construction of internet portals with machine learning. Inf. Retr. 3(2), 127–163 (2000)

    Article  Google Scholar 

  47. Kohavi, R.: Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96), Portland, Oregon, USA, pp. 202–207. AAAI Press, Menlo Park, CA (1996)

    Google Scholar 

  48. Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial examples in the physical world. In: 5th International Conference on Learning Representations (ICLR 2017), pp. 1–14. OpenReview.net, Toulon, France (2017)

    Google Scholar 

  49. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: IEEE Symposium on Security and Privacy (SP 2017), pp. 39–57. IEEE Computer Society, San Jose, CA, USA (2017)

    Google Scholar 

  50. Chen, J., Zheng, H., **ong, H., Chen, R., Du, T., Hong, Z., Ji, S.: Finefool: A novel DNN object contour attack on image recognition based on the attention perturbation adversarial technique. Comput. Secur. 104, 102220 (2021)

    Article  Google Scholar 

  51. Hosseini, H., Poovendran, R.: Semantic adversarial examples. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops. CVPR Workshops 2018, pp. 1614–1619. Computer Vision Foundation/IEEE Computer Society, Salt Lake City, UT, USA (2018)

    Google Scholar 

  52. Chen, J., Zhang, L., Zheng, H., Wang, X., Ming, Z.: Deeppoison: feature transfer based stealthy poisoning attack for dnns. IEEE Trans. Circuits Syst. II Express Briefs 68(7), 2618–2622 (2021)

    Google Scholar 

  53. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning (ICML 2015). vol. 37, pp. 448–456. JMLR.org, Lille, France (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Cyberspace Security, College of Information Engineering, Zhejiang University of Technology, Hangzhou, Zhejiang, China

    **yin Chen & Haibin Zheng

  2. College of Information Engineering, Zhejiang University of Technology, Hangzhou, Zhejiang, China

    **min Zhang

Authors
  1. **yin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. **min Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haibin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to **yin Chen .

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Chen, J., Zhang, X., Zheng, H. (2024). Certifiable Prioritization for Deep Neural Networks via Movement Cost in Feature Space. In: Attacks, Defenses and Testing for Deep Learning. Springer, Singapore. https://doi.org/10.1007/978-981-97-0425-5_18

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-0425-5_18

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-0424-8

  • Online ISBN: 978-981-97-0425-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation