SpringerLink
Log in

Lightweight Alpha Matting Network Using Distillation-Based Channel Pruning

  • Conference paper
  • First Online:
Computer Vision – ACCV 2022 (ACCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13843))

Included in the following conference series:

  • 755 Accesses

Abstract

Recently, alpha matting has received a lot of attention because of its usefulness in mobile applications such as selfies. Therefore, there has been a demand for a lightweight alpha matting model due to the limited computational resources of commercial portable devices. To this end, we suggest a distillation-based channel pruning method for the alpha matting networks. In the pruning step, we remove channels of a student network having fewer impacts on mimicking the knowledge of a teacher network. Then, the pruned lightweight student network is trained by the same distillation loss. A lightweight alpha matting model from the proposed method outperforms existing lightweight methods. To show superiority of our algorithm, we provide various quantitative and qualitative experiments with in-depth analyses. Furthermore, we demonstrate the versatility of the proposed distillation-based channel pruning method by applying it to semantic segmentation.

Project page is at https://github.com/DongGeun-Yoon/DCP.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Aksoy, Y., Aydin, T.O., Pollefeys, M.: Designing effective inter-pixel information flow for natural image matting. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  2. Cai, S., et al.: Disentangled image matting. In: Proceedings of International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  3. Changpinyo, S., Sandler, M., Zhmoginov, A.: The power of sparsity in convolutional neural networks. Ar**v abs/1702.06257 (2017)

    Google Scholar 

  4. Chen, Q., Li, D., Tang, C.K.: KNN matting. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2012)

    Google Scholar 

  5. Chen, X., Zou, D., Zhou, S.Z., Zhao, Q., Tan, P.: Image matting with local and nonlocal smooth priors. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2013)

    Google Scholar 

  6. Chen, X., Wang, Y., Zhang, Y., Du, P., Xu, C., Xu, C.: Multi-task pruning for semantic segmentation networks. Ar**v abs/2007.08386 (2020)

    Google Scholar 

  7. Cho, D., Kim, S., Tai, Y.-W.: Consistent matting for light field images. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 90–104. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_7

    Chapter  Google Scholar 

  8. Cho, D., Kim, S., Tai, Y.W., Kweon, I.S.: Automatic trimap generation and consistent matting for light-field images. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 39(8), 1504–1517 (2016)

    Article  Google Scholar 

  9. Cho, D., Tai, Y.W., Kweon, I.S.: Deep convolutional neural network for natural image matting using initial alpha mattes. IEEE Trans. Image Process. (TIP) 28(3), 1054–1067 (2018)

    Article  MathSciNet  Google Scholar 

  10. Cho, D., Tai, Y.-W., Kweon, I.: Natural image matting using deep convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 626–643. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_39

    Chapter  Google Scholar 

  11. Choi, I., Lee, M., Tai, Y.-W.: Video matting using multi-frame nonlocal matting laplacian. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 540–553. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_39

    Chapter  Google Scholar 

  12. Chuang, Y.Y., Curless, B., Salesin, D.H., Szeliski, R.: A Bayesian approach to digital matting. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2001)

    Google Scholar 

  13. Everingham, M., Eslami, S.M.A., Gool, L.V., Williams, C.K.I., Winn, J.M., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vision 111, 98–136 (2014)

    Article  Google Scholar 

  14. Frankle, J., Carbin, M.: The lottery ticket hypothesis: finding sparse, trainable neural networks. In: International Conference on Learning Representations (2019)

    Google Scholar 

  15. Gao, X., Zhao, Y., Łukasz Dudziak, Mullins, R., zhong Xu, C.: Dynamic channel pruning: feature boosting and suppression. In: International Conference on Learning Representations (2019)

    Google Scholar 

  16. Gastal, E.S.L., Oliveira, M.M.: Shared sampling for real-time alpha matting. In: Eurographics (2010)

    Google Scholar 

  17. Ge, S., Zhao, S., Li, C., Li, J.: Low-resolution face recognition in the wild via selective knowledge distillation. IEEE Trans. Image Process. (TIP) 28(4), 2051–2062 (2019)

    Article  MathSciNet  Google Scholar 

  18. Han, S., Pool, J., Tran, J., Dally, W.J.: Learning both weights and connections for efficient neural networks. In: Proceedings of Neural Information Processing Systems (NeurIPS), pp. 1135–1143 (2015)

    Google Scholar 

  19. He, K., Rhemann, C., Rother, C., Tang, X., Sun, J.: A global sampling method for alpha matting. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2011)

    Google Scholar 

  20. He, K., Sun, J., Tang, X.: Fast matting using large kernel matting laplacian matrices. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2010)

    Google Scholar 

  21. He, W., Wu, M., Liang, M., Lam, S.K.: CAP: context-aware pruning for semantic segmentation. In: Proceedings of Winter Conference on Applications of Computer Vision (WACV), pp. 960–969 (2021)

    Google Scholar 

  22. Heo, B., Kim, J., Yun, S., Park, H., Kwak, N., Choi, J.Y.: A comprehensive overhaul of feature distillation. In: Proceedings of International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  23. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. In: Proceedings of Neural Information Processing Systems Workshops (NeurIPSW) (2015)

    Google Scholar 

  24. Huang, Z., Wang, N.: Like what you like: knowledge distill via neuron selectivity transfer. Ar**v abs/1707.01219 (2017)

    Google Scholar 

  25. Karacan, L., Erdem, A., Erdem, E.: Image matting with kl-divergence based sparse sampling. In: Proceedings of International Conference on Computer Vision (ICCV) (2015)

    Google Scholar 

  26. Ko, S., Park, J., Chae, B., Cho, D.: Learning lightweight low-light enhancement network using pseudo well-exposed images. IEEE Signal Process. Lett. (SPL) 29, 289–293 (2022)

    Article  Google Scholar 

  27. Lee, P., Wu, Y.: Nonlocal matting. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2011)

    Google Scholar 

  28. Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 30(2), 0162–8828 (2008)

    Google Scholar 

  29. Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. In: International Conference on Learning Representation (ICLR) (2017)

    Google Scholar 

  30. Li, Y., Lu, H.: Natural image matting via guided contextual attention. In: Association for the Advancement of Artificial Intelligence (AAAI) (2020)

    Google Scholar 

  31. Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., Zhang, C.: Learning efficient convolutional networks through network slimming. In: Proceedings of International Conference on Computer Vision (ICCV), pp. 2736–2744 (2017)

    Google Scholar 

  32. Liu, Z., Sun, M., Zhou, T., Huang, G., Darrell, T.: Rethinking the value of network pruning. In: ICLR (2019)

    Google Scholar 

  33. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2015)

    Google Scholar 

  34. Lu, H., Dai, Y., Shen, C., Xu, S.: Indices matter: learning to index for deep image matting. In: Proceedings of International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  35. Lutz, S., Amplianitis, K., Smolic, A.: AlphaGAN: generative adversarial networks for natural image matting. In: British Machine Vision Conference (BMVC) (2018)

    Google Scholar 

  36. Molchanov, D., Ashukha, A., Vetrov, D.: Variational dropout sparsifies deep neural networks. In: Proceedings of International Conference on Machine Learning (ICML), pp. 2498–2507 (2017)

    Google Scholar 

  37. Qiao, Y., et al.: Attention-guided hierarchical structure aggregation for image matting. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  38. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  39. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetv 2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Google Scholar 

  40. Shahrian, E., Rajan, D., Price, B., Cohen, S.: Improving image matting using comprehensive sampling sets. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2013)

    Google Scholar 

  41. Shahrian, E., Rajan, D.: Weighted color and texture sample selection for image matting. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2012)

    Google Scholar 

  42. Shen, X., Tao, X., Gao, H., Zhou, C., Jia, J.: Deep automatic portrait matting. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 92–107. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_6

    Chapter  Google Scholar 

  43. Sun, J., Jia, J., Tang, C.K., Shum, H.Y.: Poisson matting. ACM Trans. Graph. (ToG) 23(3), 315–321 (2004)

    Article  Google Scholar 

  44. Sun, Y., Tang, C.K., Tai, Y.W.: Semantic image matting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11120–11129 (2021)

    Google Scholar 

  45. Tanaka, H., Kunin, D., Yamins, D.L., Ganguli, S.: Pruning neural networks without any data by iteratively conserving synaptic flow. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 6377–6389. Curran Associates, Inc. (2020)

    Google Scholar 

  46. Tang, J., Aksoy, Y., Oztireli, C., Gross, M., Aydin, T.O.: Learning-based sampling for natural image matting. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  47. Tung, F., Mori, G.: Similarity-preserving knowledge distillation. In: Proceedings of International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  48. Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2007)

    Google Scholar 

  49. Wen, W., Wu, C., Wang, Y., Chen, Y., Li, H.: Learning structured sparsity in deep neural networks. In: Proceedings of Neural Information Processing Systems (NeurIPS) (2016)

    Google Scholar 

  50. Xu, N., Price, B.L., Cohen, S., Huang, T.S.: Deep image matting. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  51. Yim, J., Joo, D., Bae, J., Kim, J.: A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  52. Yoon, D., Park, J., Cho, D.: Lightweight deep CNN for natural image matting via similarity-preserving knowledge distillation. IEEE Signal Process. Lett. 27, 2139–2143 (2020)

    Article  Google Scholar 

  53. Yu, Q., et al.: Mask guided matting via progressive refinement network. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 1154–1163 (2021)

    Google Scholar 

  54. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  55. Zheng, Y., Kambhamettu, C.: Learning based digital matting. In: Proceedings of International Conference on Computer Vision (ICCV) (2009)

    Google Scholar 

Download references

Acknowledgements

This work was partly supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2022-00155857, Artificial Intelligence Convergence Innovation Human Resources Development (Chungnam National University)) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT, No. 2021R1A4A1032580 and No. 2022R1C1C1009334).

Author information

Authors and Affiliations

  1. Chungnam National University, Daejeon, Republic of Korea

    Donggeun Yoon & Donghyeon Cho

  2. Pusan National University, Busan, Republic of Korea

    **sun Park

Authors
  1. Donggeun Yoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. **sun Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Donghyeon Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Donghyeon Cho .

Editor information

Editors and Affiliations

  1. University of Wollongong, Wollongong, NSW, Australia

    Lei Wang

  2. University of Bonn, Bonn, Germany

    Juergen Gall

  3. University of Adelaide, Adelaide, SA, Australia

    Tat-Jun Chin

  4. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  5. Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 70530 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yoon, D., Park, J., Cho, D. (2023). Lightweight Alpha Matting Network Using Distillation-Based Channel Pruning. In: Wang, L., Gall, J., Chin, TJ., Sato, I., Chellappa, R. (eds) Computer Vision – ACCV 2022. ACCV 2022. Lecture Notes in Computer Science, vol 13843. Springer, Cham. https://doi.org/10.1007/978-3-031-26313-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26313-2_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26312-5

  • Online ISBN: 978-3-031-26313-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation