SpringerLink
Log in

HSSAN: hair synthesis with style-guided spatially adaptive normalization on generative adversarial network

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Hair synthesis plays a crucial role in generating facial images, but the complex textures and varied shapes of hair create obstacles in creating genuine images of hair on photographs utilizing generative adversarial networks. This research paper proposes an inventive normalization technique, HSSAN (Hair Style-Guided Spatially Adaptive Normalization), that incorporates four connected phases, each set exclusively for hair feature attributes, and uses them to improve the generator to generate hairstyle transfer images. The hair synthesizer generator utilizes several HSSAN residual blocks in the network framework, while the input modules comprise only an appearance module and a background module. Furthermore, a regularized loss function is introduced to regulate the style vector. Through the network, realistic hair generation images can be generated. We employed the FFHQ dataset to perform our experiments and observed that our methodology generates hair images surpassing existing generative adversarial network-based methods in terms of visual realism and Fréchet Inception Distance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Lin, C., **ong, S., Lu, X.: Disentangled face editing via individual walk in personalized facial semantic field. Vis. Comput. 1–10 (2022)

  2. Chi, J., Gao, S., Zhang, C.: Interactive facial expression editing based on spatio-temporal coherency. Vis. Comput. 33, 981–991 (2017)

    Article  Google Scholar 

  3. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Adv. Neural Inform. Process. Syst. 27,(2014)

  4. Abdal, R., Qin, Y., Wonka, P.: Image2stylegan: How to embed images into the stylegan latent space? In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4432–4441 (2019)

  5. Park, T., Liu, M.-Y., Wang, T.-C., Zhu, J.-Y.: Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2337–2346 (2019)

  6. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)

  7. Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8110–8119 (2020)

  8. Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1501–1510 (2017)

  9. Qiu, H., Wang, C., Zhu, H., Zhu, X., Gu, J., Han, X.: Two-phase hair image synthesis by self-enhancing generative model. In: Computer Graphics Forum. Wiley Online Library, vol. 38, pp. 403–412 (2019)

  10. **ao, C., Yu, D., Han, X., Zheng, Y., Fu, H.: Sketchhairsalon: deep sketch-based hair image synthesis. (2021) ar**v preprint ar**v:2109.07874

  11. Zhu, P., Abdal, R., Qin, Y., Wonka, P.: Sean: Image synthesis with semantic region-adaptive normalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5104–5113 (2020)

  12. Tan, Z., Chai, M., Chen, D., Liao, J., Chu, Q., Yuan, L., Tulyakov, S., Yu, N.: Michigan: multi-input-conditioned hair image generation for portrait editing. (2020) ar**v preprint ar**v:2010.16417

  13. Wang, H., **an, M., Vakanski, A., Shareef, B.: Sian: style-guided instance-adaptive normalization for multi-organ histopathology image synthesis. (2022) ar**v preprint ar**v:2209.02412

  14. Zhang, M., Zheng, Y.: Hair-gan: recovering 3D hair structure from a single image using generative adversarial networks. Vis. Inform. 3(2), 102–112 (2019)

    Article  Google Scholar 

  15. Jo, Y., Park, J.: Sc-fegan: face editing generative adversarial network with user’s sketch and color. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1745–1753 (2019)

  16. Saha, R., Duke, B., Shkurti, F., Taylor, G.W., Aarabi, P.: Loho: latent optimization of hairstyles via orthogonalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1984–1993 (2021)

  17. Mirza, M., Osindero, S.: Conditional generative adversarial nets. (2014) ar**v preprint ar**v:1411.1784

  18. Wei, T., Chen, D., Zhou, W., Liao, J., Tan, Z., Yuan, L., Zhang, W., Yu, N.: Hairclip: design your hair by text and reference image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18072–18081 (2022)

  19. Richardson, E., Alaluf, Y., Patashnik, O., Nitzan, Y., Azar, Y., Shapiro, S., Cohen-Or, D.: Encoding in style: a stylegan encoder for image-to-image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2287–2296 (2021)

  20. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)

  21. Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8798–8807 (2018)

  22. Ulyanov, D., Vedaldi, A., Lempitsky, V.: Instance normalization: the missing ingredient for fast stylization. (2016) ar**v preprint ar**v:1607.08022

  23. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning. PMLR, pp. 448–456 (2015)

  24. Dolhansky, B., Ferrer, C.C.: Eye in-painting with exemplar generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7902–7911 (2018)

  25. Olszewski, K., Ceylan, D., **ng, J., Echevarria, J., Chen, Z., Chen, W., Li, H.: Intuitive, interactive beard and hair synthesis with generative models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7446–7456 (2020)

  26. Lee, C.-H., Liu, Z., Wu, L., Luo, P.: Maskgan: towards diverse and interactive facial image manipulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5549–5558 (2020)

  27. Luo, L., Li, H., Rusinkiewicz, S.: Structure-aware hair capture. ACM Trans. Gr. (TOG) 32(4), 1–12 (2013)

    Article  MATH  Google Scholar 

  28. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. (2014) ar**v preprint ar**v:1409.1556

  29. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. (2014) ar**v preprint ar**v:1412.6980

  30. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. Adv. Neural Inform. Process. Syst. 30 (2017)

  31. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Engineering Research Center of Hubei Province for Clothing Information, Wuhan, 430200, China

    **nrong Hu, Qing Chang, Junjie Huang, Ruiqi Luo, Bangchao Wang & Chang Hu

  2. School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan, 430200, China

    **nrong Hu, Qing Chang, Junjie Huang, Ruiqi Luo, Bangchao Wang & Chang Hu

Authors
  1. **nrong Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junjie Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruiqi Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bangchao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junjie Huang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, X., Chang, Q., Huang, J. et al. HSSAN: hair synthesis with style-guided spatially adaptive normalization on generative adversarial network. Vis Comput 39, 3311–3318 (2023). https://doi.org/10.1007/s00371-023-02998-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-02998-5

Keywords

Search

Navigation