SpringerLink
Log in

Facial Point Graphs for Stroke Identification

  • Conference paper
  • First Online:
Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications (CIARP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14469))

Included in the following conference series:

  • 454 Accesses

Abstract

Stroke can cause significant damage to neurons, resulting in various sequelae that negatively impact the patient’s ability to perform essential daily activities such as chewing, swallowing, and verbal communication. Therefore, it is important for patients with such difficulties to undergo a treatment process and be monitored during its execution to assess the improvement of their health condition. The use of computerized tools and algorithms that can quickly and affordably detect such sequelae proves helpful in aiding the patient’s recovery. Due to the death of internal brain cells, a stroke often leads to facial paralysis, resulting in certain asymmetry between the two sides of the face. This paper focuses on analyzing this asymmetry using a deep learning method without relying on handcrafted calculations, introducing the Facial Point Graphs (FPG) model, a novel approach that excels in learning geometric information and effectively handling variations beyond the scope of manual calculations. FPG allows the model to effectively detect orofacial impairment caused by a stroke using video data. The experimental findings on the Toronto Neuroface dataset revealed the proposed approach surpassed state-of-the-art results, promising substantial advancements in this domain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 82.38
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 104.85
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    For the BLOW subtask, there were 15 participants, consisting of 8 stroke patients and 7 healthy controls (HC).

  2. 2.

    \(C =[2^{-5}, 2^{-3}, .. 2^{15}]\), and \(\gamma = [2^{-15}, 2^{-13}, ..., 2^{3}]\).

  3. 3.

    Note: The experimental evaluation of FPG was conducted using PyTorch Geometric [9] on a GPU. An important point to mention is that, up to the current paper, the implementation of Graph Neural Networks (GNNs) using scatter operation on a GPU introduces non-deterministic behavior.

References

  1. Baltrusaitis, T., Zadeh, A., Lim, Y.C., Morency, L.P.: Openface 2.0: facial behavior analysis toolkit. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 59–66. IEEE (2018)

    Google Scholar 

  2. Bandini, A., Green, J.R., Richburg, B., Yunusova, Y.: Automatic detection of orofacial impairment in stroke. In: Interspeech, pp. 1711–1715 (2018)

    Google Scholar 

  3. Bandini, A., et al.: A new dataset for facial motion analysis in individuals with neurological disorders. IEEE J. Biomed. Health Inform. 25(4), 1111–1119 (2020)

    Article  Google Scholar 

  4. Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). In: International Conference on Computer Vision (2017)

    Google Scholar 

  5. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (1995)

    Google Scholar 

  6. Delaunay, B., et al.: Sur la sphere vide. Izv. Akad. Nauk SSSR, Otdelenie Matematicheskii i Estestvennyka Nauk 7(793-800), 1–2 (1934)

    Google Scholar 

  7. Dhall, A., Goecke, R., Lucey, S., Gedeon, T., et al.: Collecting large, richly annotated facial-expression databases from movies. IEEE Multimedia 19(3), 34 (2012)

    Article  Google Scholar 

  8. Dong, X., Yan, Y., Ouyang, W., Yang, Y.: Style aggregated network for facial landmark detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 379–388 (2018)

    Google Scholar 

  9. Fey, M., Lenssen, J.E.: Fast graph representation learning with pytorch geometric. ar**v preprint ar**v:1903.02428 (2019)

  10. Fix, E., Hodges, J.L.: Discriminatory analysis. nonparametric discrimination: Consistency properties. Inter. Stat. Review/Revue Internationale de Statistique 57(3), 238–247 (1989)

    Google Scholar 

  11. Gomes, N.B., Yoshida, A., Roder, M., de Oliveira, G.C., Papa, J.P.: Facial point graphs for amyotrophic lateral sclerosis identification. ar**v preprint ar**v:2307.12159 (2023)

  12. Greene, J.J., et al.: The spectrum of facial palsy: the meei facial palsy photo and video standard set. Laryngoscope 130(1), 32–37 (2020)

    Article  Google Scholar 

  13. Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28(5), 807–813 (2010)

    Article  Google Scholar 

  14. Guarin, D.L., et al.: Toward an automatic system for computer-aided assessment in facial palsy. Facial Plastic Surgery Aesthetic Med. 22(1), 42–49 (2020)

    Article  MathSciNet  Google Scholar 

  15. Haykin, S.: Neural networks: a comprehensive foundation. Prentice Hall PTR (1994)

    Google Scholar 

  16. Ho, T.K.: Random decision forests. In: Proceedings of 3rd International Conference on Document Analysis and Recognition, vol. 1, pp. 278–282. IEEE (1995)

    Google Scholar 

  17. Kaewmahanin, W., et al.: Automatic facial asymmetry analysis for elderly stroke detection by using cosine similarity. In: 2022 19th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), pp. 1–4. IEEE (2022)

    Google Scholar 

  18. Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)

    Google Scholar 

  19. Kim, H.S., Kim, S.Y., Kim, Y.H., Park, K.S.: A smartphone-based automatic diagnosis system for facial nerve palsy. Sensors 15(10), 26756–26768 (2015)

    Article  Google Scholar 

  20. King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)

    Google Scholar 

  21. Lapchak, P.A., Zhang, J.H.: The high cost of stroke and stroke cytoprotection research. Transl. Stroke Res. 8, 307–317 (2017)

    Article  Google Scholar 

  22. Lou, J., Yu, H., Wang, F.Y.: A review on automated facial nerve function assessment from visual face capture. IEEE Trans. Neural Syst. Rehabil. Eng. 28(2), 488–497 (2019)

    Article  Google Scholar 

  23. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-workshops, pp. 94–101. IEEE (2010)

    Google Scholar 

  24. Mullen, M., Loomis, C., et al.: Differentiating facial weakness caused by bell’s palsy vs. acute stroke. JEMS (2014)

    Google Scholar 

  25. Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29

    Chapter  Google Scholar 

  26. Ngoc, Q.T., Lee, S., Song, B.C.: Facial landmark-based emotion recognition via directed graph neural network. Electronics 9(5), 764 (2020)

    Article  Google Scholar 

  27. Pantic, M., Valstar, M., Rademaker, R., Maat, L.: Web-based database for facial expression analysis. In: 2005 IEEE International Conference on Multimedia and Expo, pp. 5–pp. IEEE (2005)

    Google Scholar 

  28. Parra-Dominguez, G.S., Sanchez-Yanez, R.E., Garcia-Capulin, C.H.: Facial paralysis detection on images using key point analysis. Appl. Sci. 11(5), 2435 (2021)

    Article  Google Scholar 

  29. Pecundo, A.M., Abu, P.A., Alampay, R.: Amyotrophic lateral sclerosis and post-stroke orofacial impairment video-based multi-class classification. In: Proceedings of the 2022 5th Artificial Intelligence and Cloud Computing Conference, pp. 150–157 (2022)

    Google Scholar 

  30. Samsudin, W.W., Sundaraj, K.: Image processing on facial paralysis for facial rehabilitation system: A review. In: 2012 IEEE International Conference on Control System, Computing and Engineering, pp. 259–263. IEEE (2012)

    Google Scholar 

  31. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61–80 (2008)

    Article  Google Scholar 

  32. Schimmel, M., Ono, T., Lam, O., Müller, F.: Oro-facial impairment in stroke patients. J. Oral Rehabil. 44(4), 313–326 (2017)

    Article  Google Scholar 

  33. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. ar**v preprint ar**v:1710.10903 (2017)

  34. **ong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. São Paulo State University (UNESP), Bauru, CEP, 17033-360, Brazil

    Nicolas Barbosa Gomes, Arissa Yoshida, Guilherme Camargo de Oliveira, Mateus Roder & João Paulo Papa

Authors
  1. Nicolas Barbosa Gomes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arissa Yoshida
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guilherme Camargo de Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mateus Roder
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. João Paulo Papa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to João Paulo Papa .

Editor information

Editors and Affiliations

  1. Polytechnic Institute of Coimbra, Coimbra Institute of Engineering, Coimbra, Portugal

    Verónica Vasconcelos

  2. Polytechnic Institute of Coimbra, Coimbra Institute of Engineering, Coimbra, Portugal

    Inês Domingues

  3. Polytechnic Institute of Coimbra, Coimbra Institute of Engineering, Coimbra, Portugal

    Simão Paredes

Rights and permissions

Reprints and permissions

Copyright information

© 2024 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gomes, N.B., Yoshida, A., de Oliveira, G.C., Roder, M., Papa, J.P. (2024). Facial Point Graphs for Stroke Identification. In: Vasconcelos, V., Domingues, I., Paredes, S. (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. CIARP 2023. Lecture Notes in Computer Science, vol 14469. Springer, Cham. https://doi.org/10.1007/978-3-031-49018-7_49

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49018-7_49

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49017-0

  • Online ISBN: 978-3-031-49018-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation