SpringerLink
Log in

Radial mesh pattern: a handcrafted feature descriptor for facial expression recognition

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Facial expressions are extremely important in the social interaction as they can display the internal emotions and intentions of an individual. Accurately classifying the facial expressions into various categories is the main task in Automatic Facial Expression Recognition (AFER) systems. The existing local based techniques, at times suffer and generate same feature values for different image portions such as edge, corner and flat regions. To address this issue, Radial Mesh Pattern (RMP), a local texture based approach based on the chess game rules is proposed. With reference to the center pixel in a \(5\times 5\) neighborhood, the possible positions of Rook, Bishop and Knight are determined and based on these positions, the features are extracted. In this paper, not only binary weights, but also other weights such as fibonacci, prime, natural, squares, odd and even weights have been utilized for feature extraction. To validate the efficiency of the proposed method, RMP is implemented on six ‘in the lab’ datasets. The performance is measured through recognition accuracy and the results obtained from experiments demonstrate the efficiency of RMP over standard existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and materials

The datasets used in this work are available in the below links. JAFFE: https://zenodo.org/record/3451524#.YMyv2eFR02x TFEID: https://bml.ym.edu.tw/tfeid/modules/wfdownloads/ KDEF: https://www.kdef.se/download-2/register.html CK+: https://www.pitt.edu/~emotion/ck-spread.htm MUG: https://mug.ee.auth.gr/fed/ OULU-CASIA https://www.oulu.fi/cmvs/node/41316.

References

  • Aghamaleki JA, Chenarlogh VA (2019) Multi-stream cnn for facial expression recognition in limited training data. Multimedia Tools Appl 78(16):22861–22882

    Article  Google Scholar 

  • Aifanti N, Papachristou C, Delopoulos A (2010) The mug facial expression database. In: 11th International workshop on image analysis for multimedia interactive services WIAMIS 10, IEEE, pp 1–4

  • Alenazy WM, Alqahtani AS (2021) Gravitational search algorithm based optimized deep learning model with diverse set of features for facial expression recognition. J Ambient Intell Human Comput 12:1631–1646. https://doi.org/10.1007/s12652-020-02235-0

    Article  Google Scholar 

  • Ali AM, Zhuang H, Ibrahim AK (2017) An approach for facial expression classification. Int J Biom 9(2):96–112

    Google Scholar 

  • Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  • Chen L-F, Yen Y-S (2007) Taiwanese facial expression image database. Institute of Brain ScienceTaiwan, Taiwan

    Google Scholar 

  • Etemad K, Chellappa R (1997) Discriminant analysis for recognition of human face images. Josa A 14(8):1724–1733

    Article  Google Scholar 

  • Goeleven E, De Raedt R, Leyman L, Verschuere B (2008) The Karolinska directed emotional faces: a validation study. Cognit Emot 22(6):1094–1118

    Article  Google Scholar 

  • Iqbal MTB, Ryu B, Song G, Kim J, Makhmudkhujaev F, Chae O (2016) Exploring positional ternary pattern (ptp) for conventional facial expression recognition from static images. In: Korea Comput Congress, pp 853–855

  • Kartheek MN, Prasad MVNK, Bhukya R (2020) Local optimal oriented pattern for person independent facial expression recognition. In: Twelfth international conference on machine vision (ICMV 2019), vol 11433, International Society for Optics and Photonics, p 114330R1-8

  • Kola DGR, Samayamantula SK (2021) Facial expression recognition using singular values and wavelet-based lgc-hd operator. IET Biometric 10:207–218

    Article  Google Scholar 

  • Kung H-W, Yi-Han T, Hsu C-T (2015) Dual subspace nonnegative graph embedding for identity-independent expression recognition. IEEE Trans Inf Forensics Secur 10(3):626–639

    Article  Google Scholar 

  • Lai C-C, Ko C-H (2014) Facial expression recognition based on two-stage features extraction. Optik 125(22):6678–6680

    Article  Google Scholar 

  • Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE computer society conference on computer vision and pattern recognition-workshops, IEEE, pp 94–101

  • Lyons M, Akamatsu Shigeru, Kamachi M, Gyoba J (1998) Coding facial expressions with gabor wavelets. In: Proceedings third IEEE international conference on automatic face and gesture recognition, IEEE, pp 200–205

  • Maheswari VU, Varaprasad G, Raju SV (2020) Local directional maximum edge patterns for facial expression recognition. J Ambient Intell Humaniz Comput 12:1–9

    Google Scholar 

  • Makhmudkhujaev F, Abdullah-Al-Wadud M, Iqbal MT, Ryu BB, Chae O (2019) Facial expression recognition with local prominent directional pattern. Signal Process Image Commun 74:1–12

    Article  Google Scholar 

  • Makhmudkhujaev F, Iqbal MT, Ryu BB, Chae O (2019) Local directional-structural pattern for person-independent facial expression recognition. Turk J Electr Eng Comput Sci 27(1):516–531

    Article  Google Scholar 

  • Mandal M, Verma M, Mathur S, Kumar VS, Subrahmanyam M, Kranthi KD (2019) Regional adaptive affinitive patterns (radap) with logical operators for facial expression recognition. IET Image Process 13(5):850–861

    Article  Google Scholar 

  • Martinez A, Du S (2012) A model of the perception of facial expressions of emotion by humans: research overview and perspectives. J Mach Learn Res 13:1589–1608

    MathSciNet  Google Scholar 

  • Murala S, Wu JQM (2013) Local mesh patterns versus local binary patterns: biomedical image indexing and retrieval. IEEE J Biomed Health Inform 18(3):929–938

    Article  Google Scholar 

  • Reddy PCS, Rao PVP, Reddy PKK, Sridhar M (2019) Motif shape primitives on fibonacci weighted neighborhood pattern for age classification. Soft Comput Signal Process. Springer, Berlin, pp 273–280

    Chapter  Google Scholar 

  • Rivera AR, Jorge RC, Chae O (2012) Local directional number pattern for face analysis: Face and expression recognition. IEEE Trans Image Process 22(5):1740–1752

    Article  MathSciNet  MATH  Google Scholar 

  • Rivera AR, Jorge RC, Chae O (2015) Local directional texture pattern image descriptor. Pattern Recogn Lett 51:94–100

    Article  Google Scholar 

  • Ryu B, Rivera AR, Kim K, Chae O (2017) Local directional ternary pattern for facial expression recognition. IEEE Trans Image Process 26(12):6006–6018

    Article  MathSciNet  Google Scholar 

  • Sen D, Datta S, Balasubramanian R (2019) Facial emotion classification using concatenated geometric and textural features. Multimedia Tools Appl 78(8):10287–10323

    Article  Google Scholar 

  • Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27(6):803–816

    Article  Google Scholar 

  • Sun Z, Hu Z-P, Wang M, Zhao S-H (2017) Individual-free representation-based classification for facial expression recognition. Signal Image Video Process 11(4):597–604

    Article  Google Scholar 

  • Taskeed J, Hasanul KM, Oksam C (2010) Robust facial expression recognition based on local directional pattern. ETRI J 32(5):784–794

    Article  Google Scholar 

  • Tian Y-L, Kanade T, Cohn JF (2005) Facial expression analysis. Handbook of face recognition. Springer, Berlin, pp 247–275

    Chapter  Google Scholar 

  • Tong Y, Chen R (2019) Local dominant directional symmetrical coding patterns for facial expression recognition. Comput Intell Neurosci 1–13:2019

    Google Scholar 

  • Tuncer T, Dogan S, Ataman V (2019) A novel and accurate chess pattern for automated texture classification. Phys A Stat Mech Appl 536:122584

    Article  Google Scholar 

  • Turan C, Lam K-M, He X (2018) Soft locality preserving map (slpm) for facial expression recognition. ar**v preprint. ar**v:1801.03754

  • Turk M, Alex P (1991) Face recognition using eigenfaces. In: Proceedings of 1991 IEEE computer society conference on computer vision and pattern recognition, pp 586–587

  • Verma V, Kumar VS, Singh G (2019) Hinet: hybrid inherited feature learning network for facial expression recognition. IEEE Lett Comput Soc 2(4):36–39

    Article  Google Scholar 

  • Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  • **e S, Haifeng H, Yongbo W (2019) Deep multi-path convolutional neural network joint with salient region attention for facial expression recognition. Pattern Recogn 92:177–191

    Article  Google Scholar 

  • Zhao G, Huang X, Taini M, Li SZ, PietikäInen M (2011) Facial expression recognition from near-infrared videos. Image Vis Comput 29(9):607–619

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Development and Research in Banking Technology, Hyderabad, India

    Mukku Nisanth Kartheek & Munaga V. N. K. Prasad

  2. Department of Computer Science and Engineering, National Institute of Technology, Warangal, India

    Mukku Nisanth Kartheek & Raju Bhukya

Authors
  1. Mukku Nisanth Kartheek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Munaga V. N. K. Prasad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Raju Bhukya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mukku Nisanth Kartheek.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kartheek, M.N., Prasad, M.V.N.K. & Bhukya, R. Radial mesh pattern: a handcrafted feature descriptor for facial expression recognition. J Ambient Intell Human Comput 14, 1619–1631 (2023). https://doi.org/10.1007/s12652-021-03384-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-021-03384-6

Keywords

Search

Navigation