Abstract
Information Technology has touched new vistas for a couple of decades mostly to simplify the day-to-day life of the humans. One of the key contributions of Information Technology is the application of Artificial Intelligence to achieve better results. The advent of artificial intelligence has given rise to a new branch of Natural Language Processing (NLP) called Computational Linguistics, which generates frameworks for intelligently manipulating spoken language knowledge and has brought human-machine onto a new stage. In this context, speech has arisen to be one of the imperative forms of interfaces, which is the basic mode of communication for us, and generally the most preferred one. Language identification, being the front-end for various natural language processing tasks, plays an important role in language translation. Owing to this, the focus has been given on the field of speech recognition involving the identification & recognition of languages by a machine. Spoken language identification is the identification of language present in a speech segment despite its size (duration & speed), ambiance (topic & emotion), and moderator (gender, age, demographic region). This paper has investigated various existing spoken language identification models implemented using different deep learning approaches, datasets, and performance measures utilized for their analysis. It also highlights the main features and challenges faced by these models. A comprehensive comparative study of deep learning techniques has been carried out for spoken language identification. Moreover, this review analyzes the efficiency of the spoken language models that can help the researchers to propose new language identification models for speech signals.
Similar content being viewed by others
References
Agatonovic-Kustrin S, Beresford R (2000) Basic concepts of artificial neural network (ANN) modeling and its application in pharmaceutical research. J Pharm Biomed Anal 22(5):717–727
Albadr MAA, Tiun S (2020) Spoken language identification based on particle swarm optimisation–extreme learning machine approach. Circ Syst Signal Process 1–27
Albadr MAA, Tiun S, Ayob M, AL-Dhief FT (2019) Spoken language identification based on optimised genetic algorithm–extreme learning machine approach. Int J Speech Technol 22(3):711–727
Albawi S, Mohammed TA, Al-Zawi S (2017) Understanding of a convolutional neural network. In 2017 international conference on engineering and Technology (ICET) (pp 1–6). Ieee
Ambikairajah E, Li H, Wang L, Yin B, Sethu V (2011) Language identification: A tutorial. IEEE Circ Syst Mag 11(2):82–108
Bartz C, Herold T, Yang H, Meinel C (2017) Language identification using deep convolutional recurrent neural networks. In international conference on neural information processing (pp 880–889). Springer, Cham
Bastanfard A, Amirkhani D (2020) Detect hidden message in reverse timestamp in Farsi. In 2020 25th international computer conference, computer society of Iran (CSICC) (pp 1–8). IEEE
Bastanfard A, Amirkhani D, Hasani M Increasing the accuracy of automatic speaker age estimation by using multiple UBMs. In 2019 5th conference on knowledge based engineering and innovation (KBEI) (pp 592–598). IEEE
Bastanfard A, Amirkhani D, Naderi S (2020) A singing voice separation method from Persian music based on pitch detection methods. In 2020 6th Iranian conference on signal processing and intelligent systems (ICSPIS) (pp 1–7). IEEE
Buhmann MD (2000) Radial basis functions. Acta Numer 9:1–38
Buscema M (1998) Back propagation neural networks. Subst Use Misuse 33(2):233–270
Cai W, Cai D, Huang S, Li M (2019) Utterance-level end-to-end language identification using attention-based CNN-BLSTM. In ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp 5991–5995). IEEE
Chowdhury AA, Borkar VS, Birajdar GK (2020) Indian language identification using time-frequency image textural descriptors and GWO-based feature selection. J Exp Theor Artif Intell 32(1):111–132
Das A, Guha S, Singh PK, Ahmadian A, Senu N, Sarkar R (2020) A hybrid meta-heuristic feature selection method for identification of Indian spoken languages from audio signals. IEEE Access 8:181432–181449
Das HS, Roy P (2019) Optimal prosodic feature extraction and classification in parametric excitation source information for Indian language identification using neural network based Q-learning algorithm. Int J Speech Technol 22(1):67–77
Das HS, Roy P (2019) A deep dive into deep learning techniques for solving spoken language identification problems. In intelligent speech signal processing (pp 81–100). Academic Press
Dehak N, Kenny PJ, Dehak R, Dumouchel P, Ouellet P (2010) Front-end factor analysis for speaker verification. IEEE Trans Audio Speech Lang Process 19(4):788–798
Deshwal D, Sangwan P, Kumar D (2020) A language identification system using hybrid features and back-propagation neural network. Appl Acoust 164:107289
Ferrer L, Lei Y, McLaren M, Scheffer N (2015) Study of senone-based deep neural network approaches for spoken language recognition. IEEE/ACM Trans Audio Speech Lang Process 24(1):105–116
Fine TL (2006) Feedforward neural network methodology. Springer Science & Business Media
Fu Z, Lu G, Ting KM, Zhang D (2010) A survey of audio-based music classification and annotation. IEEE Trans Multimed 13(2):303–319
Gadekallu TR, Khare N, Bhattacharya S, Singh S, Reddy Maddikunta PK, Ra IH, Alazab M (2020) Early detection of diabetic retinopathy using PCA-firefly based deep learning model. Electronics 9(2):274
Ganapathy S, Han K, Thomas S, Omar M, Segbroeck MV, Narayanan SS (2014) Robust language identification using convolutional neural network features. In Fifteenth annual conference of the international speech communication association
Garain A, Singh PK, Sarkar R (2021) FuzzyGCP: a deep learning architecture for automatic spoken language identification from speech signals. Expert Syst Appl 168:114416
Gelly G, Gauvain JL, Le VB, Messaoudi A (2016) A divide-and-conquer approach for language identification based on recurrent neural networks. In INTERSPEECH (pp 3231–3235)
Geng W, Wang W, Zhao Y, Cai X, Xu B, **nyuan C (2016) End-to-end language identification using attention-based recurrent neural networks. In Interspeech (pp 2944–2948)
Gonzalez-Dominguez J, Lopez-Moreno I, Sak H, Gonzalez-Rodriguez J, Moreno PJ (2014) Automatic language identification using long short-term memory recurrent neural networks. In Fifteenth annual conference of the international speech communication association
Gordon RG Jr (2005) Ethnologue, languages of the world. http://www.ethnologue.com/
Guha S, Das A, Singh PK, Ahmadian A, Senu N, Sarkar R (2020) Hybrid feature selection method based on harmony search and naked mole-rat algorithms for spoken language identification from audio signals. IEEE Access 8:182868–182887
Heracleous P, Takai K, Yasuda K, Mohammad Y, Yoneyama A (2018) Comparative study on spoken language identification based on deep learning. In 2018 26th European signal processing conference (EUSIPCO) (pp 2265–2269). IEEE
Hinton G, Deng L, Yu D, Dahl GE, Mohamed AR, Jaitly N, … Kingsbury B (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97
Hori C, Hori T, Watanabe S, Hershey JR (2016) Context-sensitive and role-dependent spoken language understanding using bidirectional and attention LSTMs. In Interspeech (pp 3236–3240)
** M, Song Y, McLoughlin IV (2017) End-to-end dnn-cnn classification for language identification
Jothilakshmi S, Ramalingam V, Palanivel S (2012) A hierarchical language identification system for Indian languages. Digit Signal Process 22(3):544–553
Kim H, Park JS (2020) Automatic language identification using speech rhythm features for multi-lingual speech recognition. Appl Sci 10(7):2225
Lee CH (2008) Principles of spoken language recognition. In springer handbook of speech processing (pp 785–796). Springer, Berlin, Heidelberg
Lei Y, Ferrer L, Lawson A, McLaren M, Scheffer N (2014) Application of Convolutional Neural Networks to Language Identification in Noisy Conditions. In Odyssey
Li H, Ma B, Lee KA (2013) Spoken language recognition: from fundamentals to practice. Proc IEEE 101(5):1136–1159
Liu B, Lane I (2016) Joint online spoken language understanding and language modeling with recurrent neural networks. ar**v preprint ar**v:1609.01462
Lopez-Moreno I, Gonzalez-Dominguez J, Martinez D, Plchot O, Gonzalez-Rodriguez J, Moreno PJ (2016) On the use of deep feedforward neural networks for automatic language identification. Comput Speech Lang 40:46–59
Lounnas K, Satori H, Hamidi M, Teffahi H, Abbas M, Lichouri M (2020) CLIASR: a combined automatic speech recognition and language identification system. In 2020 1st international conference on innovative research in applied science, engineering and Technology (IRASET) (pp 1–5). IEEE
Lu X, Shen P, Tsao Y, Kawai H (2017) Regularization of neural network model with distance metric learning for i-vector based spoken language identification. Comput Speech Lang 44:48–60
Ma B, Li H, Tong R (2007) Spoken language recognition using ensemble classifiers. IEEE Trans Audio Speech Lang Process 15(7):2053–2062
Mahdavi R, Bastanfard A, Amirkhani D (2020) Persian accents identification using modeling of speech articulatory features. In 2020 25th international computer conference, Computer Society of Iran (CSICC) (pp 1–9). IEEE
Masumura R, Asami T, Masataki H, Aono Y (2017) Parallel phonetically aware DNNs and LSTM-RNNs for frame-by-frame discriminative modeling of spoken language identification. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp 5260–5264). IEEE
Miao X, McLoughlin I, Yan Y (2019) A new time-frequency attention mechanism for TDNN and CNN-LSTM-TDNN, with application to language identification. In Interspeech (pp 4080–4084)
Mikolov T, Karafiát M, Burget L, Černocký J, Khudanpur S (2010) Recurrent neural network based language model. In Eleventh annual conference of the international speech communication association
Mohamed AR, Dahl GE, Hinton G (2011) Acoustic modeling using deep belief networks. IEEE Trans Audio Speech Lang Process 20(1):14–22
Montavon G (2009) Deep learning for spoken language identification. In NIPS workshop on deep learning for speech recognition and related applications (pp 1–4)
Morchid M (2017) Internal memory gate for recurrent neural networks with application to spoken language understanding. In INTERSPEECH (pp 3316–3319)
Morchid M Bidirectional internal memory gate recurrent neural networks for spoken language understanding. Int J Speech Technol 1–9
Mukherjee H, Obaidullah SM, Santosh KC, Phadikar S, Roy K (2020) A lazy learning-based language identification from speech using MFCC-2 features. Int J Mach Learn Cybern 11(1):1–14
Muthusamy YK, Barnard E, Cole RA (1994) Reviewing automatic language identification. IEEE Signal Process Mag 11(4):33–41
Navratil J (2001) Spoken language recognition-a step toward multilinguality in speech processing. IEEE Trans Speech Audio Process 9(6):678–685
Peche M, Davel MH, Barnard E (2009) Development of a spoken language identification system for south African languages. SAIEE Africa Res J 100(4):97–103
Poncelet J, Renkens V (2021) Low resource end-to-end spoken language understanding with capsule networks. Comput Speech Lang 66:101142
Qian Y, Ubale R, Lange P, Evanini K, Ramanarayanan V, Soong FK (2019) Spoken language understanding of human-machine conversations for language learning applications. J Signal Process Syst 1–13
Richardson F, Reynolds D, Dehak N (2015) Deep neural network approaches to speaker and language recognition. IEEE Signal Process Lett 22(10):1671–1675
Sharma V, Rai S, Dev A (2012) A comprehensive study of artificial neural networks. Int J Adv Res Comput Sci Softw Eng 2(10)
Shen P, Lu X, Li S, Kawai H (2020) Knowledge distillation-based representation learning for short-utterance spoken language identification. IEEE/ACM Trans Audio Speech Lang Process 28:2674–2683
Sim KC, Li H (2008) On acoustic diversification front-end for spoken language identification. IEEE Trans Audio Speech Lang Process 16(5):1029–1037
Song Y, Cui R, Hong X, Mcloughlin I, Shi J, Dai L (2015) Improved language identification using deep bottleneck network. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp 4200–4204). IEEE
Torres-Carrasquillo PA, Reynolds DA, Deller JR (2002) Language identification using Gaussian mixture model tokenization. In 2002 IEEE international conference on acoustics, speech, and signal processing (Vol 1, pp I–757). IEEE
Tzanetakis G, Cook P (2002) Musical genre classification of audio signals. IEEE Trans Speech Audio Process 10(5):293–302
Ultsch A (1993) Self-organizing neural networks for visualisation and classification. In information and classification (pp 307–313). Springer, Berlin, Heidelberg
Vasan D, Alazab M, Wassan S, Safaei B, Zheng Q (2020) Image-based malware classification using ensemble of CNN architectures (IMCEC). Comput Secur 92:101748
Veisi H, Ghoreishi SA, Bastanfard A (2021) Spoken term detection for Persian news of Islamic Republic of Iran broadcasting. Signal Data Process 17(4):67–88
Vuddagiri RK, Vydana HK, Vuppala AK (2018) Curriculum learning based approach for noise robust language identification using DNN with attention. Expert Syst Appl 110:290–297
Wikipedia, “https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons”
Wong KYE (2004) Automatic spoken language identification utilizing acoustic and phonetic speech information (Doctoral dissertation, Queensland University of Technology)
Wu WL, Lu RZ, Duan JY, Liu H, Gao F, Chen YQ (2010) Spoken language understanding using weakly supervised learning. Comput Speech Lang 24(2):358–382
Yasmin G, Das AK, Nayak J, Pelusi D, Ding W (2020) Graph based feature selection investigating boundary region of rough set for language identification. Expert Syst Appl 158:113575
Yu D, Deng L (2010) Deep learning and its applications to signal and information processing [exploratory dsp]. IEEE Signal Process Mag 28(1):145–154
Zissman MA, Berkling KM (2001) Automatic language identification. Speech Commun 35(1-2):115–124
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Thukroo, I.A., Bashir, R. & Giri, K.J. A review into deep learning techniques for spoken language identification. Multimed Tools Appl 81, 32593–32624 (2022). https://doi.org/10.1007/s11042-022-13054-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-13054-0
Keywords
- Spoken language identification
- Gaussian mixture model
- Support vector machine
- Hidden Markov model
- Deep neural networks
- Artificial neural network
- Feed-forward neural network
- Recurrent neural network
- Convolutional neural network
- Ensemble learning; hybridization approaches
- Mel frequency cepstral coefficient features