SpringerLink
Log in

A hybrid deep-based model for scene text detection and recognition in meter reading

  • Original Research
  • Published:
International Journal of Information Technology Aims and scope Submit manuscript

Abstract

Scene text detection and recognition are important tasks. However, scene text detection and recognition in real world images is one of the most challenging tasks in text mining. This paper focuses on one of the scene text detection and recognition applications, electrical meter reading. For this purpose, we propose a two-step model. First, a detection method is adapted to detect digit regions. Then, an end-to-end neural network is used, which is a combination of convolutional neural networks (CNNs), recurrent neural networks (RNNs), and connectionist temporal classification (CTC) loss. The convolutional part pulls out important features, while the RNN part encodes and decodes sequences of features. Moreover, we gather a large dataset of meter devices whose numbers are in Persian. In order to evaluate our model, we compare it with different architectures and models. The analysis of the results shows that our model achieves higher accuracy and outperforms the baseline approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

Data are available based on the request.

Notes

  1. E-meter dataset.

References

  1. Liao M, Zhang J, Wan Z, **e F, Liang J, Lyu P et al (2019) Scene text recognition from two-dimensional perspective. In: Proceedings of the AAAI conference on artificial intelligence, Vol 33, No. 01, pp 8714–8721

  2. Tian Z, Huang W, He T, He P, Qiao Y (2016) Detecting text in natural image with connectionist text proposal network. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9912 LNCS, pp. 56–72. https://doi.org/10.1007/978-3-319-46484-8_4

  3. Lei Z, Zhao S, Song H, Shen J (2018) Scene text recognition using residual convolutional recurrent neural network. Mach Vis Appl 29(5):861–871. https://doi.org/10.1007/s00138-018-0942-y

    Article  Google Scholar 

  4. Chen X, ** L, Zhu Y, Luo C, Wang T (2021) Text recognition in the wild: a survey. ACM Comput Survey 54(2)1–35. https://doi.org/10.1145/3440756

  5. Bai J, Posner R, Wang T, Yang C, Nabavi S (2021) Applying deep learning in digital breast tomosynthesis for automatic breast cancer detection: a review. Medical Image Analysis 71:102049. https://doi.org/10.1016/j.media.2021.102049

    Article  Google Scholar 

  6. Bharti R, Khamparia A, Shabaz M, Dhiman G, Pande S, Singh P (2021) Prediction of heart disease using a combination of machine learning and deep learning. Comput Intell Neurosci. https://doi.org/10.1155/2021/8387680

    Article  Google Scholar 

  7. Ibrahim DM, Elshennawy NM, Sarhan AM (2021) Deep-chest: multi-classification deep learning model for diagnosing COVID-19, pneumonia, and lung cancer chest diseases. Comput Biol Med 132:104348. https://doi.org/10.1016/j.compbiomed.2021.104348

    Article  Google Scholar 

  8. Nassif AB, Elnagar A, Shahin I, Henno S (2021) Deep learning for Arabic subjective sentiment analysis: challenges and research opportunities. Appl Soft Comput 98:106836. https://doi.org/10.1016/j.asoc.2020.106836

    Article  Google Scholar 

  9. Abdu SA, Yousef AH, Salem A (2021) Multimodal video sentiment analysis using deep learning approaches, a survey. Inform Fusion 76:204–226. https://doi.org/10.1016/j.inffus.2021.06.003

    Article  Google Scholar 

  10. He M, Liao M, Yang Z, Zhong H, Tang J, Cheng W et al (2021) MOST: a multi-oriented scene text detector with localization refinement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8813–8822

  11. Dai P, Zhang S, Zhang H, Cao X (2021) Progressive contour regression for arbitrary-shape scene text detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7393–7402

  12. El Bourakadi D, Ramadan H, Yahyaouy A, Boumhidi J (2022) A novel solar power prediction model based on stacked BiLSTM deep learning and improved extreme learning machine. Int J Inf Technol. https://doi.org/10.1007/s41870-022-01118-1

    Article  Google Scholar 

  13. Sachar S, Kumar A (2022) Deep ensemble learning for automatic medicinal leaf identification. Int J Inf Technol 14(6):3089–3097. https://doi.org/10.1007/s41870-022-01055-z

    Article  Google Scholar 

  14. Ngo VM, Duong TVT, Nguyen TBT, Dang CN, Conlan O (2023) A big data smart agricultural system: recommending optimum fertilisers for crops. Int J Inf Technol. https://doi.org/10.1007/s41870-022-01150-1

    Article  Google Scholar 

  15. Nithya B, Brijesh D, Kumar SK, Pathmakarthik J (2023) Pilot based channel estimation of OFDM systems using deep learning techniques. Int J Inf Technol. https://doi.org/10.1007/s41870-023-01155-4

    Article  Google Scholar 

  16. Yousef M, Hussain KF, Mohammed US (2020) Accurate, data-efficient, unconstrained text recognition with convolutional neural networks. Pattern Recognit. 108:107482. https://doi.org/10.1016/j.patcog.2020.107482

    Article  Google Scholar 

  17. Breuel TM, Ul-Hasan A, Al-Azawi MA, Shafait F (2013) High-performance OCR for printed English and Fraktur using lstm networks. In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, pp. 683–687. https://doi.org/10.1109/ICDAR.2013.140

  18. Graves A, Liwicki M, Fernández S, Bertolami R, Bunke H, Schmidhuber J (2009) A novel connectionist system for unconstrained handwriting recognition. IEEE Trans Pattern Anal Mach Intell 31(5):855–868. https://doi.org/10.1109/TPAMI.2008.137

    Article  Google Scholar 

  19. Jaderberg M, Simonyan K, Vedaldi A, Zisserman A (2016) Reading text in the wild with convolutional neural networks. Int J Comput Vis 116:1–20

  20. Cai H, Sun J, **ong Y (2021) Revisiting classification perspective on scene text recognition. ar**v preprint. https://arxiv.org/abs/2102.10884

  21. Salomon G, Laroca R, Menotti D (2020) Deep learning for image-based automatic dial meter reading: dataset and baselines. Proc Int Jt Conf Neural Netw. https://doi.org/10.1109/IJCNN48605.2020.9207318

    Article  Google Scholar 

  22. Borisyuk F, Gordo A, Sivakumar V (2018) Rosetta: large scale system for text detection and recognition in images. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 71–79. https://doi.org/10.1145/3219819.3219861

  23. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst, 28

  24. Zhang X, Zhou X, Lin M, Sun J (2018) ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  25. Wang W et al (2019) Shape robust text detection with progressive scale expansion network

  26. Liu X, Liang D, Yan S, Chen D, Qiao Y, Yan J (2018) FOTS: fast oriented text spotting with a unified network

  27. Liu Y, Chen H, Shen C, He T, ** L, Wang L (2020) ABCNet: real-time scene text spotting with adaptive Bezier-curve network

  28. Liu Z, Li Y, Ren F, Goh WL, Yu H (2018) SqueezedText: a real-time scene text recognition by binary convolutional encoder-decoder network. In: 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, vol. 32, no. 1, pp. 7194–7201, https://doi.org/10.1609/aaai.v32i1.12252

  29. Shi B, Bai X, Yao C (2017) An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans Pattern Anal Mach Intell 39(11):2298–2304. https://doi.org/10.1109/TPAMI.2016.2646371

    Article  Google Scholar 

  30. Graves A, Fernández S, Gomez F, Schmidhuber J (2006) Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. ACM International Conference Proceeding Series 148:369–376. https://doi.org/10.1145/1143844.1143891

    Article  Google Scholar 

  31. Gao Y, Huang Z, Dai Y, Xu C, Chen K, Guo J (2019) DSAN: double supervised network with attention mechanism for scene text recognition. In: 2019 IEEE International Conference on Visual Communications and Image Processing, VCIP 2019. https://doi.org/10.1109/VCIP47243.2019.8965779

  32. Ghosh SK, Valveny E, Bagdanov AD (2017) Visual attention models for scene text recognition. Proceedings of the International Conference on Document Analysis and Recognition, ICDAR 1:943–948. https://doi.org/10.1109/ICDAR.2017.158

    Article  Google Scholar 

  33. Bai S, Tang H, An S (2019) Coordinate CNNs and LSTMs to categorize scene images with multi-views and multi-levels of abstraction. Expert Syst Appl 120:298–309. https://doi.org/10.1016/j.eswa.2018.08.056

    Article  Google Scholar 

  34. Wojna Z et al (2017) Attention-Based Extraction of Structured Information from Street View Imagery. Proceedings of the International Conference on Document Analysis and Recognition, ICDAR 1:844–850. https://doi.org/10.1109/ICDAR.2017.143

    Article  Google Scholar 

  35. Dutta A, Zisserman A (2019) The {VIA} annotation software for images, audio and video. In: Proceedings of the 27th ACM International Conference on Multimedia, https://doi.org/10.1145/3343031.3350535

  36. Dutta A, Gupta A, Zissermann A (2016) {VGG} Image annotator ({VIA})

  37. Long S, He X, Yao C (2021) Scene text detection and recognition: the deep learning era. Int J Comput Vis 129(1):161–184

    Article  Google Scholar 

  38. Zhao Y, Cai Y, Wu W, Wang W (2022) Explore faster localization learning for scene text detection. ar**v Prepr. ar**v2207.01342

  39. Zhou X et al (2017) EAST: an efficient and accurate scene text detector

Download references

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering Department, University of Tabriz, 29 Bahman Blvd., Tabriz, 5166616471, Iran

    Adil Abdullah Abdulhussein Alshawi, Jafar Tanha, Mohammad Ali Balafar & Soodabeh Imanzadeh

Authors
  1. Adil Abdullah Abdulhussein Alshawi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jafar Tanha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Ali Balafar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Soodabeh Imanzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jafar Tanha.

Ethics declarations

Conflict of interest

All authors have participated in (a) conception and design, or analysis and interpretation of the data; (b) drafting the article, and (c) approval of the final version. This manuscript has not been submitted to, nor is under review at, another journal or other publishing venue.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alshawi, A.A.A., Tanha, J., Balafar, M.A. et al. A hybrid deep-based model for scene text detection and recognition in meter reading. Int. j. inf. tecnol. 15, 3575–3581 (2023). https://doi.org/10.1007/s41870-023-01383-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41870-023-01383-8

Keywords

Search

Navigation