Abstract
Text classification is a hot topic in the field of natural language processing and has achieved great success. Existing character-level text classification methods mainly use convolutional neural networks to extract character-level local features, making them ineffective in modeling the hierarchical spatial relationship information on the character-level features, reducing the classification performance. This paper proposes a new character-level text classification framework based on the capsule network called CharCaps to solve the above problem. The proposed CharCaps framework first extracts character-level text features using seven convolutional layers and then reconstructs them based on the capsule vector representation to obtain the hierarchical spatial relationship information between character-level features effectively and achieve a significant classification without pre-trained models. Experimental results on five challenging benchmark datasets demonstrate that our proposed method outperforms state-of-the-art character-level text classification models, especially convolutional neural network-based models.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wan, J., Li, J., Lai, Z., Du, B., Zhang, L.: Robust face alignment by cascaded regression and de-occlusion. Neural Netw. 123, 261–272 (2020)
Wan, J., et al.: Robust facial landmark detection by cross-order cross-semantic deep network. Neural Netw. 136, 233–243 (2021)
Wu, Y., Li, J., Song, C., Chang, J.: Words in pairs neural networks for text classiffcation. Chin. J. Electron. 29, 491–500 (2020)
Sergio, G.C., Lee, M.: Stacked debert: all attention in incomplete data for text classiffcation. Neural Netw. 136, 87–96 (2021)
Kim, Y.: Convolutional Neural Networks for Sentence Classification. In: Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751. ACL, Doha, Qatar (2014)
Liu, P., Qiu, X., Huang, X.: Recurrent Neural Network for Text Classification with Multi-Task Learning. In: 25th International Joint Conference on Artificial Intelligence, pp. 2873–2879. IJCAI/AAAI Press, New York, NY, USA (2016)
Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent Convolutional Neural Networks for Text Classification. In: 29th AAAI Conference on Artificial Intelligence, pp. 2267–2273. AAAI Press, Austin, Texas, USA (2015)
Le, Q.V., Mikolov, T.: Distributed representations of sentences and documents. In: 31th International Conference on Machine Learning, pp. 1188–1196. JMLR, Bei**g, China (2014)
Pennington, J., Socher, R., Manning, C.D.: Contribution title. In: Conference on Empirical Methods in Natural Language Processing, pp. 1532–1543. ACL, Doha, Qatar (2014)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186. ACL, Minneapolis, MN, USA (2019)
Mekala, D., Shang, J.: Contextualized Weak Supervision for Text Classification. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 323–333. ACL, Online (2020)
Croce, D., Castellucci, G., Basili, R.: GAN-BERT: generative adversarial learning for robust text classification with a bunch of labeled examples. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 2114–2119. ACL, Online (2020)
Qin, Q., Hu, W., Liu, B.: Feature projection for improved text classification. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 8161–8171. ACL, Online (2020)
Chen, H., Zheng, G., Ji, Y.: Generating hierarchical explanations on text classification via feature interaction detection. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 5578–5593. ACL, Online (2020)
Zhang, X., Zhao, J., LeCun, Y.: Character-level Convolutional Networks for Text Classification. In: 28th Annual Conference on Neural Information Processing Systems, pp. 649–657. Montreal, Quebec, Canada (2015)
Kim, Y., Jernite, Y., Sontag, D., Rush, A.M.: Character-aware neural language models. In: 9th International Proceedings on Proceedings, pp. 2741–2749. AAAI Press, Phoenix, Arizona, USA (2016)
Liu, B., Zhou, Y., Sun, W.: Character-level text classification via convolutional neural network and gated recurrent unit. Int. J. Mach. Learn. Cybern. 11(8), 1939–1949 (2020). https://doi.org/10.1007/s13042-020-01084-9
Londt, T., Gao, X., Andreae, P.: Evolving character-level densenet architectures using genetic programming. In: Castillo, P.A., JiménezLaredo, J.L. (eds.) Applications of Evolutionary Computation. LNCS, vol. 12694, pp. 665–680. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72699-7_42
Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: 30th Annual Conference on Neural Information Processing, pp. 3856–3866. Long Beach, CA, USA (2017)
Wu, Y., Li, J., Chen, V., Chang, J., Ding, Z., Wang, Z.: Text classification using triplet capsule networks. in: international joint conference on neural networks, pp. 1–7. IEEE, Glasgow, United Kingdom (2020)
Wu, Y., Li, J., Wu, J., Chang, J.: Siamese capsule networks with global and local features for text classification. Neurocomputing 390, 88–98 (2020)
Hong, S.K., Jang, T.: LEA: meta knowledge-driven self-attentive document embedding for few-shot text classification. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 99–106. ACL, Seattle, WA, United States (2022)
Wang, J., et al.: Towards Unified Prompt Tuning for Few-shot Text Classification. In: Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 524–536. Publisher, Abu Dhabi, United Arab Emirates (2022)
Shnarch, E., et al.: Cluster & tune: boost cold start performance in text classification. In: 60th Annual Meeting of the Association for Computational Linguistics, pp. 7639–7653. ACL, Dublin, Ireland (2022)
Tsai, Y.H., Srivastava, N., Goh, H., Salakhutdinov, R.: Capsules with inverted dot-product attention routing. In: 8th International Conference on Learning Representations, ICLR, Addis Ababa, Ethiopia (2020)
Gong, J., Qiu, X., Wang, S., Huang, X.: Information aggregation via dynamic routing for sequence encoding. In: 27th International Conference on Computational Linguistics, pp. 2742–2752. COLING, Santa Fe, New Mexico, USA (2018)
Wang, Y., Sun, A., Han, J., Liu, Y., Zhu, X.: Sentiment analysis by capsules. In: Conference on World Wide Web, pp. 1165–1174. ACM, Lyon, France (2018)
Yang, M., Zhao, W., Chen, L., Qu, Q., Zhao, Z., Shen, Y.: Investigating the transferring capability of capsule networks for text classification. Neural Netw. 118, 247–261 (2019)
Zhao, W., Peng, H., Eger, S., Cambria, E., Yang, M.: Towards scalable and reliable capsule networks for challenging NLP applications. In: 57th Conference of the Association for Computational Linguistic, pp. 1549–1559. ACL, Florence, Italy (2019)
Chen, Z., Qian, T.: Transfer capsule network for aspect level sentiment classification. In: 57th Conference of the Association for Computational Linguistic, pp. 547–556. ACL, Florence, Italy (2019)
Lehmann, J., et al.: DBpedia - a large-scale, multilingual knowledge base extracted from Wikipedia. Semant. Web 6(2), 167–195 (2015)
McAuley, J., Leskovec, J.: Hidden factors and hidden topics: understanding rating dimensions with review text. In: 7th ACM Conference on Recommender Systems, pp. 165–172. ACM, Hong Kong, China (2013)
Rojas, K.R., Bustamante, G., Cabezudo, M.A.S., Oncevay, A.: Efficient Strategies for Hierarchical Text Classification: External Knowledge and Auxiliary Tasks. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 2252–2257. ACL, Online (2020)
Chen, J., Yang, Z., Yang, D.: MixText: linguistically-informed interpolation of hidden space for semi-supervised text classification. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 2147–2157. ACL, Online (2020)
Sinha, K., Dong, Y., Cheung, J.C.K., Ruths, D.: A hierarchical neural attention-based text classifier. In: Conference on Empirical Methods in Natural Language Processing, pp. 817–823. ACL, Brussels, Belgium (2018)
Acknowledgments
This work was Sponsored by Natural Science Foundation of Shanghai (No. 22ZR1445000) and Research Foundation of Shanghai Sanda University (No. 2020BSZX005, No. 2021BSZX006).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wu, Y., Guo, X., Zhan, K. (2023). CharCaps: Character-Level Text Classification Using Capsule Networks. In: Huang, DS., Premaratne, P., **, B., Qu, B., Jo, KH., Hussain, A. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2023. Lecture Notes in Computer Science, vol 14087. Springer, Singapore. https://doi.org/10.1007/978-981-99-4742-3_15
Download citation
DOI: https://doi.org/10.1007/978-981-99-4742-3_15
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-4741-6
Online ISBN: 978-981-99-4742-3
eBook Packages: Computer ScienceComputer Science (R0)