Abstract
This article introduces a recurrent CNN based framework for the classification of arbitrary length text in natural sentence. In our model, we present a complete CNN design with recurrent structure to capture the contextual information as far as possible when learning sentences, which allows arbitrary-length sentences and more flexibility to analyze complete sentences compared with traditional CNN based neural networks. In addition, our model greatly reduces the number of layers in the architecture and requires fewer training parameters, which leads to less memory consumption, and it can reach \(O\left( \log n\right) \) time complexity. As a result, this model can achieve enhancement in training accuracy. Moreover, the design and implementation can be easily deployed in the current text classification systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bengio, Y., Courville, A.C., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Conneau, A., Schwenk, H., Barrault, L., LeCun, Y.: Very deep convolutional networks for text classification. In: EACL, vol. 1, pp. 1107–1116. Association for Computational Linguistics (2017)
Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)
Georgakopoulos, S.V., Tasoulis, S.K., Vrahatis, A.G., Plagianakos, V.P.: Convolutional neural networks for toxic comment classification. In: SETN, pp. 35:1–35:6. ACM (2018)
Guggilla, C., Miller, T., Gurevych, I.: CNN- and lSTM-based claim classification in online user comments. In: COLING, pp. 2740–2751. ACL (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Huang, Q., Chen, R., Zheng, X., Dong, Z.: Deep sentiment representation based on CNN and lSTM. In: 2017 International Conference on Green Informatics (ICGI), pp. 30–33. IEEE (2017)
Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: ACL, vol. 1, pp. 655–665. The Association for Computer Linguistics (2014)
Kim, Y.: Convolutional neural networks for sentence classification. In: EMNLP, pp. 1746–1751. ACL (2014)
Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: AAAI, pp. 2267–2273. AAAI Press (2015)
Lakshmi, B.S., Raj, P.S., Vikram, R.R., et al.: Sentiment analysis using deep learning technique CNN with K means. Int. J. Pure Appl. Math. 114(11), 47–57 (2017)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Liang, M., Hu, X.: Recurrent convolutional neural network for object recognition. In: CVPR, pp. 3367–3375. IEEE Computer Society (2015)
Liu, B.: Sentiment Analysis and Opinion Mining. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers, Vermont (2012)
Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: ACL, pp. 142–150. The Association for Computer Linguistics (2011)
Mikolov, T.: Statistical language models based on neural networks. In: Presentation at Google, Mountain View, 2 April 2012
Mikolov, T., Karafiát, M., Burget, L., Cernocký, J., Khudanpur, S.: Recurrent neural network based language model. In: INTERSPEECH, pp. 1045–1048. ISCA (2010)
Nasukawa, T., Yi, J.: Sentiment analysis: capturing favorability using natural language processing. In: K-CAP, pp. 70–77. ACM (2003)
Paszke, A., et al.: Automatic differentiation in PyTorch (2017)
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: EMNLP, pp. 1532–1543. ACL (2014)
Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: EMNLP, pp. 1631–1642. ACL (2013)
Wang, J., Yu, L., Lai, K.R., Zhang, X.: Dimensional sentiment analysis using a regional CNN-LSTM model. In: ACL, vol. 2. The Association for Computer Linguistics (2016)
Wang, Y., Huang, M., Zhu, X., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: EMNLP, pp. 606–615. The Association for Computational Linguistics (2016)
Yih, W., He, X., Meek, C.: Semantic parsing for single-relation question answering. In: ACL, vol. 2, pp. 643–648. The Association for Computer Linguistics (2014)
Zhang, M., Zhang, Y., Vo, D.: Gated neural networks for targeted sentiment analysis. In: AAAI, pp. 3087–3093. AAAI Press (2016)
Zhou, Z., Chen, S., Chen, Z.: FANNC: a fast adaptive neural network classifier. Knowl. Inf. Syst. 2(1), 115–129 (2000)
Acknowledgment
The article is part of the research project funded by The Science and Technology Development Fund, Macau SAR (File no. 0001/2018/AFJ).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Chan, KH., Im, SK., Ke, W. (2020). Variable-Depth Convolutional Neural Network for Text Classification. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1333. Springer, Cham. https://doi.org/10.1007/978-3-030-63823-8_78
Download citation
DOI: https://doi.org/10.1007/978-3-030-63823-8_78
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63822-1
Online ISBN: 978-3-030-63823-8
eBook Packages: Computer ScienceComputer Science (R0)