Abstract
Discovering new intents is a crucial task in dialogue systems. Most existing methods are limited in transferring the prior knowledge from known intents to new intents. These methods also have difficulties in providing high-quality supervised signals to learn clustering-friendly features for grou** unlabeled intents. In this work, we introduce an effective method (Deep Aligned Clustering) to discover new intents with the aid of limited known intent data. Firstly, by leveragin a few labeled known intent samples as prior knowledge to pre-train the model. Then, k-means is performed to produce cluster assignments as pseudo-labels. Moreover, an alignment strategy is proposed to tackle the label inconsistency problem during clustering assignments. Finally, the intent representations are learned under the supervision of the aligned pseudo-labels. With an unknown number of new intents, the number of intent categories is predicted by eliminating low-confidence intent-wise clusters. Extensive experiments on two benchmark datasets show that the method presented is more robust and achieves substantial improvements over the state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Perkins, H., Yang, Y.: Dialog intent induction with deep multi-view clustering. Proceedings of the 58th Conferenceon Empirical Methods in Natural Language Processing, pp. 4016–4025 (2019)
Min, Q.K, Qin, L.B., Teng, Z.Y., et al.: Dialogue State Induction Using Neural Latent Variable Models. Proceedings of the 29th International Joint Conference on Artificial Intelligence, pp. 3845–3852 (2020)
Vedula, N., Lipka, N., Maneriker, P., Parthasarathy, S.: Open intent extraction from natural language interactions. Proceedings of the 29th Web Conference, pp. 2009–2020 (2020)
Platt, J.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv. Large Marg. Classif. 10(3), 61–74 (1999)
Devlin, J., Chang, M.-W., Lee, K., et al.: BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 17th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186 (2019)
Han, K., Vedaldi, A., Zisserman, A.: Learning to discover novel visual categories via deep transfer clustering. Proceedings of the 16th International Conference on Computer Vision, pp. 8400–8408 (2019)
Caron, M., Bojanowski, P., Joulin, A., et al.: Deep clustering for unsupervised learning of visual features. Proceedings of the 15th International Conference on Computer Vision, pp. 132–149 (2018)
Zhan, X., **e, J., Liu, Z., et al.: Online deep clustering for unsupervised representation learning. Proceedings of the 43rd Institute of Electrical and Electronics Engineers Conference on Computer Vision and Pattern Recognition. pp. 6688–6697 (2020)
Kuhn, H.W.: The Hungarian method for the assignment problem. Naval Res. Logist. Quart. 2(1–2), 83–97 (1955)
Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20, 53–65 (1987)
Larson, S., Mahendran, A., Peper, J.J., et al.: An evaluation dataset for intent classification and out-of-scope prediction. Proceedings of the 58th Conferenceon Empirical Methods in Natural Language Processing, pp. 1311–1316 (2019)
Casanueva, I., Temcinas, T., Gerz, D., et al.: Efficient intent detection with dual sentence encoders. Proceedings of the 2nd Workshop on Natural Language Processing for Conversational Artificial Intelligence, pp. 38–45 (2020)
MacQueen, J., et al. Some methods for classification and analysis of multivariate observations. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)
Gowda, K.C., Krishna, G.: Agglomerative clustering using the concept of mutual nearest neighbourhood. Pattern Recogn. 10(2), 105–112 (1978)
**e, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. Proceedings of the 33rd International Conference on Machine Learning, pp. 478–487 (2016)
Schölkopf, B., Platt, J.C., Shawe-Taylor, J., et al.: Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)
Chang, J., Wang, I., Meng, G., et al.: Deep adaptive image clustering. Proceedings of the 15th Institute of Electrical and Electronics Engineers International Conference on Computer Vision, pp. 5879–5887 (2017)
Pennington J, Socher R, Manning C.: Glove: Global vectors for word representation. Proceedings of the 19th Conference on Empirical Methods in Natural Language Processing, pp. 1532–1543 (2014)
Basu, S., Banerjee, A., Mooney, R.J.: Active semi-supervision for pairwise constrained clustering. Proceedings of the 6th Society for Industrial and Applied Mathematics International Conference on Data Mining, pp. 333–344 (2004)
Hsu, Y.C., Lv, Z., Kira, Z.: Learning to cluster in order to transfer across domains and tasks. Proceedings of the 6th International Conference on Learning Representations (2018)
Hsu, Y.-C., Lv, Z., Schlosser, J., Odom, P., Kira, Z.: Multi-class classification without multi-class labels. Proceedings of the 16th International Conference on Computer Vision (2019)
Lin, T.E., Xu, H.: Deep unknown intent detection with margin loss. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5491–5496 (2019)
Thomas, W., Lysandre, D., Victor, S., et al.: Transformers: state-of-the-art natural language processing. Proceedings of the 26th Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online. Association for Computational Linguistics, pp. 38–45 (2020)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Xu, H., Zhang, H., Lin, TE. (2023). Discovering New Intents with Deep Aligned Clustering. In: Intent Recognition for Human-Machine Interactions . SpringerBriefs in Computer Science. Springer, Singapore. https://doi.org/10.1007/978-981-99-3885-8_9
Download citation
DOI: https://doi.org/10.1007/978-981-99-3885-8_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-3884-1
Online ISBN: 978-981-99-3885-8
eBook Packages: Computer ScienceComputer Science (R0)