Abstract
This paper proposes a transfer learning algorithm for end-to-end dialogue state tracking (DST) to handle new slots with a small set of training data, which has not yet been discussed in the literature on conventional approaches. The goal of transfer learning is to improve DST performance for new slots by leveraging slot-independent parameters extracted from DST models for existing slots. An end-to-end DST model is composed of a spoken language understanding module and an update module. We assume that parameters of the update module can be slot-independent. To make the parameters slot-independent, a DST model for each existing slot is trained by sharing the parameters of the update module across all existing slots. The slot-independent parameters are transferred to a DST model for the new slot. Experimental results show that the proposed algorithm achieves 82.5% accuracy on the DSTC2 dataset, outperforming a baseline algorithm by 1.8% when applied to a small set of training data. We also show its potential robustness for the network architecture of update modules.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wang Z, Lemon O (2013) A simple and generic belief tracking mechanism for the dialog state tracking challenge: on the believability of observed information. In: Proceedings of the SIGDIAL 2013 conference, pp 423–432
Williams JD (2014) Web-style ranking and slu combination for dialog state tracking. In: Proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (SIGDIAL), pp 282–291
Sun K, Chen L, Zhu S, Yu K (2014) A generalized rule based tracker for dialogue state tracking. In: 2014 IEEE spoken language technology workshop (SLT). IEEE, pp 330–335
Perez J, Liu F (2017) Dialog state tracking, a machine reading approach using memory network. In: Proceedings of the 15th conference of the European chapter of the association for computational linguistics, vol 1, pp 305–314
Rastogi A, Hakkani-Tür D, Heck L (2017) Scalable multi-domain dialogue state tracking. In: 2017 IEEE automatic speech recognition and understanding workshop (ASRU). IEEE, pp 561–568
Henderson M, Thomson B, Young S (2014) Word-based dialog state tracking with recurrent neural networks. In: Proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (SIGDIAL), Philadelphia, PA, U.S.A., June 2014, Association for Computational Linguistics, pp 292–299
Henderson M, Thomson B, Young S (2014) Robust dialog state tracking using delexicalised recurrent neural networks and unsupervised adaptation. In: 2014 IEEE spoken language technology workshop (SLT). IEEE, pp 360–365
Mrkšić N, Séaghdha DÓ, Thomson B, Gasic M, Su P-H, Vandyke D, Wen T-H, Young S (2015) Multi-domain dialog state tracking using recurrent neural networks. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, vol 2, pp 794–799
Lukas Z, Filip J (2015) Incremental lstm-based dialog state tracker. In: IEEE workshop on automatic speech recognition and understanding (ASRU). IEEE, pp 757–762
Yoshida T, Iwata K, Fujimura H, Akamine M (2018) Dialog state tracking for unseen values using an extended attention mechanism. In: Proceedings of the 8th international workshop on spoken dialogue systems (IWSDS)
Mrkšić N, Vulić I (2018) Fully statistical neural belief tracking. In: Proceedings of the 56th annual meeting of the association for computational linguistics, vol 2, pp 108–113
Xu P, Hu Q (2018) An end-to-end approach for handling unknown slot values in dialogue state tracking. In: Proceedings of the 56th annual meeting of the association for computational linguistics
Zhong V, **ong C, Socher R (2018) Global-locally self-attentive dialogue state tracker. In: Proceedings of the 56th annual meeting of the association for computational linguistics
Ramadan O, Budzianowski P, Gašić M (2018) Large-scale multi-domain belief tracking with knowledge sharing. In: Proceedings of the 56th annual meeting of the association for computational linguistics
Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359
Dong W, Fang ZT (2015) Transfer learning for speech and language processing. In: 2015 Asia-Pacific Signal and information processing association annual summit and conference (APSIPA). IEEE, pp. 1225–1237
Passban P, Liu Q, Way A (2017) Translating low-resource languages by vocabulary adaptation from close counterparts. ACM Trans Asian Low-Resour Lang Inf Process (TALLIP) 16(4):29
Yu J, Qiu M, Jiang J, Huang J, Song S, Chu W, Chen H (2018) Modelling domain relationships for transfer learning on retrieval-based question answering systems in e-commerce. In: Proceedings of the 11th ACM international conference on web search and data mining. ACM, pp 682–690
Lee JY, Dernoncourt F, Szolovits P (2018) Transfer learning for named-entity recognition with neural networks. In: 11th international conference on language resources and evaluation (LREC)
Jeong M, Lee GG (2009) Multi-domain spoken language understanding with transfer learning. Speech Commun 51(5):412–424
Kim Y-B, Stratos K, Sarikaya R, Jeong M (2015) New transfer learning techniques for disparate label sets. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, vol 1, pp 473–482
Bapna A, Tur G, Hakkani-Tur D, Heck L (2017) Towards zero-shot frame semantic parsing for domain scaling. In: Proceedings of the interspeech, pp 2476–2480
Henderson M, Thomson B, Williams J (2014) The second dialog state tracking challenge. In: 15th annual meeting of the special interest group on discourse and dialogue, vol 263
Tokui S, Oono K, Hido S, Clayton J (2015) Chainer: a next-generation open source framework for deep learning. In: Proceedings of workshop on machine learning systems (LearningSys) in the twenty-ninth annual conference on neural information processing systems (NIPS)
Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Empirical methods in natural language processing (EMNLP), pp 1532–1543
Iyyer M, Manjunatha V, Boyd-Graber J, Daumé III H (2015) Deep unordered composition rivals syntactic methods for text classification. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, vol 1, pp 1681–1691
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Iwata, K., Yoshida, T., Fujimura, H., Akamine, M. (2021). Transfer Learning for Unseen Slots in End-to-End Dialogue State Tracking. In: Marchi, E., Siniscalchi, S.M., Cumani, S., Salerno, V.M., Li, H. (eds) Increasing Naturalness and Flexibility in Spoken Dialogue Interaction. Lecture Notes in Electrical Engineering, vol 714. Springer, Singapore. https://doi.org/10.1007/978-981-15-9323-9_5
Download citation
DOI: https://doi.org/10.1007/978-981-15-9323-9_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-9322-2
Online ISBN: 978-981-15-9323-9
eBook Packages: EngineeringEngineering (R0)