SpringerLink
Log in

Deep JEDi: Deep Joint Entity Disambiguation to Wikipedia for Russian

  • Conference paper
  • First Online:
Analysis of Images, Social Networks and Texts (AIST 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11832))

  • 1029 Accesses

Abstract

Over the past few years there has been a leap forward in both Entity Disambiguation and Entity Linking tasks. Meanwhile, Entity Disambiguation for Russian still lags behind advanced neural approaches developed for other languages. This paper introduces Deep JEDi—purely neural architecture, intended to identify the correct meaning for each mention in text. Combining sequence translation and sequence labeling approaches, our model achieves promising results on the Russian Wikipedia dataset. Significant improvement of its performance is attained by specific decoder that incorporates information about target mention position into attention mechanism. Additionally, we compare different approaches for learning distributed representations for tokens and entities and prove the importance of enriching joint embeddings with information about knowledge base structure.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Thailand)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 42.79
Price includes VAT (Thailand)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 49.99
Price excludes VAT (Thailand)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Hereinafter examples are taken from Wikipedia.

  2. 2.

    Typewriter font is used for Russian example, italics—for transliteration, —for English version.

References

  1. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: Proceedings of 3rd International Conference for Learning Representations, pp. 1–15 (2015)

    Google Scholar 

  2. Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)

    Article  Google Scholar 

  3. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)

    Google Scholar 

  4. Francis-Landau, M., Durrett, G., Klein, D.: Capturing semantic similarity for entity linking with convolutional neural networks. In: Proceedings of NAACL-HLT, pp. 1256–1261 (2016)

    Google Scholar 

  5. Ganea, O.E., Hofmann, T.: Deep joint entity disambiguation with local neural attention. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2619–2629. Association for Computational Linguistics (2017)

    Google Scholar 

  6. Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N.: Convolutional sequence to sequence learning. In: International Conference on Machine Learning, pp. 1243–1252 (2017)

    Google Scholar 

  7. Gers, F.A., Schmidhuber, J., Cummins, F.: Learning to forget: continual prediction with LSTM. Neural Comput. 12(10), 2451–2471 (2000)

    Article  Google Scholar 

  8. Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864. ACM (2016)

    Google Scholar 

  9. Guo, X., Zhang, H., Yang, H., Xu, L., Ye, Z.: A single attention-based combination of CNN and RNN for relation classification. IEEE Access (2019)

    Google Scholar 

  10. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  11. Huang, H., Heck, L.P., Ji, H.: Leveraging deep neural networks and knowledge graphs for entity disambiguation. CoRR abs/1504.07678 (2015)

    Google Scholar 

  12. Le, P., Titov, I.: Improving entity linking by modeling latent relations between mentions. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1595–1604. Association for Computational Linguistics, Melbourne, July 2018

    Google Scholar 

  13. LeCun, Y.: Generalization and network design strategies. In: Connectionism in Perspective, vol. 19. Citeseer (1989)

    Google Scholar 

  14. Li, J., Cai, Y., Cai, Z., Leung, H., Yang, K.: Wikipedia based short text classification method. In: Bao, Z., Trajcevski, G., Chang, L., Hua, W. (eds.) DASFAA 2017. LNCS, vol. 10179, pp. 275–286. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-55705-2_22

    Chapter  Google Scholar 

  15. Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1412–1421 (2015)

    Google Scholar 

  16. Mihalcea, R., Csomai, A.: Wikify!: linking documents to encyclopedic knowledge. In: Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, pp. 233–242. ACM (2007)

    Google Scholar 

  17. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. ar**v preprint ar**v:1301.3781 (2013)

  18. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 26, pp. 3111–3119. Curran Associates, Inc. (2013)

    Google Scholar 

  19. Milne, D., Witten, I.H.: Learning to link with Wikipedia. In: Proceedings of the 17th ACM Conference on Information and Knowledge Management, pp. 509–518. ACM (2008)

    Google Scholar 

  20. Nothman, J., Ringland, N., Radford, W., Murphy, T., Curran, J.R.: Learning multilingual named entity recognition from Wikipedia. Artif. Intell. 194, 151–175 (2013)

    Article  MathSciNet  Google Scholar 

  21. Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

    Google Scholar 

  22. Ratinov, L., Roth, D., Downey, D., Anderson, M.: Local and global algorithms for disambiguation to Wikipedia. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 1375–1384. Association for Computational Linguistics (2011)

    Google Scholar 

  23. Sergeev, A., Balso, M.D.: Horovod: fast and easy distributed deep learning in TensorFlow. ar**v preprint ar**v:1802.05799 (2018)

  24. Socher, R., Chen, D., Manning, C.D., Ng, A.: Reasoning with neural tensor networks for knowledge base completion. In: Advances in Neural Information Processing Systems, pp. 926–934 (2013)

    Google Scholar 

  25. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)

    Google Scholar 

  26. Sysoev, A., Andrianov, I.: Named entity recognition in Russian: the power of wiki-based approach. In: Proceedings of International Conference “Dialogue-2016”, pp. 746–755 (2016)

    Google Scholar 

  27. Sysoev, A., Nikishina, I.: Smart context generation for disambiguation to Wikipedia. In: Ustalov, D., Filchenkov, A., Pivovarova, L., Žižka, J. (eds.) AINL 2018. CCIS, vol. 930, pp. 11–22. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01204-5_2

    Chapter  Google Scholar 

  28. Turdakov, D., et al.: Semantic analysis of texts using Texterra system (2014). http://www.dialog-21.ru/digests/dialog2014/materials/pdf/TurdakovDY.pdf

  29. Wang, X., Jiang, W., Luo, Z.: Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In: Proceedings of COLING 2016, 26th International Conference on Computational Linguistics: Technical Papers, pp. 2428–2437 (2016)

    Google Scholar 

  30. Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph and text jointly embedding. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1591–1601 (2014)

    Google Scholar 

  31. Yamada, I., Shindo, H., Takeda, H., Takefuji, Y.: Joint learning of the embedding of words and entities for named entity disambiguation. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pp. 250–259. Association for Computational Linguistics, Berlin, Germany, August 2016

    Google Scholar 

  32. Yamada, I., Shindo, H., Takeda, H., Takefuji, Y.: Learning distributed representations of texts and entities from knowledge base. Trans. Assoc. Comput. Linguist. 5, 397–411 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ivannikov Institute for System Programming of the Russian Academy of Sciences, Moscow, Russia

    Andrey Sysoev & Irina Nikishina

  2. Higher School of Economics, Moscow, Russia

    Irina Nikishina

Authors
  1. Andrey Sysoev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Irina Nikishina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrey Sysoev .

Editor information

Editors and Affiliations

  1. RWTH Aachen University, Aachen, Germany

    Wil M. P. van der Aalst

  2. University of Ljubljana, Ljubljana, Slovenia

    Vladimir Batagelj

  3. National Research University Higher School of Economics, Moscow, Russia

    Dmitry I. Ignatov

  4. Krasovskii Institute of Mathematics and Mechanics, Yekaterinburg, Russia

    Michael Khachay

  5. National Research University Higher School of Economics, Moscow, Russia

    Valentina Kuskova

  6. University of Oslo, Oslo, Norway

    Andrey Kutuzov

  7. National Research University Higher School of Economics, Moscow, Russia

    Sergei O. Kuznetsov

  8. National Research University Higher School of Economics, Moscow, Russia

    Irina A. Lomazova

  9. Lomonosov Moscow State University, Moscow, Russia

    Natalia Loukachevitch

  10. LORIA, Vandœuvre-lès-Nancy, France

    Amedeo Napoli

  11. University of Florida, Gainesville, FL, USA

    Panos M. Pardalos

  12. Ca Foscari University of Venice, Venice, Italy

    Marcello Pelillo

  13. National Research University Higher School of Economics, Nizhny Novgorod, Russia

    Andrey V. Savchenko

  14. Kazan Federal University, Kazan, Russia

    Elena Tutubalina

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sysoev, A., Nikishina, I. (2019). Deep JEDi: Deep Joint Entity Disambiguation to Wikipedia for Russian. In: van der Aalst, W., et al. Analysis of Images, Social Networks and Texts. AIST 2019. Lecture Notes in Computer Science(), vol 11832. Springer, Cham. https://doi.org/10.1007/978-3-030-37334-4_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37334-4_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37333-7

  • Online ISBN: 978-3-030-37334-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation