Abstract
Knowledge graph embedding has become a promising method for knowledge graph completion. It aims to learn low-dimensional embeddings in continuous vector space for each entity and relation. It remains challenging to learn accurate embeddings for complex multi-relational facts. In this paper, we propose a new translation-based embedding method named ATransD-NL to address the following two observations. First, most existing translational methods do not consider contextual information that have been proved useful for improving performance of link prediction. Our method learns attention-based embeddings for each triplet taking into account influence of one-hop or potentially multi-hop neighbourhood entities. Second, we apply nonlinear dynamic projection of head and tail entities to relational space, to capture nonlinear correlations among entities and relations due to complex multi-relational facts. As an extension of TransD, our model only introduces one more extra parameter, giving a good tradeoff between model complexity and the state-of-the-art predictive accuracy. Compared with state-of-the-art translation-based methods and the neural-network based methods, experiment results show that our method delivers substantial improvements over baselines on the MeanRank metric of link prediction, e.g., an improvement of 35.6% over the attention-based graph embedding method KBGAT and an improvement of 64% over the translational method TransMS on WN18 database, with comparable performance on the Hits@10 metric.
Z. Wang and H. Xu are co-first authors. This work was supported in part by the Ministry of Science and Technology of China under Grant No. 2018YFC0830400, National Natural Science Foundation of China under Grant No. 61802126, 61832015, 62072176, the Inria-CAS joint project Quasar and Shanghai Pujiang Program under Grant No. 17PJ1402200.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250 (2008)
Bordes, A., Glorot, X., Weston, J., Bengio, Y.: A semantic matching energy function for learning with multi-relational data. Mach. Learn. 94(2), 233–259 (2013). https://doi.org/10.1007/s10994-013-5363-6
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)
Bordes, A., Weston, J., Collobert, R., Bengio, Y.: Learning structured embeddings of knowledge bases. In: Proceedings of AAAI 2011, pp. 301–306 (2011)
Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Proceedings of AAAI 2018, pp. 1811–1818 (2018)
Fabian, M., Gjergji, K., Gerhard, W., et al.: Yago: a core of semantic knowledge unifying Wordnet and Wikipedia. In: Proceedings of 16th International World Wide Web Conference, pp. 697–706 (2007)
Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic map** matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (volume 1: Long Papers), pp. 687–696 (2015)
Ji, G., Liu, K., He, S., Zhao, J.: Knowledge graph completion with adaptive sparse transfer matrix. In: Proceedings of AAAI 2016, pp. 985–991 (2016)
Lehmann, J., et al.: DBpedia-a large-scale, multilingual knowledge base extracted from Wikipedia. Semant. Web 6(2), 167–195 (2015)
Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., Liu, S.: Modeling relation paths for representation learning of knowledge bases. ar**v preprint ar**v:1506.00379 (2015)
Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of AAAI 2015, pp. 2181–2187 (2015)
Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)
Nathani, D., Chauhan, J., Sharma, C., Kaul, M.: Learning attention-based embeddings for relation prediction in knowledge graphs. ar**v preprint ar**v:1906.01195 (2019)
Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., Phung, D.: A novel embedding model for knowledge base completion based on convolutional neural network. ar**v preprint ar**v:1712.02121 (2017)
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38
Shang, C., Tang, Y., Huang, J., Bi, J., He, X., Zhou, B.: End-to-end structure-aware convolutional networks for knowledge base completion. In: Proceedings of the AAAI 2019, vol. 33, pp. 3060–3067 (2019)
Trouillon, T., Welbl, J., Riedel, S., Gaussier, E., Bouchard, G.: Complex embeddings for simple link prediction. In: Proceedings of ICML 2016, pp. 2071–2080 (2016)
Vu, T., Nguyen, T.D., Nguyen, D.Q., Phung, D., et al.: A capsule network-based embedding model for knowledge graph completion and search personalization. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2180–2189 (2019)
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of AAAI 2014, pp. 1112–1119 (2014)
**ao, H., Huang, M., Hao, Y., Zhu, X.: TransA: an adaptive approach for knowledge graph embedding. ar**v preprint ar**v:1509.05490 (2015)
**ao, H., Huang, M., Hao, Y., Zhu, X.: TransG: a generative mixture model for knowledge graph embedding. ar**v preprint ar**v:1509.05488 (2015)
Yang, B., Yih, W.t., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. ar**v preprint ar**v:1412.6575 (2014)
Yang, S., Tian, J., Zhang, H., Yan, J., He, H., **, Y.: TransMS: knowledge graph embedding for complex relations by multidirectional semantics. In: Proceedings of IJCAI 2019, pp. 1935–1942 (2019)
Author information
Authors and Affiliations
Corresponding author
Download citation
DOI: https://doi.org/10.1007/978-3-030-75768-7_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-75767-0
Online ISBN: 978-3-030-75768-7
eBook Packages: Computer ScienceComputer Science (R0)