Abstract
Graph representation learning aims to convert the graph-structured data into a low dimensional space in which the graph structural information and graph properties are maximumly preserved. Graph Neural Networks (GNN)-based methods have shown to be effective in dealing with the graph representation learning task. However, most GNN-based methods belong to supervised learning, which depends heavily on the data labels that are difficult to access in real-world scenarios. In addition, the inherent incompleteness in data will further degrade the performance of GNN-based models. In this paper, we propose a novel self-supervised graph representation learning model with variational inference. First, we strengthen the semantic relation between node and graph level in a self-supervised manner to alleviate the issue of over-dependence on data labels. Second, we utilize the variational inference technique to capture the general pattern underlying the data, thus guaranteeing the model robustness under some data missing circumstances. Extensive experiments on three widely used citation network datasets show that our proposed method has achieved or matched state-of-the-art results on link prediction and node classification tasks.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Our reference code may be found at https://github.com/DLUTElvis/SGRLVI.
References
Bhagat, S., Cormode, G., Muthukrishnan, S.: Node classification in social networks. In: SNDA, pp. 115–148 (2011)
Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations. In: ICLR (2018)
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: SIGKDD, pp. 855–864 (2016)
Grover, A., Zweig, A., Ermon, S.: Graphite: iterative generative modeling of graphs. In: ICML, pp. 2434–2444 (2019)
Hamilton, W.L., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: NeurIPS, pp. 1024–1034 (2017)
Hasanzadeh, A., Hajiramezanali, E., Narayanan, K.R., Duffield, N., Zhou, M., Qian, X.: Semi-implicit graph variational auto-encoders. In: NeurIPS, pp. 10711–10722 (2019)
Hjelm, R.D., et al.: Learning deep representations by mutual information estimation and maximization. In: ICLR (2019)
Hou, Y., Chen, H., Li, C., Cheng, J., Yang, M.: A representation learning framework for property graphs. In: SIGKDD, pp. 65–73 (2019)
Jaiswal, A., Babu, A.R., Zadeh, M.Z., Banerjee, D., Makedon, F.: A survey on contrastive self-supervised learning. CoRR (2020)
Jang, E., Devin, C., Vanhoucke, V., Levine, S.: Grasp2vec: learning object representations from self-supervised gras**. In: CoRL, pp. 99–112 (2018)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: ICLR (2014)
Kipf, T., Welling, M.: Variational graph auto-encoders. arxiv (2016)
Kipf, T., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)
Liu, X., et al.: Self-supervised learning: generative or contrastive. CoRR (2020)
Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. JMLR 9, 2579–2605 (2008)
Meng, Z., Liang, S., Bao, H., Zhang, X.: Co-embedding attributed networks. In: WSDM, pp. 393–401 (2019)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NeurIPS, pp. 3111–3119 (2013)
Namata, G., London, B., Getoor, L., Huang, B.: Query-driven active surveying for collective classification. In: MLG (2012)
Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: SIGKDD, pp. 701–710 (2014)
Schneider, S., Baevski, A., Collobert, R., Auli, M.: wav2vec: unsupervised pre-training for speech recognition. In: ISCA, pp. 3465–3469 (2019)
Sen, P., Namata, G., Bilgic, M., Getoor, L., Gallagher, B., Eliassi-Rad, T.: Collective classification in network data. AI Mag. 29, 93–106 (2008)
Sharma, C., Chauhan, J., Kaul, M.: Learning representations using spectral-biased random walks on graphs. In: IJCNN, pp. 1–8 (2020)
Shijie, J., **, W., Peiyi, J., Si**, H.: Research on data augmentation for image classification based on convolution neural networks. In: CAC, pp. 4165–4170 (2017)
Timofte, R., Rothe, R., Gool, L.V.: Seven ways to improve example-based single image super resolution. In: CVPR, pp. 1865–1873 (2016)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: ICLR (2018)
Velickovic, P., Fedus, W., Hamilton, W.L., Liò, P., Bengio, Y., Hjelm, R.D.: Deep graph infomax. In: ICLR (2019)
Yang, C., Liu, Z., Zhao, D., Sun, M., Chang, E.Y.: Network representation learning with rich text information. In: IJCAI, pp. 2111–2117 (2015)
Acknowledgements
This work was supported by the National Science Foundation of China (Grant No. 61632019), the Fundamental Research Funds for the Central Universities (Grant No. DUT19RC(3)048, DUT20RC(3)040 and DUT20GF106).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liao, Z., Liang, W., Liu, H., Mu, J., Zhang, X. (2021). Self-supervised Graph Representation Learning with Variational Inference. In: Karlapalem, K., et al. Advances in Knowledge Discovery and Data Mining. PAKDD 2021. Lecture Notes in Computer Science(), vol 12714. Springer, Cham. https://doi.org/10.1007/978-3-030-75768-7_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-75768-7_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-75767-0
Online ISBN: 978-3-030-75768-7
eBook Packages: Computer ScienceComputer Science (R0)