SpringerLink
Log in

Simple hierarchical PageRank graph neural networks

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Graph neural networks (GNNs) have many variants for graph representation learning. Several works introduce PageRank into GNNs to improve its neighborhood aggregation capabilities. However, these methods leverage the general PageRank to perform complex neighborhood aggregation to obtain the final feature representation, which leads to high computational cost and oversmoothing. In this paper, we propose simple hierarchical PageRank graph neural networks (SHP-GNNs), which first utilize the simple PageRank to aggregate different neighborhood ranges of each node and then leverage a jum** architecture to combine these aggregated features to enable hierarchical structure-aware representation. In this case, first, the simple PageRank turns the neighborhood aggregation process to no-learning, thereby reducing the computational complexity of the model. Then, the jum** structure combines the aggregation features of each node’s different hierarchy (neighborhood range) to learn more informative feature representation. Finally, the successful combination of the above methods alleviates the oversmoothing problem of deep GNNs. Our experimental evaluation demonstrates that SHP-GNNs achieve or match state-of-the-art results in node classification tasks, text classification tasks, and community prediction tasks. Moreover, since SHP-GNNs’ neighborhood aggregation is a no-learning process, SHP-GNNs are more suitable for node clustering tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

This declaration is “not applicable.” Data will be made available on request.

Notes

  1. https://github.com/youngflyasd/SHP-GNN.

References

  1. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations

  2. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: 6th International Conference on Learning Representations

  3. Zhang X, Liu H, Li Q, Wu XM (2019) Attributed graph clustering via adaptive graph convolution. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pp 4327–4333

  4. Zhu H, Koniusz P (2021) Simple spectral graph convolution. In: 9th International Conference on Learning Representations

  5. Roy I, De A, Chakrabarti S (2021) Adversarial permutation guided node representations for link prediction. Proc AAAI Conf Artif Intell 35:9445–9453

    Google Scholar 

  6. Zhang Q, Wang R, Yang J, Xue L (2021) Knowledge graph embedding by translating in time domain space for link prediction. Knowl-Based Syst. https://doi.org/10.1016/j.knosys.2020.106564

    Article  PubMed  PubMed Central  Google Scholar 

  7. Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. In: 2th International Conference on Learning Representations

  8. Atwood J, Towsley D (2015) Deep convolutional networks on graph-structured data. ar** knowledge networks. In: Proceedings of the 35th International Conference on Machine Learning

  9. Rong Y, Huang W, Xu T, Huang J (2020) Dropedge: towards deep graph convolutional network on node classification. In: Proceedings of the 37th International Conference on Machine Learning

  10. Chen M, Wei Z, Huang Z, Ding B, Li Y (2020) Simple and deep graph convolutional networks. In: Proceedings of the 37th International Conference on Machine Learning, pp 1725–1735

  11. Li R, Wang S, Zhu F, Huang J (2018) Adaptive graph convolutional neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence

  12. NT H, Maehara T (2019) Revisiting graph neural networks: all we have is low-pass filters. ar**v:1905.09550

  13. Sun K, Zhu Z, Lin Z (2021) Adagcn: adaboosting graph convolutional networks into deep models. In: 9th International Conference on Learning Representations

  14. Duvenaud D, Maclaurin D, Aguilera-Iparraguirre J, Gomez-Bombarelli R, Hirzel T, Aspuru-Guzik A, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints. In: Advances in Neural Information Processing Systems, pp 2224–2232

  15. Atwood J, Towsley D (2016) Diffusion–convolutional neural networks. In: Advances in Neural Information Processing Systems, pp 1993–2001

  16. Niepert M, Ahmed M, Kutzkov K (2016) Learning convolutional neural networks for graphs. In: Proceedings of The 33rd International Conference on Machine Learning, vol 48, pp 2014–2023

  17. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, pp 1024–1034

  18. Monti F, Boscaini D, Masci J, Rodola E, Svoboda J, Bronstein MM (2017) Geometric deep learning on graphs and manifolds using mixture model CNNs. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Honolulu, HI, pp 5425–5434. https://doi.org/10.1109/CVPR.2017.576

  19. Gao H, Wang Z, Ji S (2018) Large-scale learnable graph convolutional networks. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, London United Kingdom, pp 1416–1424. https://doi.org/10.1145/3219819.3219947

  20. Chen J, Ma T, **ao C (2018) FastGCN: fast learning with graph convolutional networks via importance sampling. In: 6th International Conference on Learning Representations

  21. Huang W, Zhang T, Rong Y, Huang J (2018) Adaptive sampling towards fast graph representation learning. In: Advances in Neural Information Processing Systems

  22. Thekumparampil KK, Wang C, Oh S, Li LJ (2018) Attention-based graph neural network for semi-supervised learning. ar**v:1803.03735

  23. Lee J, Lee I, Kang J (2019) Self-attention graph pooling. In: Proceedings of the 36th International Conference on Machine Learning

  24. Abu-El-Haija S, Perozzi B, Kapoor A, Alipourfard N, Lerman K, Harutyunyan H, Steeg GV, Galstyan A (2019) MixHop: higher-order graph convolutional architectures via sparsified neighborhood mixing. In: Proceedings of the 36th International Conference on Machine Learning, pp 21–29

  25. Zhang K, Zhu Y, Wang J, Zhang J (2020) Adaptive structural fingerprints for graph attention networks. In: 8th International Conference on Learning Representations

  26. Chiang WL, Liu X, Si S, Li Y, Bengio S, Hsieh CJ (2019) Cluster-GCN: an efficient algorithm for training deep and large graph convolutional networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, Anchorage, AK, USA, pp 257–266. https://doi.org/10.1145/3292500.3330925

  27. Li G, Muller M, Thabet A, Ghanem B (2019) DeepGCNs: can GCNs go as deep as CNNs? In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, Seoul, Korea (South), pp 9266–9275. https://doi.org/10.1109/ICCV.2019.00936

  28. Loukas A (2020) What graph neural networks cannot learn: depth vs width. In: 8th International Conference on Learning Representations

  29. Oono K, Suzuki T (2020) Graph neural networks exponentially lose expressive power for node classification. In: 8th International Conference on Learning Representations

  30. Zhou K, Huang X, Li Y, Zha D, Chen R, Hu X (2020) Towards deeper graph neural networks with differentiable group normalization. In: Advances in Neural Information Processing Systems, vol 33

  31. Li Q, Han Z, Wu XM (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 3538–3545

  32. Zhou J, Cui G, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M (2019) Graph neural networks: a review of methods and applications. In: 7th International Conference on Learning Representations

  33. Chen L, Wu L, Hong R, Zhang K, Wang M (2020) Revisiting graph based collaborative filtering: a linear residual graph convolutional network approach. Proc AAAI Conf Artif Intell 34:27–34. https://doi.org/10.1609/aaai.v34i01.5330

    Article  Google Scholar 

  34. Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2021) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24. https://doi.org/10.1109/TNNLS.2020.2978386

    Article  MathSciNet  PubMed  Google Scholar 

  35. Veličković P, Fedus W, Hamilton WL, Liò P, Bengio Y, Hjelm RD (2019) Deep graph infomax. In: 7th International Conference on Learning Representations

  36. Xu B, Shen H, Cao Q, Qiu Y, Cheng X (2019) Graph wavelet neural network. In: 7th International Conference on Learning Representations

  37. Zhu H, Feng F, He X, Wang X, Li Y, Zheng K, Zhang Y (2020) Bilinear graph neural network with neighbor interactions. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence Organization, Yokohama, Japan, vol 5, pp 1452–1458. https://doi.org/10.24963/ijcai.2020/202

  38. Gao H, Ji S (2019) Graph U-Nets. In: Proceedings of the 36th International Conference on Machine Learning

  39. Bianchi FM, Grattarola D, Alippi C (2020) Spectral clustering with graph neural networks for graph pooling. In: Proceedings of the 37th International Conference on Machine Learning

  40. Ying R, You J, Morris C, Ren X, Hamilton WL, Leskovec J (2018) Hierarchical graph representation learning with differentiable pooling. In: Advances in Neural Information Processing Systems

  41. Cai D, Lam W (2020) Graph transformer for graph-to-sequence learning. In: Proceedings of the AAAI Conference on Artificial Intelligence

  42. Xhonneux LPAC, Qu M, Tang J (2020) Continuous graph neural networks. In: Proceedings of the 37th International Conference on Machine Learning, pp 10432–10411

  43. Avelar PHC, Tavares AR, Gori M, Lamb LC (2021) Discrete and continuous deep residual learning over graphs. In: Proceedings of the 13th International Conference on Agents and Artificial Intelligence, vol 2, pp 119–131. https://doi.org/10.5220/0010231501190131

  44. Chung FR, Graham FC (1997) Spectral graph theory. American Mathematical Soc

  45. Chen Z, Wu Z, Lin Z, Wang S, Plant C, Guo W (2023) AGNN: alternating graph-regularized neural networks to alleviate over-smoothing. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2023.3271623

    Article  PubMed  Google Scholar 

  46. Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93. https://doi.org/10.1609/aimag.v29i3.2157

    Article  Google Scholar 

  47. Yang Z, Cohen WW, Salakhutdinov R (2016) Revisiting semi-supervised learning with graph embeddings. In: Proceedings of the 33th International Conference on Machine Learning, pp 40–48

  48. Pei H, Wei B, Chang KCC, Lei Y, Yang B (2020) Geom-GCN: geometric graph convolutional networks. In: 8th International Conference on Learning Representations

  49. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: 3th International Conference on Learning Representations

  50. Xu K, Hu W, Leskovec J, Jegelka S (2019) How powerful are graph neural networks? In: 7th International Conference on Learning Representations

  51. Li Q, Wu XM, Liu H, Zhang X, Guan Z (2019) Label efficient semi-supervised learning via graph filtering. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Long Beach, CA, USA, pp 9574–9583. https://doi.org/10.1109/CVPR.2019.00981

  52. Perozzi B, Al-Rfou R, Skiena S (2014) DeepWalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM Press, New York, New York, USA, pp 701–710. https://doi.org/10.1145/2623330.2623732

  53. Cao S, Lu W, Xu Q (2016) Deep neural networks for learning graph representations. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 1145–1152

  54. Kipf TN, Welling M (2016) Variational graph auto-encoders. In: NIPS Workshop on Bayesian Deep Learning

  55. Wang C, Pan S, Long G, Zhu X, Jiang J (2017) MGAE: marginalized graph autoencoder for graph clustering. In: Proceedings of the 2017 ACM Conference on Information and Knowledge Management, pp 889–898

  56. Pan S, Hu R, Long G, Jiang J, Yao L, Zhang C (2018) Adversarially regularized graph autoencoder for graph embedding. In: Proceedings of the Twenty-Seven International Joint Conference on Artificial Intelligence, pp 2609–2615

  57. Liu DC, Nocedal J (1989) On the limited memory BFGS method for large scale optimization. Math Program 45(1–3):503–528. https://doi.org/10.1007/BF01589116

    Article  MathSciNet  Google Scholar 

  58. Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence

  59. Zhu JJ, Kouridi C, Nemmour Y, Schölkopf B (2022) Adversarially robust kernel smoothing. ar**v:2102.08474

  60. Xu Z, ** R, Zhu S, Lyu M, King I (2010) Smooth optimization for effective multiple kernel learning. Proc AAAI Conf Artif Intell 24:637–642. https://doi.org/10.1609/aaai.v24i1.7675

    Article  Google Scholar 

  61. Dai B, Wang J, Shen X, Qu A (2019) Smooth neighborhood recommender systems. J Mach Learn Res 20(16):1–24

    MathSciNet  Google Scholar 

  62. Ji S, Sun L, ** R, Ye J (2008) Multi-label multiple kernel learning. In: Advances in Neural Information Processing Systems, pp 777–784

Download references

Funding

This document is the results of the research project funded by the National Natural Science Foundation of China (Grant No. 61772386).

Author information

Authors and Affiliations

  1. China North Communication Technology Co., Ltd, Wuhan Branch, Wuhan, China

    Fei Yang

  2. School of Computer Science, Wuhan University, Wuhan, China

    Huyin Zhang & Shiming Tao

  3. School of Computer and Communication Engineering, University of Science and Technology Bei**g, Bei**g, China

    **ying Fan

Authors
  1. Fei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huyin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiming Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. **ying Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

FY contributed to methodology, modeling, coding, and writing. HZ was involved in supervision. ST provided software and contributed to coding. XF was involved in data and visualization.

Corresponding author

Correspondence to Fei Yang.

Ethics declarations

Conflict of interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, and there is no professional or other personal interest of any nature or kind in any product, service, and/or company that could be construed as influencing that position in, or the review of, the manuscript entitled.

Ethical approval

This declaration is “not applicable.”

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, F., Zhang, H., Tao, S. et al. Simple hierarchical PageRank graph neural networks. J Supercomput 80, 5509–5539 (2024). https://doi.org/10.1007/s11227-023-05666-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05666-6

Keywords

Search

Navigation