SpringerLink
Log in

Heterogeneous graph convolutional network pre-training as side information for improving recommendation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

For the recommendation domain, most of the existing integrated graph neural network (GNN)-based architectures have still much focused on encoding the associated extra side information in forms of heterogeneous information network (HIN). Then, it is simultaneously utilized in multiple fine-tuning processes to effectively learn preferences from user–item interaction data which might require tremendous computational efforts. In addition, these approaches have also failed to incorporate with previous learnt and transferable knowledge from pre-trained models to better fine-tune for recommendation task. To meet these challenges, in this paper we propose a novel heterogeneous graph neural architecture, named: PreHIN4Rec. The proposed PreHIN4Rec is considered as the graph pre-training approach for leveraging the performance of recommendation task in both accuracy and scalability aspects. In general, our proposed PreHIN4Rec is designed to efficiently preserve both heterogeneous schematic and local structural latent features of user–item interactions in forms of HINs. It can effectively support to better fine-tune for achieving remarkable improvements in recommendation tasks through integrating with existing recommendation frameworks. The extensive experiments in benchmark datasets demonstrate the effectiveness of our proposed PreHIN4Rec model in comparing with recent state-of-the-art GNN-based recommendation baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. MovieLens dataset: https://grouplens.org/datasets/movielens/.

  2. Foursquare dataset: https://sites.google.com/site/yangdingqi/home/foursquare-dataset.

  3. Amazon-Book dataset: http://snap.stanford.edu/data/web-Amazon-links.html.

  4. Metapath2Vec: https://ericdongyx.github.io/metapath2vec/m2v.html.

  5. NCF model: https://github.com/hexiangnan/neural_collaborative_filtering.

  6. NGCF model: https://github.com/xiangwang1223/neural_graph_collaborative_filtering.

  7. LightGCN model: https://github.com/kuandeng/LightGCN (Tensorflow), https://github.com/gusye1234/LightGCN-PyTorch (PyTorch).

  8. SGL model: https://github.com/wujcan/SGL.

References

  1. Zhao X, Zhang Z, Bi X, Sun Y (2020) A new point-of-interest group recommendation method in location-based social networks. Neural Comput Appl 8:1–12

    Google Scholar 

  2. Tahmasebi H, Ravanmehr R, Mohamadrezaei R (2021) Social movie recommender system based on deep autoencoder network using Twitter data. Neural Comput Appl 33(5):1607–1623

    Article  Google Scholar 

  3. Zhang L (2021) Top-N recommendation algorithm integrated neural network. Neural Comput Appl 33(9):3881–3889

    Article  Google Scholar 

  4. Zhang S, Yao L, Sun A, Tay Y (2019) Deep learning based recommender system: a survey and new perspectives. ACM Comput Surveys (CSUR) 52(1):1–38

    Article  Google Scholar 

  5. Rendle S, Krichene W, Zhang L, Anderson J (2020) Neural collaborative filtering vs. matrix factorization revisited. RecSys

  6. He X, Liao L, Zhang H, Nie L, Hu X, Chua TS (2017) Neural collaborative filtering. WWW

  7. Wang X, He X, Wang M, Feng F, Chua TS (2019) Neural graph collaborative filtering. SIGIR

  8. Zhang Z, Cui P, Zhu W (2020) Deep learning on graphs: a survey. IEEE Trans Knowled Data Eng

  9. Sun Y, Han J, Yan X, Yu PS, Wu T (2011) Pathsim: meta path-based top-k similarity search in heterogeneous information networks. VLDB

  10. Shi C, Li Y, Zhang J, Sun Y, Philip SY (2016) A survey of heterogeneous information network analysis. IEEE Trans Knowl Data Eng 29(1):17–37

    Article  Google Scholar 

  11. Pham P, Do P (2019) W-MetaPath2Vec: The topic-driven meta-path-based model for large-scaled content-based heterogeneous information network representation learning. Expert Syst Appl 123:328–344

    Article  Google Scholar 

  12. Do P, Pham P (2021) W-KG2Vec: a weighted text-enhanced meta-path-based knowledge graph embedding for similarity search. Neural Comput Appl 33:1–23

    Article  MathSciNet  Google Scholar 

  13. Pham P, Do P (2021) W-MMP2Vec: topic-driven network embedding model for link prediction in content-based heterogeneous information network. Intel Data Anal 25(3):711–738

    Article  Google Scholar 

  14. Pham P, Do P (2020) W-Com2Vec: a topic-driven meta-path-based intra-community embedding for content-based heterogeneous information network. Intel Data Anal 24(5):1207–1233

    Article  Google Scholar 

  15. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. ICLR

  16. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. NIPS

  17. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. ICLR

  18. He X, Deng K, Wang X, Li Y, Zhang Y, Wang M (2020) Lightgcn: Simplifying and powering graph convolution network for recommendation. SIGIR

  19. Zhao H, Yao Q, Li J, Song Y, Lee DL (2017) Meta-graph based recommendation fusion over heterogeneous information networks. SIGKDD

  20. Shi C, Hu B, Zhao WX, Philip SY (2018) Heterogeneous information network embedding for recommendation. IEEE Trans Knowl Data Eng 31(2):357–370

    Article  Google Scholar 

  21. Fu X, Zhang J, Meng Z, King I (2020) MAGNN: metapath aggregated graph neural network for heterogeneous graph embedding. WWW

  22. Lu Y, Fang Y, Shi C (2020) Meta-learning on heterogeneous information networks for cold-start recommendation. SIGKDD

  23. Zhao Z, Zhang X, Zhou H, Li C, Gong M, Wang Y (2020) HetNERec: Heterogeneous network embedding based recommendation. Knowl-Based Syst 204:106218

    Article  Google Scholar 

  24. Pham P, Do P (2020) W-Metagraph2Vec: a novel approval of enriched schematic topic-driven heterogeneous information network embedding. Int J Mach Learn Cybern 11(8):1855–1874

    Article  Google Scholar 

  25. Meng Z, Liu S, Macdonald C, Ounis I (2021) Graph neural pre-training for enhancing recommendations using side information. ar**v preprint ar**v:2107.03936.

  26. Jiang X, Lu Y, Fang Y, Shi C (2021) Contrastive pre-training of GNNs on heterogeneous graphs. In: Proceedings of the 30th ACM international conference on information knowledge management

  27. Yang C, **ao Y, Zhang Y, Sun Y, Han J (2020) Heterogeneous network representation learning: a unified framework with survey and benchmark. IEEE Trans Knowl Data Eng

  28. Fu T. Y, Lee W. C, Lei Z (2017) Hin2vec: Explore meta-paths in heterogeneous information networks for representation learning. CIKM

  29. Dong Y, Chawla N. V, Swami A (2017) metapath2vec: Scalable representation learning for heterogeneous networks. SIGKDD

  30. Wu J, Wang X, Feng F, He X, Chen L, Lian J, **e X (2021) Self-supervised graph learning for recommendation. In: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval

  31. Mnih A, Salakhutdinov RR (2008) Probabilistic matrix factorization. Advances in neural information processing systems

  32. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791

    Article  Google Scholar 

  33. Rendle S (2012) Factorization machines with libfm. ACM Trans Intel Syst Technol (TIST) 3(3):1–22

    Article  Google Scholar 

  34. Wu Z, Pan S, Chen F, Long G, Zhang C, Philip SY (2020) A comprehensive survey on graph neural networks. IEEE Trans Neural Networks Learn Syst 32(1):4–24

    Article  MathSciNet  Google Scholar 

  35. Zhang H, McAuley J (2020) Stacked mixed-order graph convolutional networks for collaborative filtering. SDM

  36. Chen L, Wu L, Hong R, Zhang K, Wang M (2020) Revisiting graph based collaborative filtering: a linear residual graph convolutional network approach. AAAI

  37. Yu J, Yin H, Li J, Gao M, Huang Z, Cui L (2020) Enhance social recommendation with adversarial graph convolutional networks. IEEE Trans Knowl Data Eng

  38. Liu F, Cheng Z, Zhu L, Liu C, Nie L (2020) A2-GCN: an attribute-aware attentive GCN model for recommendation. IEEE Trans Knowl Data Eng

  39. Liu F, Cheng Z, Zhu L, Gao Z, Nie L (2021) Interest-aware message-passing GCN for recommendation. In: Proceedings of the web conference 2021

  40. Shi C, Zhou C, Kong X, Yu P. S, Liu G, Wang B (2012) Heterecom: a semantic-based recommendation system in heterogeneous networks. SIGKDD

  41. Luo C, Pang W, Wang Z, Lin C (2014) Hete-cf: social-based collaborative filtering recommendation using heterogeneous relations. ICDM

  42. Shi C, Zhang Z, Luo P, Yu P. S, Yue Y, Wu B (2015) Semantic path based personalized recommendation on weighted heterogeneous information networks. CIKM

  43. Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational. Advances in neural information processing systems

  44. Grover A, Leskovec J (2016) node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining

  45. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: 5th international conference on learning representations ICLR

  46. McAuley J, Leskovec J (2013) Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM conference on recommender systems

  47. Rendle S, Freudenthaler C Gantner Z, Schmidt-Thieme L (2009) BPR: bayesian personalized ranking from implicit feedback. In: Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence

Download references

Funding

Viet Nam National University Ho Chi Minh City , DS2020-26-01, Phuc Do.

Author information

Authors and Affiliations

  1. University of Information Technology (UIT), Viet Nam National University, Ho Chi Minh City, Vietnam

    Phuc Do

  2. Faculty of Information Technology, HUTECH University, Ho Chi Minh City, Vietnam

    Phu Pham

Authors
  1. Phuc Do
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Phu Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Phuc Do.

Ethics declarations

Conflict of interest

This statement is to certify that all authors have seen and approved the manuscript being submitted. We warrant that we have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Do, P., Pham, P. Heterogeneous graph convolutional network pre-training as side information for improving recommendation. Neural Comput & Applic 34, 15945–15961 (2022). https://doi.org/10.1007/s00521-022-07251-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07251-z

Keywords

Search

Navigation