SpringerLink
Log in

Dirichlet Graph Convolution Coupled Neural Differential Equation for Spatio-temporal Time Series Prediction

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In recent years, multivariate time series prediction has attracted extensive research interests. However, the dynamic changes of the spatial topology and the temporal evolution of multivariate variables bring great challenges to the spatio-temporal time series prediction. In this paper, a novel Dirichlet graph convolution module is introduced to automatically learn the spatio-temporal representation, and we combine graph attention (GAT) and neural differential equation (NDE) based on nonlinear state transition to model spatio-temporal state evolution of nonlinear systems. Specifically, the spatial topology is revealed by the cosine similarity of node embeddings. The use of multi-layer Dirichlet graph convolution aims to enhance the representation ability of the model while suppressing the phenomenon of over-smoothing or over-separation. The GCN and LSTM-based network is used as the nonlinear operator to model the evolution law of the dynamic system, and the GAT updates the strength of the connection. In addition, the Euler trapezoidal integral method is used to model the temporal dynamics and makes medium and long-term prediction in latent space from the perspective of nonlinear state transition. The proposed model can adaptively mine spatial correlations and discover spatio-temporal dynamic evolution patterns through the coupled NDE, which makes the modeling process more interpretable. Experiment results demonstrate the effectiveness of spatio-temporal dynamic discovery on predictive performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1:
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Li D, Han M, Wang J (2012) Chaotic time series prediction based on a novel robust echo state network. IEEE Trans Neural Netw Learn Syst 23(5):787–799

    Article  Google Scholar 

  2. Quan H, Srinivasan D, Khosravi A (2014) Short-term load and wind power forecasting using neural network-based prediction intervals. IEEE Trans Neural Netw Learn Syst 25(2):303–315

    Article  Google Scholar 

  3. Pal S, Ma L, Zhang Y, Coates M (2021) RNN with particle flow for probabilistic spatio-temporal forecasting. In: Proceedings of ICML, pp 8336–8348

  4. Spadondesouza G, Hong S, Brandoli B, Matwin S, Rodrigues JF, Sun J (2021) Pay attention to evolution: time series forecasting with deep graph-evolution learning. IEEE Trans Pattern Anal Mach Intell 44:1–17

    Google Scholar 

  5. Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum DS (2016) Urban water quality prediction based on multi-task multi-view learning. In: Proceedings of IJCAI

  6. Lv Y, Duan Y, Kang W, Li Z, Wang FY (2015) Traffic flow prediction with big data: a deep learning approach. IEEE Trans Intell Transp Syst 16(2):865–873

    Google Scholar 

  7. Han M, Feng S, Chen CP, Xu M, Qiu T (2018) Structured manifold broad learning system: a manifold perspective for large-scale chaotic time series analysis and prediction. IEEE Trans Knowl Data Eng 31(9):1809–1821

    Article  Google Scholar 

  8. Yu B, Yin H, Zhu Z (2018) Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: Proceedings of IJCAI, pp 3634–3640

  9. Song C, Lin Y, Guo S, Wan H (2020) Spatial-temporal synchronous graph convolutional networks: a new framework for spatial-temporal network data forecasting. In: Proceedings of AAAI, vol 34, pp 914–921

  10. Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2021) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24

    Article  MathSciNet  Google Scholar 

  11. Spinelli I, Scardapane S, Uncini A (2020) Adaptive propagation graph convolutional network. IEEE Trans Neural Netw Learn Syst 32(10):4755–4760

    Article  Google Scholar 

  12. Zhou K, Huang X, Zha D, Chen R, Li L, Choi S-H, Hu X (2021) Dirichlet energy constrained learning for deep graph neural networks. In: Proceedings of NeurIPS

  13. Wang X, Zhu M, Bo D, Cui P, Shi C, Pei J (2020) AM-GCN: adaptive multi-channel graph convolutional networks. In: Proceedings of ACM SIGKDD, pp 1243–1253

  14. Shi S, **e P, Luo X, Qiao K, Wang L, Chen J, Yan B (2022) Adaptive multi-layer contrastive graph neural networks. Neural Process Lett 55:1–20

    Google Scholar 

  15. Wang C, Zhang H, Chen B, Wang D, Wang Z, Zhou M (2020) Deep relational topic modeling via graph Poisson gamma belief network. In: Proceedings of NeurIPS, pp 488–500

  16. Huang Z, Sun Y, Wang W (2021) Coupled graph ode for learning interacting system dynamics. In: Proceedings of ACM SIGKDD, pp 705–715

  17. Lai G, Chang W-C, Yang Y, Liu H (2018) Modeling long-and short-term temporal patterns with deep neural networks. In: Proceedings of ACM SIGIR, pp 95–104

  18. Jalali SMJ, Ahmadian S, Kavousi-Fard A, Khosravi A, Nahavandi S (2021) Automated deep CNN-LSTM architecture design for solar irradiance forecasting. IEEE Trans Syst Man Cybern Syst 52(1):54–65

    Article  Google Scholar 

  19. Hu J, Wang X, Zhang Y, Zhang D, Zhang M, Xue J (2020) Time series prediction method based on variant LSTM recurrent neural network. Neural Process Lett 52:1485–1500

    Article  Google Scholar 

  20. Chen TQ, Rubanova Y, Bettencourt J, Duvenaud DK (2018) Neural ordinary differential equations. In: Proceedings of NeurIPS

  21. Zhao L, Song Y, Zhang C, Liu Y, Wang P, Lin T, Deng M, Li H (2019) T-GCN: a temporal graph convolutional network for traffic prediction. IEEE Trans Intell Transp Syst 21(9):3848–3858

    Article  Google Scholar 

  22. Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and deep locally connected networks on graphs. In: Proceedings of ICLR, pp 1–14

  23. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of ICLR

  24. Xu Z, Kang Y, Cao Y, Li Z (2020) Spatiotemporal graph convolution multifusion network for urban vehicle emission prediction. IEEE Trans Neural Netw Learn Syst 32(8):3342–3354

    Article  Google Scholar 

  25. Li L, Zhu H, Wen L, Lan W, Yang Z (2021) An approach of combining convolution neural network and graph convolution network to predict the progression of myopia. Neural Process Lett 55:1–11

    Google Scholar 

  26. Huang R, Huang C, Liu Y, Dai G, Kong W (2020) LSGCN: long short-term traffic prediction with graph convolutional networks. In: Proceedings of IJCAI, pp 2355–2361

  27. Li R, Wang S, Zhu F, Huang J (2018) Adaptive graph convolutional neural networks. In: Proceedings of AAAI, vol 32

  28. He Y, Wang C, Zhang H, Chen B, Zhou M (2022) A variational edge partition model for supervised graph representation learning. In: Proceedings of NeurIPS, pp 12339–12351

  29. Sareminia S (2022) A support vector based hybrid forecasting model for chaotic time series: spare part consumption prediction. Neural Process Lett 55:1–17

    Google Scholar 

  30. Zhou H, Zhang S, Peng J, Zhang S, Li J, **ong H, Zhang W (2021) Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of AAAI, pp 1–9

  31. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A.N, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Proceedings of NeurIPS, pp 5998–6008

  32. Bandara K, Bergmeir C, Hewamalage H (2020) LSTM-MSNET: leveraging forecasts on sets of related time series with multiple seasonal patterns. IEEE Trans Neural Netw Learn Syst 32(4):1586–1599

    Article  Google Scholar 

  33. Mei J, Ren W, Song Y (2021) A unified framework for adaptive leaderless consensus of uncertain multiagent systems under directed graphs. IEEE Trans Autom Control 66(12):6179–6186

    Article  MathSciNet  Google Scholar 

  34. Erichson NB, Muehlebach M, Mahoney MW (2019) Physics-informed autoencoders for Lyapunov-stable fluid flow prediction. In: Proceedings of NeurIPS, pp 1–6

  35. Zhang Z, Cui P, Pei J, Wang X, Zhu W (2021) Eigen-GNN: a graph structure preserving plug-in for GNNs. IEEE Trans Knowl Data Eng 1–12

  36. Li X, Wen C, Zou Y (2020) Adaptive backstep** control for fractional-order nonlinear systems with external disturbance and uncertain parameters using smooth control. IEEE Trans Syst Man Cybern Systms 51(12):7860–7869

    Article  Google Scholar 

  37. Wang X, Zhu M, Bo D, Cui P, Shi C, Pei J (2020) AM-GCN: adaptive multi-channel graph convolutional networks. In: Proceedings of ACM SIGKDD, pp 1243–1253

  38. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  39. Abramowitz M, Stegun IA (1964) Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. U.S. Department of Commerce, National Bureau of Standards Applied Mathematics Series

  40. Lorenz EN (1996) Predictability: a problem partly solved. In: Proceedings of seminar on predictability

  41. Oreshkin BN, Carpov D, Chapados N, Bengio Y (2020) N-beats: neural basis expansion analysis for interpretable time series forecasting. In: Proceedings of ICLR

  42. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. In: Proceedings of NeurIPS

  43. Lai G, Chang W-C, Yang Y, Liu H (2018) Modeling long-and short-term temporal patterns with deep neural networks. In: Proceedings of ACM SIGIR, pp 95–104

  44. Wu Z, Pan S, Long G, Jiang J, Zhang C (2019) Graph wavenet for deep spatial-temporal graph modeling. In: Proceedings of IJCAI

  45. Wang X, Ma Y, Wang Y, ** W, Wang X, Tang J, Jia C, Yu J (2020) Traffic flow prediction via spatial temporal graph neural network. In: Proceedings of WWW, pp 1082–1092

  46. Lan S, Ma Y, Huang W, Wang W, Yang H, Li P (2022) DSTAGNN: dynamic spatial-temporal aware graph neural network for traffic flow forecasting. In: Proceedings of ICML, pp 11906–11917

  47. Wu H, Xu J, Wang J, Long M (2021) Autoformer: decomposition transformers with auto-correlation for long-term series forecasting. In: Proceedings of NeurIPS, pp 22419–22430

Download references

Funding

This work was supported by the National Natural Science Foundation of China (Grant 62173063).

Author information

Authors and Affiliations

  1. Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, 116024, China

    Qipeng Wang

  2. Key Laboratory of Intelligent Control and Optimization for Industrial Equipment of Ministry of Education, Dalian University of Technology, Dalian, 116024, China

    Min Han

  3. Professional Technology Innovation Center of Distributed Control for Industrial Equipment of Liaoning Province, Dalian University of Technology, Dalian, 116024, China

    Min Han

Authors
  1. Qipeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

QW and MH wrote the manuscript. All authors reviewed the manuscript.

Corresponding author

Correspondence to Min Han.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Q., Han, M. Dirichlet Graph Convolution Coupled Neural Differential Equation for Spatio-temporal Time Series Prediction. Neural Process Lett 55, 12347–12366 (2023). https://doi.org/10.1007/s11063-023-11423-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11423-w

Keywords

Search

Navigation