SpringerLink
Log in

Deep spatial-temporal bi-directional residual optimisation based on tensor decomposition for traffic data imputation on urban road network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The capacity of fully exploiting underlying spatial-temporal dependencies holds the key for missing traffic data imputation, however, previous studies have neglected the residual information from recovery models. To refine this task, we propose a spatial-temporal bi-directional residual optimisation (ST-BiRT) model on the basis of tensor decomposition to effectively improve the imputation performance. The novelty of our approach concentrates on a well-designed bi-directional residual structure, which reduces model errors dramatically. We can greatly exploit the potential of the optimisation structure by dynamically stacking massive residual units, thereby significantly enhancing the recovery capability. When faced with various combinations of missing scenario and missing rate problems, ST-BiRT model can perform with better accuracy and robustness. Here, the experiments on the Guangzhou traffic speed dataset demonstrate that the proposed ST-BiRT model outperforms the state-of-the-art baseline models. In addition, the mechanism of the bi-directional residual optimisation can address extreme cases and their evaluation metrics can reach acceptable values even when the missing rate exceeds 90%. Finally, the superiority of the ST-BiRT model in repairing loss or low-quality traffic data is confirmed by visualising the experimental results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Lee D, Shin K (2021) Robust factorization of real-world tensor streams with patterns, missing values, and outliers. In: 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, pp 840-851

  2. Tan H, Feng G, Feng J, Wang W, Zhang YJ, Li F (2013) A tensor-based method for missing traffic data completion. Transp Res Part C: Emerg Technol 28:15–27

    Article  Google Scholar 

  3. Bae B, Kim H, Lim H, Liu Y, Han LD, Freeze PB (2018) Missing data imputation for traffic flow speed using spatio-temporal cokriging. Transp Res Part C Emerg Technol 88:124–139

    Article  Google Scholar 

  4. Chen X, He Z, Sun L (2019) A Bayesian tensor decomposition approach for spatiotemporal traffic data imputation. Transp Res Part C: Emerg Technol 98:73–84

    Article  Google Scholar 

  5. Li H, Li M, Lin X, He F, Wang Y (2020) A spatiotemporal approach for traffic data imputation with complicated missing patterns. Transp Res Part C: Emerg Technol 119:102730

    Article  Google Scholar 

  6. Zhang T, Zhang DG, Yan HR, Qiu JN, Gao JX (2021) A new method of data missing estimation with FNN-based tensor heterogeneous ensemble learning for internet of vehicle. Neurocomputing 420:98–110

    Article  Google Scholar 

  7. Chen X, Sun L (2021) Bayesian temporal factorization for multidimensional time series prediction. IEEE Trans Pattern Anal Mach Intell

  8. Yang H, Yang J, Han LD, Liu X, Pu L, Chin SM, Hwang HL (2018) A Kriging based spatiotemporal approach for traffic volume data imputation. PLoS One 13(4):e0195957

  9. Laña I, Olabarrieta II, Vélez M, Del Ser J (2018) On the imputation of missing data for road traffic forecasting: New insights and novel techniques. Transp Res Part C: Emerg Technol 90:18–33

    Article  Google Scholar 

  10. Yoon J, Jordon J, Schaar M (2018) Gain: Missing data imputation using generative adversarial nets. In: International Conference on Machine Learning. PMLR, pp 5689-5698

  11. Cui Z, Ke R, Pu Z, Wang Y (2020) Stacked bidirectional and unidirectional LSTM recurrent neural network for forecasting network-wide traffic state with missing values. Transp Res Part C: Emerg Technol 118:102674

    Article  Google Scholar 

  12. Xu D, Wei C, Peng P, Xuan Q, Guo H (2020) GE-GAN: A novel deep learning framework for road traffic state estimation. Transp Res Part C: Emerg Technol 117:102635

    Article  Google Scholar 

  13. Xu D, Wang Y, Peng P, Beilun S, Deng Z, Guo H (2020) Real-time road traffic state prediction based on kernel-KNN. Transp A: Transp Sci 16(1):104–118

    Google Scholar 

  14. Qu L, Zhang Y, Hu J, Jia L, Li L (2008) A BPCA based missing value imputing method for traffic flow volume data. In: 2008 IEEE Intelligent Vehicles Symposium. IEEE, pp 985-990

  15. Qu L, Li L, Zhang Y, Hu J (2009) PPCA-based missing data imputation for traffic flow volume: A systematical approach. IEEE Trans Intell Transp Syst 10(3):512–522

    Article  Google Scholar 

  16. Ma S, Goldfarb D, Chen L (2011) Fixed point and Bregman iterative methods for matrix rank minimization. Math Program 128(1):321–353

    Article  MathSciNet  Google Scholar 

  17. Li Y, Li Z, Li L, Zhang Y, ** M (2013) Comparison on PPCA, KPPCA and MPPCA based missing data imputing for traffic flow. In: ICTIS 2013: Improving Multimodal Transportation Systems-Information, Safety, and Integration, pp 1151-1156

  18. Tang K, Tan C, Cao Y, Yao J, Sun J (2020) A tensor decomposition method for cycle-based traffic volume estimation using sampled vehicle trajectories. Emerg Technol, Transp Res Part C 118:102739

  19. Liu J, Musialski P, Wonka P, Ye J (2012) Tensor completion for estimating missing values in visual data. IEEE Trans Pattern Anal Mach Intell 35(1):208–220

    Article  Google Scholar 

  20. Ran B, Tan H, Wu Y, ** PJ (2016) Tensor based missing traffic data completion with spatial–temporal correlation. Phys A: Stat Mech Appl 446:54–63

  21. Chen X, Lei M, Saunier N, Sun L (2021) Low-rank autoregressive tensor completion for spatiotemporal traffic data imputation. ar**v preprint ar**v:2104.14936

  22. Chen X, Yang J, Sun L (2020) A nonconvex low-rank tensor completion model for spatiotemporal traffic data imputation. Transp Res Part C Emerg Technol 117:102673

    Article  Google Scholar 

  23. Chen X, He Z, Wang J (2018) Spatial-temporal traffic speed patterns discovery and incomplete data recovery via SVD-combined tensor decomposition. Transp Res Part C Emerg Technol 86:59–77

    Article  Google Scholar 

  24. Chen X, He Z, Chen Y, Lu Y, Wang J (2019) Missing traffic data imputation and pattern discovery with a Bayesian augmented tensor factorization model. Transp Res Part C Emerg Technol 104:66–77

    Article  Google Scholar 

  25. Li Q, Tan H, Jiang Z, Wu Y, Ye L (2021) Nonrecurrent traffic congestion detection with a coupled scalable Bayesian robust tensor factorization model. Neurocomputing 430:138–149

    Article  Google Scholar 

  26. Li L, Zhang J, Wang Y, Ran B (2018) Missing value imputation for traffic-related time series data based on a multi-view learning method. IEEE Trans Intell Transp Syst 20(8):2933–2943

    Article  Google Scholar 

  27. Li L, Du B, Wang Y, Qin L, Tan H (2020) Estimation of missing values in heterogeneous traffic data: Application of multimodal deep learning model. Knowl Based Syst 194:105592

    Article  Google Scholar 

  28. Ma T, Antoniou C, Toledo T (2020) Hybrid machine learning algorithm and statistical time series model for network-wide traffic forecast. Transp Res Part C Emerg Technol 111:352–372

    Article  Google Scholar 

  29. Lv Z, Li J, Li H, Xu Z, Wang Y (2021) Blind travel prediction based on obstacle avoidance in indoor scene. Wirel Commun Mob Comput 2021

  30. Basly H, Ouarda W, Sayadi FE, Ouni B, Alimi AM (2021) DTR-HAR: deep temporal residual representation for human activity recognition. Vis Comput 1–21

  31. Du B, Peng H, Wang S, Bhuiyan MZA, Wang L, Gong Q, … Li J (2019) Deep irregular convolutional residual LSTM for urban traffic passenger flows prediction. IEEE Trans Intell Transp Syst 21(3):972-985

  32. Zhang J, Zheng Y, Qi D (2017) Deep spatio-temporal residual networks for citywide crowd flows prediction. In: Thirty-first AAAI conference on artificial intelligence

  33. Guo G, Zhang T (2020) A residual spatio-temporal architecture for travel demand forecasting. Transp Res Part C: Emerg Technol 115:102639

    Article  Google Scholar 

  34. Lv Z, Li J, Dong C, Li H, Xu Z (2021) Deep learning in the COVID-19 epidemic: A deep model for urban traffic revitalization index. Data Knowl Eng 135:101912

  35. Guo S, Lin Y, Li S, Chen Z, Wan H (2019) Deep spatial–temporal 3D convolutional neural networks for traffic data forecasting. IEEE Trans Intell Transp Syst 20(10):3913–3926

    Article  Google Scholar 

  36. Song C, Lin Y, Guo S, Wan H (2020) Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, no 01, pp 914-921)

  37. Lv Z, Li J, Dong C, Zhao W (2020) A deep spatial-temporal network for vehicle trajectory prediction. In: International Conference on Wireless Algorithms, Systems, and Applications. Springer, Cham, pp 359-369

  38. Yu B, Yin H, Zhu Z (2017) Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. ar**v preprint ar**v:1709.04875

  39. Zhang H, Chen P, Zheng J, Zhu J, Yu G, Wang Y, Liu HX (2019) Missing data detection and imputation for urban ANPR system using an iterative tensor decomposition approach. Transp Res Part C: Emerg Technol 107:337–355

    Article  Google Scholar 

  40. Kenneth GE (2021) Statistical application of regression techniques in modeling road accidents in Edo State, Nigeria. Sch J Phys Math Stat 1:14–18

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of P.R. China under Grant 52072130, in part by the National Natural Science Foundation of P.R. China under Grant 11702099.

Author information

Authors and Affiliations

  1. School of Civil Engineering and Transportation, South China University of Technology, Guangzhou, 510641, China

    **long Li, Lunhui Xu, Pan Wu & Zilin Huang

  2. School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, Shenzhen, 518055, China

    Ruonan Li

Authors
  1. **long Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lunhui Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruonan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zilin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lunhui Xu.

Ethics declarations

Conflict of interest

No conflict of interest exits in the submission of this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Xu, L., Li, R. et al. Deep spatial-temporal bi-directional residual optimisation based on tensor decomposition for traffic data imputation on urban road network. Appl Intell 52, 11363–11381 (2022). https://doi.org/10.1007/s10489-021-03060-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03060-4

Keywords

Search

Navigation