SpringerLink
Log in

Deep Learning Overview

  • Chapter
  • First Online:
Session-Based Recommender Systems Using Deep Learning

Abstract

Among the various machine learning algorithms, deep learning has recently been dramatically used in different scopes. Deep learning models have been significantly employed in effectively extracting hidden patterns from vast amounts of data and modeling interdependent variables to solve complex problems. Since this book aims to discuss the session-based recommender system approaches using deep learning models, brief explanations of various deep neural networks are provided in this chapter. For this purpose, the history, basic concepts, advantages/applications, and fundamental models of deep learning are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now
Chapter
EUR 29.95
Price includes VAT (France)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 139.09
Price includes VAT (France)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
EUR 179.34
Price includes VAT (France)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016. https://www.deeplearningbook.org

    Google Scholar 

  2. Merima Kulin, Carolina Fortuna, Eli De Poorter, Dirk Deschrijver, and Ingrid Moerman. "Data-driven design of intelligent wireless networks: An overview and tutorial." Sensors 16, no. 6 (2016): 790. https://doi.org/10.3390/s16060790

    Article  Google Scholar 

  3. Vasant Dhar. "Data science and prediction." Communications of the ACM 56, no. 12 (2013): 64-73. https://doi.org/10.1145/2500499

  4. Paul Fergus, and Carl Chalmers. Applied Deep Learning: Tools, Techniques, and Implementation. Springer Nature, 2022. https://doi.org/10.1007/978-3-031-04420-5

  5. Sergey I. Nikolenko. Introduction: The Data Problem. In: Synthetic Data for Deep Learning. Springer Optimization and Its Applications 174, (2021). https://doi.org/10.1007/978-3-030-75178-4_1

  6. Iqbal H. Sarker. "Deep learning: a comprehensive overview on techniques, taxonomy, applications and research directions." SN Computer Science 2, no. 6 (2021): 420. https://doi.org/10.1007/s42979-021-00815-1

    Article  Google Scholar 

  7. Andrew Ng, and Michael Jordan. "On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes." Advances in neural information processing systems 14 (2001). https://dl.acm.org/doi/abs/10.5555/2980539.2980648

  8. Volodymyr Kuleshov, and Stefano Ermon. "Deep hybrid models: Bridging discriminative and generative approaches." In Proceedings of the Conference on Uncertainty in AI (UAI). Sydney, Australia, August 12-14, 2017.

    Google Scholar 

  9. Li Deng, and Navdeep Jaitly. "Deep discriminative and generative models for speech pattern recognition." In Handbook of pattern recognition and computer vision, pp. 27-52. 2016. https://doi.org/10.1142/9789814656535_0002

  10. Yoshua Bengio, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35, no. 8 (2013): 1798-1828. https://doi.org/10.1109/TPAMI.2013.50

    Article  Google Scholar 

  11. Léon Bottou. "Large-scale machine learning with stochastic gradient descent." In Proceedings of 19th International Conference on Computational Statistics Paris France, August 22-27, 2010 Keynote, Invited and Contributed Papers, pp. 177-186. Physica-Verlag HD, 2010. https://doi.org/10.1007/978-3-7908-2604-3_16

  12. Dong C. Liu, and Jorge Nocedal. "On the limited memory BFGS method for large scale optimization." Mathematical programming 45, no. 1-3 (1989): 503-528. https://doi.org/10.1007/BF01589116

    Article  MathSciNet  Google Scholar 

  13. Diederik P. Kingma and Jimmy Lei Ba. “Adam: a Method for Stochastic Optimization”. International Conference on Learning Representations, San Diego, CA, USA, May 7-9, 2015, page 1-13. https://arxiv.org/pdf/1412.6980.pdf

  14. Robert Mansel Gower, Nicolas Loizou, Xun Qian, Alibek Sailanbayev, Egor Shulgin, and Peter Richtárik. "SGD: General analysis and improved rates." International conference on machine learning, Long Beach, CA, USA, Jun 10-15, 2019 pp. 5200-5209. PMLR, 2019. https://doi.org/10.48550/ar**v.1901.09401

  15. John Duchi, Elad Hazan, and Yoram Singer. "Adaptive subgradient methods for online learning and stochastic optimization." Journal of machine learning research 12, no. 7 (2011). https://jmlr.org/papers/v12/duchi11a.html

  16. Matthew D. Zeiler. "Adadelta: an adaptive learning rate method." ar**v preprint ar**v:1212.5701 (2012). https://doi.org/10.48550/ar**v.1212.5701

  17. Shiv Ram Dubey, Satish Kumar Singh, and Bidyut Baran Chaudhuri. "Activation functions in deep learning: A comprehensive survey and benchmark." Neurocomputing (2022). https://doi.org/10.1016/j.neucom.2022.06.111

  18. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. "Gradient-based learning applied to document recognition." Proceedings of the IEEE 86, no. 11 (1998): 2278-2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  19. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Communications of the ACM 60, no. 6 (2017): 84-90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  20. Karen Simonyan, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." ar**v preprint ar**v:1409.1556 (2014). https://doi.org/10.48550/ar**v.1409.1556

  21. Min Lin, Qiang Chen, and Shuicheng Yan. "Network in network." ar**v preprint ar**v:1312.4400 (2013). https://doi.org/10.48550/ar**v.1312.4400

  22. Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and M. Riedmiller. "Striving for Simplicity: The All Convolutional Net." In ICLR (workshop track). 2015.

    Google Scholar 

  23. Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. "Densely Connected Convolutional Networks." In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, July 21-26, 2017 pp. 2261-2269. https://doi.org/10.1109/CVPR.2017.243

  24. Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. "FractalNet: Ultra-Deep Neural Networks without Residuals." In International Conference on Learning Representations ICLR 2017, Toulon, France, April 24-26, 2017. https://doi.org/10.48550/ar**v.1605.07648

  25. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. "Going deeper with convolutions." In Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA, June 7-12, 2015, pp. 1-9. 2015.

    Google Scholar 

  26. Kaiming He, **angyu Zhang, Shaoqing Ren, and Jian Sun. "Deep residual learning for image recognition." In Proceedings of the IEEE conference on computer vision and pattern recognition, Caesars Palace, Jun 26-July 1, 2016 pp. 770-778. https://doi.org/10.1109/CVPR.2016.90

  27. Laith Alzubaidi, **glan Zhang, Amjad J. Humaidi, Ayad Al-Dujaili, Ye Duan, Omran Al-Shamma, José Santamaría, Mohammed A. Fadhel, Muthana Al-Amidie, and Laith Farhan. "Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions." Journal of big Data 8 (2021): 1-74. https://doi.org/10.1186/s40537-021-00444-8

    Article  Google Scholar 

  28. Alex Sherstinsky. "Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network." Physica D: Nonlinear Phenomena 404 (2020): 132306. https://doi.org/10.1016/j.physd.2019.132306

    Article  MathSciNet  Google Scholar 

  29. Michael I. Jordan. "Serial order: A parallel distributed processing approach." In Advances in psychology, vol. 121, pp. 471-495. North-Holland, 1997. https://doi.org/10.1016/S0166-4115(97)80111-2

    Chapter  Google Scholar 

  30. Jeffrey L. Elman. "Finding structure in time." Cognitive science 14, no. 2 (1990): 179-211. https://doi.org/10.1207/s15516709cog1402_1

    Article  Google Scholar 

  31. G. R. Kanagachidambaresan, Adarsha Ruwali, Debrup Banerjee, and Kolla Bhanu Prakash. "Recurrent neural network." Programming with TensorFlow: Solution for Edge Computing Applications (2021): 53-61. https://doi.org/10.1007/978-3-030-57077-4_7

  32. Sepp Hochreiter, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9, no. 8 (1997): 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  33. Santiago Fernández, Alex Graves, and Jürgen Schmidhuber. "Sequence labelling in structured domains with hierarchical recurrent neural networks." In Proceedings of the 20th International Joint Conference on Artificial Intelligence, IJCAI 2007, Hyderabad, India. 6 – 12 January, 2007.

    Google Scholar 

  34. Santiago Fernández, Alex Graves, and Jürgen Schmidhuber. "An application of recurrent neural networks to discriminative keyword spotting." In Artificial Neural Networks–ICANN 2007: 17th International Conference, Porto, Portugal, September 9-13, 2007, Proceedings, Part II 17, pp. 220-229. Springer Berlin Heidelberg, 2007. https://doi.org/10.1007/978-3-540-74695-9_23

  35. Alex Graves, and Jürgen Schmidhuber. "Framewise phoneme classification with bidirectional LSTM and other neural network architectures." Neural networks 18, no. 5-6 (2005): 602-610. https://doi.org/10.1016/j.neunet.2005.06.042

    Article  Google Scholar 

  36. Tara N. Sainath, Oriol Vinyals, Andrew Senior, and Haşim Sak. "Convolutional, long short-term memory, fully connected deep neural networks." In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), South Brisbane, Queensland, Australia, April 19-24, 2015, pp. 4580-4584. Ieee, 2015. https://doi.org/10.1109/ICASSP.2015.7178838

  37. Alex Graves, Santiago Fernández, and Jürgen Schmidhuber. "Multi-dimensional recurrent neural networks." In Artificial Neural Networks–ICANN 2007: 17th International Conference, Porto, Portugal, September 9-13, 2007, Proceedings, Part I 17, pp. 549-558. Springer Berlin Heidelberg, 2007. https://doi.org/10.1007/978-3-540-74690-4_56

  38. **aodan Liang, **aohui Shen, Donglai **ang, Jiashi Feng, Liang Lin, and Shuicheng Yan. "Semantic object parsing with local-global long short-term memory." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Caesars Palace, Jun 26-July 1, 2016, pp. 3185-3193. 2016.

    Google Scholar 

  39. Mike Schuster, and Kuldip K. Paliwal. "Bidirectional recurrent neural networks." IEEE transactions on Signal Processing 45, no. 11 (1997): 2673-2681. https://doi.org/10.1109/78.650093

    Article  Google Scholar 

  40. Yong Yu, **aosheng Si, Changhua Hu, and Jianxun Zhang. "A review of recurrent neural networks: LSTM cells and network architectures." Neural computation 31, no. 7 (2019): 1235-1270. https://doi.org/10.1162/neco_a_01199

    Article  MathSciNet  Google Scholar 

  41. Kyunghyun Cho, Bart Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. "Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation." Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, Doha, Qatar, October 25-29, 2014, pp.1724-1734. 2014. https://doi.org/10.3115/v1/D14-1179

  42. Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. "Neural machine translation by jointly learning to align and translate." In 3rd International Conference on Learning Representations, ICLR 2015. 2015.

    Google Scholar 

  43. Jakub M. Tomczak, "Deep Generative Modeling", Springer Nature, 2022, https://doi.org/10.1007/978-3-030-93158-2

  44. Lars Ruthotto, and Eldad Haber. "An introduction to deep generative modeling." GAMM-Mitteilungen 44, no. 2 (2021): e202100008. https://doi.org/10.1002/gamm.202100008

    Article  MathSciNet  Google Scholar 

  45. Mark A. Kramer. "Nonlinear principal component analysis using autoassociative neural networks." AIChE journal 37, no. 2 (1991): 233-243. https://doi.org/10.1002/aic.690370209

    Article  Google Scholar 

  46. R. Indrakumari, T. Poongodi, and Kiran Singh. "Introduction to Deep Learning." Advanced Deep Learning for Engineers and Scientists: A Practical Approach (2021): 1-22. https://doi.org/10.1007/978-3-030-66519-7_1

  47. Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio. "Contractive auto-encoders: Explicit invariance during feature extraction." In Proceedings of the 28th international conference on international conference on machine learning, Bellevue, USA, June 28 - July 2, 2011, pp. 833-840.

    Google Scholar 

  48. Alireza Makhzani, Brendan Frey. K-sparse autoencoders. 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014.

    Google Scholar 

  49. Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion." Journal of machine learning research 11, no. 12 (2010).

    Google Scholar 

  50. Jonathan Masci, Ueli Meier, Dan Cireşan, and Jürgen Schmidhuber. "Stacked convolutional auto-encoders for hierarchical feature extraction." In Artificial Neural Networks and Machine Learning–ICANN 2011: 21st International Conference on Artificial Neural Networks, Espoo, Finland, June 14-17, 2011, Proceedings, Part I 21, pp. 52-59. Springer Berlin Heidelberg, 2011. https://doi.org/10.1007/978-3-642-21735-7_7

  51. DP Kingma, Welling M. Auto-encoding variational bayes. 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014.

    Google Scholar 

  52. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. "Generative adversarial nets" (Advances in neural information processing systems) (pp. 2672–2680). Red Hook, NY Curran (2014).

    Google Scholar 

  53. Alankrita Aggarwal, Mamta Mittal, and Gopi Battineni. "Generative adversarial network: An overview of theory and applications." International Journal of Information Management Data Insights 1, no. 1 (2021): 100004. https://doi.org/10.1016/j.jjimei.2020.100004

  54. David H. Ackley, Geoffrey E. Hinton, and Terrence J. Sejnowski. "A learning algorithm for Boltzmann machines." Cognitive science 9, no. 1 (1985): 147-169. https://doi.org/10.1016/S0364-0213(85)80012-4

    Article  Google Scholar 

  55. Harshvardhan GM, Mahendra Kumar Gourisaria, Manjusha Pandey, and Siddharth Swarup Rautaray. "A comprehensive survey and analysis of generative models in machine learning." Computer Science Review 38 (2020): 100285. https://doi.org/10.1016/j.cosrev.2020.100285

    Article  MathSciNet  Google Scholar 

  56. Geoffrey E. Hinton. "A practical guide to training restricted Boltzmann machines." Neural Networks: Tricks of the Trade: Second Edition (2012): 599-619. https://doi.org/10.1007/978-3-642-35289-8_32

  57. Ruslan Salakhutdinov, and Geoffrey E. Hinton. Deep Boltzmann machines, in: Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, Florida, USA, April 2009.

    Google Scholar 

  58. Geoffrey E. Hinton. "Deep belief networks." Scholarpedia 4, no. 5 (2009): 5947. https://doi.org/10.4249/scholarpedia.5947

    Article  Google Scholar 

  59. **g Ren, Mark Green, and **shi Huang. "From traditional to deep learning: Fault diagnosis for autonomous vehicles." In Learning Control, pp. 205-219. Elsevier, 2021. https://doi.org/10.1016/B978-0-12-822314-7.00013-4

    Chapter  Google Scholar 

  60. Ziwei Zhang, Peng Cui, and Wenwu Zhu. "Deep learning on graphs: A survey." IEEE Transactions on Knowledge and Data Engineering 34, no. 1 (2020): 249-270. https://doi.org/10.1109/TKDE.2020.2981333

    Article  Google Scholar 

  61. Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. "The graph neural network model." IEEE transactions on neural networks 20, no. 1 (2008). https://doi.org/61-80.10.1109/TNN.2008.2005605

  62. Thomas N. Kipf, and Max Welling. "Semi-Supervised Classification with Graph Convolutional Networks." In International Conference on Learning Representations ICLR 2017, Toulon, France, April 24-26, 2017.

    Google Scholar 

  63. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018.

    Google Scholar 

  64. Yujia Li, Richard Zemel, Marc Brockschmidt, and Daniel Tarlow. "Gated Graph Sequence Neural Networks." 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016.

    Google Scholar 

  65. Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. "Graph neural networks: A review of methods and applications." AI open 1 (2020): 57-81. https://doi.org/10.1016/j.aiopen.2021.01.001

    Article  Google Scholar 

  66. Shi Dong, ** Wang, and Khushnood Abbas. "A survey on deep learning and its applications." Computer Science Review 40 (2021): 100379. https://doi.org/10.1016/j.cosrev.2021.100379

    Article  MathSciNet  Google Scholar 

  67. Si Zhang, Hanghang Tong, Jiejun Xu, and Ross Maciejewski. "Graph convolutional networks: a comprehensive review." Computational Social Networks 6, no. 1 (2019): 1-23. https://doi.org/10.1186/s40649-019-0069-y

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Central Tehran Branch, Islamic Azad University, Tehran, Iran

    Reza Ravanmehr & Rezvan Mohamadrezaei

Authors
  1. Reza Ravanmehr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rezvan Mohamadrezaei
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Ravanmehr, R., Mohamadrezaei, R. (2024). Deep Learning Overview. In: Session-Based Recommender Systems Using Deep Learning. Springer, Cham. https://doi.org/10.1007/978-3-031-42559-2_2

Download citation

Publish with us

Policies and ethics

Search

Navigation