Abstract
This chapter delves into the fundamental concepts of calculus and optimisation related to machine learning, offering both theoretical insights and practical usecases. Starting with the motivation behind using calculus in machine learning, the chapter systematically introduces the concept of limit, which lays the foundation for understanding derivatives and their properties. The discussion on derivatives extends to their role in partial differentiation and gradients, both crucial for optimising machine learning algorithms. A significant portion of the chapter is dedicated to optimisation techniques specifically tailored for neural networks. The chapter begins with an overview of learning definitions and their implications for neural network training. This is followed by an examination of constrained versus unconstrained optimisation and an exploration of the complexities in identifying absolute and local minima of functions. The latter part of the chapter is focused on various optimisation algorithms, briefly discussing line search and trust region methods, then transitioning into specific approaches like steepest descent and gradient descent. The importance of selecting an appropriate learning rate is discussed, along with variations of gradient descent (GD) and strategies for choosing the right mini-batch size. The chapter concludes by exploring the connection between Stochastic Gradient Descent (SGD), fractals, and their implications in machine learning, providing a fascinating example of how complexity appear from very easy optimisation problems.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Michelucci, U. (2024). Calculus and Optimisation for Machine Learning. In: Fundamental Mathematical Concepts for Machine Learning in Science. Springer, Cham. https://doi.org/10.1007/978-3-031-56431-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-56431-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-56430-7
Online ISBN: 978-3-031-56431-4
eBook Packages: Computer ScienceComputer Science (R0)