Mathematical Foundations of Deep Learning Models and Algorithms Published by the American Mathematical Soiety (AMS) Konstantinos Spiliopoulos and Richard Sowers and Justin Sirignano

Deep learning uses multi-layer neural networks to model complex data patterns. Large models—with millions or even billions of parameters—are trained on massive datasets. This approach has produced revolutionary advances in image, text, and speech recognition and also has potential applications in a range of other fields such as engineering, finance, mathematics, and medicine.

The book "Mathematical Foundations of Deep Learning Models and Algorithms", published by the American Mathematical Soiety (AMS) aims to serve as an introduction to the mathematical theory underpinning the recent advances in deep learning. Detailed derivations as well as mathematical proofs are presented for many of the models and optimization methods which are commonly used in machine learning and deep learning. Applications, code, and practical approaches to training models are also included.

The book is designed for advanced undergraduates, graduate students, practitioners, and researchers. Divided into two parts, it begins with mathematical foundations before tackling advanced topics in approximation, optimization, and neural network training.

• Part 1 focuses on a mathematical introduction to deep learning. Part 1 is written for a general audience, including students in mathematics, statistics, computer science, data science, or engineering.
  
• Part 2 contains advanced topics and convergence results in deep learning.
  

Book Citation

@book{MathDLBook-2025,
    title={Mathematical Foundations of Deep Learning Models and Algorithms},
    author={Konstantinos Spiliopoulos and Richard Sowers and Justin Sirignano},
    publisher={American Mathematical Society},
    note={\url{MathDL.github.io}},
    year={2025}
}

Table of Contents

• Contents
  
• Preface
  
• Notation
  
• Website
  
• Chapter 1. Introduction
  

• Part 1. Mathematical Introduction to Deep Learning

  • Chapter 2. Linear Regression
    
  • Chapter 3. Logistic Regression
    
  • Chapter 4. Perceptron and Kernels
    
  • Chapter 5. FeedForward Networks
    
  • Chapter 6. Backpropagation
    
  • Chapter 7. Basics on Stochastic Gradient Descent
    
  • Chapter 8. Stochastic Gradient Descent for Multi-layer Networks
    
  • Chapter 9. Regularization and Dropout
    
  • Chapter 10. Batch Normalization
    
  • Chapter 11. Training, Validation and Testing
    
  • Chapter 12. Feature Importance
    
  • Chapter 13. Recurrent Neural Networks and Sequential Data
    
  • Chapter 14. Convolution Neural Networks
    
  • Chapter 15. Variational Inference and Generative Models

• Part 2. Advanced Topics and Convergence Results in Deep Learning

  • Transitioning from Part 1 to Part 2
    
  • Chapter 16. Universal Approximation Theorem
    
  • Chapter 17. Convergence Analysis of Gradient Descent
    
  • Chapter 18. Convergence Analysis of Stochastic Gradient Descent
    
  • Chapter 19. The Neural Tangent Kernel Regime
    
  • Chapter 20. Optimization in Feature Learning Regime: Mean Field Scaling
    
  • Chapter 21. Reinforcement Learning
    
  • Chapter 22. Neural Differential Equations
    
  • Chapter 23. Distributed Training
    
  • Chapter 24. Automatic Differentiation

• Part 3. Appendix

  • Appendix A. Background Material in Probability
    
  • Appendix B. Background Material in Analysis
    
• Bibliography
  
• Index
  

Code and Exercises

Errata