-
Gradient Descent and Second-Order Optimization - A Thorough Comparison
An in-depth exploration of Gradient Descent (GD) and Second-Order Gradient Descent (2GD), focusing on convergence behavior, mathematical derivations, and performance differences.
-
Gradient Descent Convergence - Prerequisites and Detailed Derivation
Understanding the convergence of gradient descent with a fixed step size and proving its rate of convergence for convex, differentiable functions.
-
Understanding Stochastic Gradient Descent (SGD)
A detailed guide to gradient descent variants, highlighting the mechanics, trade-offs, and practical insights of Stochastic Gradient Descent (SGD).
-
Gradient Descent - A Detailed Walkthrough
An in-depth exploration of gradient descent, including its convergence and step size considerations.
-
Empirical Risk Minimization (ERM)
Exploring Empirical Risk Minimization - Balancing approximation, estimation, and optimization errors to build effective supervised learning models.