Anıl's notes
My handwritten notes.
Optimization theory
Arnoldi Iteration
Arnoldi iteration method to compute maximum eigenvalue of the Hessian. The figure is taken from this video.
Gradient Descent, Chebyshev Acceleration, and Krylov Subspace Methods
Various configurations of gradient descent for linear least squares problem and their convergence rates when $\nabla^2 \mathcal{L} \succ 0$. These notes are from the class DSC 243: Advanced Optimization taught by Dmitriy Drusvyatskiy at UCSD.