A Derivation of Nesterov’s Accelerated Gradient Algorithm from Optimal Control Theory

Nesterov’s accelerated gradient algorithm is derived from first principles. The first principles are founded on the recently-developed optimal control theory for optimization. The necessary conditions for optimal control generate a controllable dynamical system for accelerated optimization. Stabilizing this system via a control Lyapunov function generates an ordinary differential equation. An Euler discretization of the differential equation produces Nesterov’s algorithm.

Article

Download

View PDF