A Minimal-Gradient Subspace Method for Unconstrained Optimization

We propose a minimal-gradient subspace method for unconstrained optimization. For strictly convex quadratics, conjugate gradient can be interpreted as exact minimization over a two-dimensional affine subspace. We use the same reduced subspace in the nonlinear case, but compute a trial step by minimizing a local model of the next gradient norm. For SPD quadratics, every such subspace containing the gradient yields a uniform contraction of the gradient norm. In the nonlinear case, the trial step minimizes the norm of a linearized gradient model built from a Hessian approximation, but need not decrease the true objective. This step is accepted only when it satisfies a sufficient-decrease test; otherwise, the method uses a fallback along the negative-gradient direction. Experiments on synthetic SPD quadratics, real SPD matrices, and smooth PyCUTEst problems show a cost–robustness trade-off. The two- and three-dimensional quadratic variants have essentially the same iteration behavior. On PyCUTEst, the proposed method attains the highest success count, whereas L-BFGS requires fewer median gradient evaluations and less CPU time.

Article

Download

View PDF