proximal gradient methods

A Stochastic Proximal-Gradient Method for Regularized Generally Constrained Continuous Optimization

Categories Constrained Nonlinear Optimization, Nonlinear Optimization, Nonsmooth Optimization, Stochastic Programming Tags , , , ,

A stochastic proximal-gradient method is proposed and analyzed for solving regularized optimization problems subject to nonlinear equality constraints and nonnegativity bounds. The objective function’s values and gradients are available only through inexact stochastic oracles. The method decomposes each trial step into one component that reduces linearized infeasibility and a second component that seeks optimality while … Read more

On the Convergence and Complexity of Proximal Gradient and Accelerated Proximal Gradient Methods under Adaptive Gradient Estimation

Categories Nonlinear Optimization, Optimization in Data Science, Stochastic Programming Tags , , , ,

In this paper, we propose a proximal gradient method and an accelerated proximal gradient method for solving composite optimization problems, where the objective function is the sum of a smooth and a convex, possibly nonsmooth, function. We consider settings where the smooth component is either a finite-sum function or an expectation of a stochastic function, … Read more

Barzilai-Borwein-like rules in proximal gradient schemes for ℓ1−regularized problems

Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , ,

We propose a novel steplength selection rule in proximal gradient methods for minimizing the sum of a differentiable function plus an ℓ1-norm penalty term. The proposed rule modifies one of the classical Barzilai-Borwein steplength, extending analogous results obtained in the context of gradient projection methods for constrained optimization. We analyze the spectral properties of the … Read more

”Active-set complexity” of proximal gradient: How long does it take to find the sparsity pattern?

Categories Convex and Nonsmooth Optimization, Convex Optimization, Nonsmooth Optimization Tags , ,

Proximal gradient methods have been found to be highly effective for solving minimization problems with non-negative constraints or L1-regularization. Under suitable nondegeneracy conditions, it is known that these algorithms identify the optimal sparsity pattern for these types of problems in a finite number of iterations. However, it is not known how many iterations this may … Read more