proximal method – Optimization Online

A Distributed Quasi-Newton Algorithm for Primal and Dual Regularized Empirical Risk Minimization

Published: 2019/12/12, Updated: 2019/12/17

Nonsmooth Optimization, Parallel Algorithms distributed optimization, empirical riskminimization, inexact method, nonsmooth optimization, proximal method, quasi-newton methods, regularized optimization, variable metrics

We propose a communication- and computation-efficient distributed optimization algorithm using second-order information for solving empirical risk minimization (ERM) problems with a nonsmooth regularization term. Our algorithm is applicable to both the primal and the dual ERM problem. Current second-order and quasi-Newton methods for this problem either do not work well in the distributed setting or … Read more

A Distributed Quasi-Newton Algorithm for Empirical Risk Minimization with Nonsmooth Regularization

Published: 2018/03/04, Updated: 2018/05/26

Stephen Wright

Cong Han Lim

Ching-pei Lee

Data-Mining, Nonlinear Optimization, Parallel Algorithms distributed optimization, empirical risk minimization, inexact method, nonsmooth optimization, proximal method, quasi-newton methods, regularized optimization, variable metrics

We propose a communication- and computation-efficient distributed optimization algorithm using second-order information for solving ERM problems with a nonsmooth regularization term. Current second-order and quasi-Newton methods for this problem either do not work well in the distributed setting or work only for specific regularizers. Our algorithm uses successive quadratic approximations, and we describe how to … Read more

Inexact Successive Quadratic Approximation for Regularized Optimization

Published: 2018/03/03, Updated: 2019/01/24

Stephen Wright

Ching-pei Lee

Nonlinear Optimization, Nonsmooth Optimization convex optimization, inexact method, nonconvex optimization, proximal method, regularized optimization, second-order approximation, variable metric

Successive quadratic approximations, or second-order proximal methods, are useful for minimizing functions that are a sum of a smooth part and a convex, possibly nonsmooth part that promotes regularization. Most analyses of iteration complexity focus on the special case of proximal gradient method, or accelerated variants thereof. There have been only a few studies of … Read more

The proximal point method for locally Lipschitz functions in multiobjective optimization

Published: 2016/06/21

Glaydston Bento

João Xavier da Cruz Neto

Antoine Soubeyran

João Carlos Souza

G. López

Applications - OR and Management Sciences, Multi-Criteria Optimization compromise problem, locally lipschitz function, multiobjective optimization, pareto critical point, proximal method, variational rationality

This paper studies the constrained multiobjective optimization problem of finding Pareto critical points of vector-valued functions. The proximal point method considered by Bonnel et al. (SIAM J. Optim., 4 (2005), pp. 953-970) is extended to locally Lipschitz functions in the finite dimensional multiobjective setting. To this end, a new approach for convergence analysis of the … Read more

A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms

Published: 2011/12/16

Laurent Condat

Convex and Nonsmooth Optimization convex and nonsmooth optimization, douglas-rachford method, fenchel-rockafellar duality, forward-backward method, monotone inclusion, monotone operator, operator splitting, primal-dual algorithms, proximal method, saddle point problem

We propose a new first-order splitting algorithm for solving jointly the primal and dual formulations of large-scale convex minimization problems involving the sum of a smooth function with Lipschitzian gradient, a nonsmooth proximable function, and linear composite functions. This is a full splitting approach in the sense that the gradient and the linear operators involved … Read more

Interior Proximal Algorithm with Variable Metric for Second-Order Cone Programming: Applications to Structural Optimization and Support Vector Machines

Published: 2009/06/05

Felipe Álvarez

Julio López

Héctor Ramírez

Second-Order Cone Programming multiple loads problem, proximal method, second-order cone programming, structural optimization, support vector machines, variable metric

In this work, we propose an inexact interior proximal type algorithm for solving convex second-order cone programs. This kind of problems consists of minimizing a convex function (possibly nonsmooth) over the intersection of an affine linear space with the Cartesian product of second-order cones. The proposed algorithm uses a distance variable metric, which is induced … Read more