aggregated gradient – Optimization Online

Analysis and Implementation of an Asynchronous Optimization Algorithm for the Parameter Server

Published: 2016/10/18

Convex and Nonsmooth Optimization, Nonlinear Optimization, Parallel Algorithms aggregated gradient, asynchronous, incremental, linear convergence, proximal

This paper presents an asynchronous incremental aggregated gradient algorithm and its implementation in a parameter server framework for solving regularized optimization problems. The algorithm can handle both general convex (possibly non-smooth) regularizers and general convex constraints. When the empirical data loss is strongly convex, we establish linear convergence rate, give explicit expressions for step-size choices … Read more

SMART: The Stochastic Monotone Aggregated Root-Finding Algorithm

Published: 2015/12/10, Updated: 2015/12/29

Damek Davis

Convex and Nonsmooth Optimization aggregated gradient, asynchronous updates, coordinate updates, operator splitting, stochastic algorithm, variance reduction

We introduce the Stochastic Monotone Aggregated Root-Finding (SMART) algorithm, a new randomized operator-splitting scheme for finding roots of finite sums of operators. These algorithms are similar to the growing class of incremental aggregated gradient algorithms, which minimize finite sums of functions; the difference is that we replace gradients of functions with black-boxes called operators, which … Read more