parallel coordinate descent – Optimization Online

Smooth minimization of nonsmooth functions with parallel coordinate descent methods

Published: 2013/09/23

Convex Optimization, Parallel Algorithms big data, convex optimization, parallel adaboost, parallel coordinate descent, separability

We study the performance of a family of randomized parallel coordinate descent methods for minimizing the sum of a nonsmooth and separable convex functions. The problem class includes as a special case L1-regularized L1 regression and the minimization of the exponential loss (“AdaBoost problem”). We assume the input data defining the loss function is contained … Read more

Parallel Coordinate Descent Methods for Big Data Optimization

Published: 2012/11/24, Updated: 2012/12/04

Peter Richtarik

Martin Takac

Convex Optimization, Parallel Algorithms big data optimization, composite objective, convex optimization, expected separable over-approximation, huge scale optimization, iteration complexity, lasso, parallel coordinate descent, partial separability

In this work we show that randomized (block) coordinate descent methods can be accelerated by parallelization when applied to the problem of minimizing the sum of a partially separable smooth convex function and a simple separable convex function. The theoretical speedup, as compared to the serial method, and referring to the number of iterations needed … Read more