block coordinate minimization – Optimization Online

An extrapolated and provably convergent algorithm for nonlinear matrix decomposition with the ReLU function

Published: 2025/03/31

Data Science Algorithms, Optimization in Data Science, Other Topics block coordinate minimization, low rank matrix, nonlinear matrix decomposition

Nonlinear matrix decomposition (NMD) with the ReLU function, denoted ReLU-NMD, is the following problem: given a sparse, nonnegative matrix \(X\) and a factorization rank \(r\), identify a rank-\(r\) matrix \(\Theta\) such that \(X\approx \max(0,\Theta)\). This decomposition finds application in data compression, matrix completion with entries missing not at random, and manifold learning. The standard ReLU-NMD … Read more

Parallel Block Coordinate Minimization with Application to Group Regularized Regression

Published: 2015/03/13

Giuseppe Calafiore

Convex and Nonsmooth Optimization block coordinate minimization, elastic net, group lasso, nondifferentiable minimization, parallel methods, regularized regression, ridge regression

This paper proposes a method for parallel block coordinate-wise minimization for convex functions. Each iteration involves a first phase where n independent minimizations are performed over the n variable blocks, followed by a phase where the results of the first phase are coordinated to obtain the whole variable update. Convergence of the method to the … Read more