Giovanni Seraghiti – Optimization Online

An extrapolated and provably convergent algorithm for nonlinear matrix decomposition with the ReLU function

Published: 2025/03/31

Data Science Algorithms, Optimization in Data Science, Other Topics block coordinate minimization, low rank matrix, nonlinear matrix decomposition

Nonlinear matrix decomposition (NMD) with the ReLU function, denoted ReLU-NMD, is the following problem: given a sparse, nonnegative matrix \(X\) and a factorization rank \(r\), identify a rank-\(r\) matrix \(\Theta\) such that \(X\approx \max(0,\Theta)\). This decomposition finds application in data compression, matrix completion with entries missing not at random, and manifold learning. The standard ReLU-NMD … Read more

prunAdag: an adaptive pruning-aware gradient method

Published: 2025/02/12

Margherita Porcelli

Giovanni Seraghiti

Philippe L. Toint

Data Science Algorithms, Nonlinear Optimization adaptive first-order methods, Complexity theory, Model pruning, objective-function-free optimization (OFFO)

A pruning-aware adaptive gradient method is proposed which classifies the variables in two sets before updating them using different strategies. This technique extends the “relevant/irrelevant” approach of Ding (2019) and Zimmer et al. (2022) and allows a posteriori sparsification of the solution of model parameter fitting problems. The new method is proved to be convergent … Read more