memory gradient methods – Optimization Online

Convergence Analysis of Block Majorize-Minimize Subspace Approaches

Published: 2021/12/06, Updated: 2022/02/05

Applications - Science and Engineering, Unconstrained Optimization block alternating optimization, majorization-minimization, memory gradient methods, non-convex optimization

Majorization-Minimization (MM) consists of a class of efficient and effective optimization algorithms that benefit from solid theoretical foundations. MM methods have shown their great ability to tackle efficiently challenging optimization problems from signal processing, image processing, inverse problems and machine learning. When processing large amount of data/variable, as it may happen in 3D image processing, … Read more

A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation

Published: 2016/01/05

Émilie Chouzenoux

Jean-Christophe Pesquet

Convex Optimization, Unconstrained Optimization adaptive filtering, descent methods, filter identification, machine learning, majorization-minimization, memory gradient methods, newton method, optimization, recursive algorithms, sparsity, stochastic approximation, subspace algorithms

Stochastic approximation techniques play an important role in solving many problems encountered in machine learning or adaptive signal processing. In these contexts, the statistics of the data are often unknown a priori or their direct computation is too intensive, and they have thus to be estimated online from the observed signals. For batch optimization of … Read more