extrapolation

Efficient Low-rank Identification via Accelerated Iteratively Reweighted Nuclear Norm Minimization

  • Xiangyu Yang
  • Hao Wang
    • Categories Nonsmooth Optimization, Optimization in Data Science Tags , , , ,

    \(\) This paper considers the problem of minimizing the sum of a smooth function and the Schatten-\(p\) norm of the matrix. Our contribution involves proposing accelerated iteratively reweighted nuclear norm methods designed for solving the nonconvex low-rank minimization problem. Two major novelties characterize our approach. Firstly, the proposed method possesses a rank identification property, enabling … Read more

    On Averaging and Extrapolation for Gradient Descent

  • Alan Luner
  • Benjamin Grimmer
    • Categories Convex Optimization, Semi-definite Programming Tags , , , ,

    \(\) This work considers the effect of averaging, and more generally extrapolation, of the iterates of gradient descent in smooth convex optimization. After running the method, rather than reporting the final iterate, one can report either a convex combination of the iterates (averaging) or a generic combination of the iterates (extrapolation). For several common stepsize … Read more

    Block Majorization Minimization with Extrapolation and Application to $\beta$-NMF

  • Le Thi Khanh Hien
  • Valentin Leplat
  • Nicolas Gillis
    • Categories Constrained Nonlinear Optimization, Convex and Nonsmooth Optimization, Optimization in Data Science Tags , , , ,

    We propose a Block Majorization Minimization method with Extrapolation (BMMe) for solving a class of multi-convex optimization problems. The extrapolation parameters of BMMe are updated using a novel adaptive update rule. By showing that block majorization minimization can be reformulated as a block mirror descent method, with the Bregman divergence adaptively updated at each iteration, … Read more

    Smoothing fast iterative hard thresholding algorithm for $\ell_0$ regularized nonsmooth convex regression problem

  • Wei Bian
  • Fan Wu
  • Xiaoping Xue
    • Categories Nonlinear Optimization Tags , , , , ,

    We investigate a class of constrained sparse regression problem with cardinality penalty, where the feasible set is defined by box constraint, and the loss function is convex, but not necessarily smooth. First, we put forward a smoothing fast iterative hard thresholding (SFIHT) algorithm for solving such optimization problems, which combines smoothing approximations, extrapolation techniques and … Read more

    A Framework of Inertial Alternating Direction Method of Multipliers for Non-Convex Non-Smooth Optimization

  • Nicolas Gillis
  • Le Thi Khanh Hien
  • Duy Nhat Phan
    • Categories Constrained Nonlinear Optimization Tags , , ,

    In this paper, we propose an algorithmic framework dubbed inertial alternating direction methods of multipliers (iADMM), for solving a class of nonconvex nonsmooth multiblock composite optimization problems with linear constraints. Our framework employs the general minimization-majorization (MM) principle to update each block of variables so as to not only unify the convergence analysis of previous … Read more