Zhaosong Lu

Accelerated first-order methods for convex optimization with locally Lipschitz continuous gradient

  • Zhaosong Lu
  • Sanyou Mei
    • Categories Convex and Nonsmooth Optimization Tags , , , , , ,

    In this paper we develop accelerated first-order methods for convex optimization with locally Lipschitz continuous gradient (LLCG), which is beyond the well-studied class of convex optimization with Lipschitz continuous gradient. In particular, we first consider unconstrained convex optimization with LLCG and propose accelerated proximal gradient (APG) methods for solving it. The proposed APG methods are … Read more

    Randomized block proximal damped Newton method for composite self-concordant minimization

  • Zhaosong Lu
    • Categories Convex and Nonsmooth Optimization Tags , , ,

    In this paper we consider the composite self-concordant (CSC) minimization problem, which minimizes the sum of a self-concordant function $f$ and a (possibly nonsmooth) proper closed convex function $g$. The CSC minimization is the cornerstone of the path-following interior point methods for solving a broad class of convex optimization problems. It has also found numerous … Read more

    Generalized Conjugate Gradient Methods for $\ell_1$ Regularized Convex Quadratic Programming with Finite Convergence

  • Xiaojun Chen
  • Zhaosong Lu
    • Categories Convex Optimization, Nonsmooth Optimization, Quadratic Programming Tags , , , ,

    The conjugate gradient (CG) method is an efficient iterative method for solving large-scale strongly convex quadratic programming (QP). In this paper we propose some generalized CG (GCG) methods for solving the $\ell_1$-regularized (possibly not strongly) convex QP that terminate at an optimal solution in a finite number of iterations. At each iteration, our methods first … Read more

    Sparse Recovery via Partial Regularization: Models, Theory and Algorithms

  • Xiaorui Li
  • Zhaosong Lu
    • Categories Nonlinear Optimization, Nonsmooth Optimization, Statistics Tags , , , , , , ,

    In the context of sparse recovery, it is known that most of existing regularizers such as $\ell_1$ suffer from some bias incurred by some leading entries (in magnitude) of the associated vector. To neutralize this bias, we propose a class of models with partial regularizers for recovering a sparse solution of a linear system. We … Read more

    Optimization over Sparse Symmetric Sets via a Nonmonotone Projected Gradient Method

  • Zhaosong Lu
    • Categories Combinatorial Optimization, Global Optimization, Nonlinear Optimization Tags , , ,

    We consider the problem of minimizing a Lipschitz differentiable function over a class of sparse symmetric sets that has wide applications in engineering and science. For this problem, it is known that any accumulation point of the classical projected gradient (PG) method with a constant stepsize $1/L$ satisfies the $L$-stationarity optimality condition that was introduced … Read more

    A proximal gradient method for ensemble density functional theory

  • Dennis Klöckner
  • Zhaosong Lu
  • Michael Ulbrich
  • Zaiwen Wen
  • Chao Yang
    • Categories Basic Sciences Applications, Constrained Nonlinear Optimization Tags , , ,

    The ensemble density functional theory is valuable for simulations of metallic systems due to the absence of a gap in the spectrum of the Hamiltonian matrices. Although the widely used self-consistent field iteration method can be extended to solve the minimization of the total energy functional with respect to orthogonality constraints, there is no theoretical … Read more

    An Accelerated Proximal Coordinate Gradient Method and its Application to Regularized Empirical Risk Minimization

  • Qihang Lin
  • Zhaosong Lu
  • Lin Xiao
    • Categories Applications - Science and Engineering, Convex and Nonsmooth Optimization, Stochastic Programming Tags , ,

    We consider the problem of minimizing the sum of two convex functions: one is smooth and given by a gradient oracle, and the other is separable over blocks of coordinates and has a simple known structure over each block. We develop an accelerated randomized proximal coordinate gradient (APCG) method for minimizing such convex composite functions. … Read more

    Randomized Block Coordinate Non-Monotone Gradient Method for a Class of Nonlinear Programming

  • Zhaosong Lu
  • Lin Xiao
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , ,

    In this paper we propose a randomized block coordinate non-monotone gradient (RBCNMG) method for minimizing the sum of a smooth (possibly nonconvex) function and a block-separable (possibly nonconvex nonsmooth) function. At each iteration, this method randomly picks a block according to any prescribed probability distribution and typically solves several associated proximal subproblems that usually have … Read more

    On the Complexity Analysis of Randomized Block-Coordinate Descent Methods

  • Zhaosong Lu
  • Lin Xiao
    • Categories Convex and Nonsmooth Optimization Tags , , , ,

    In this paper we analyze the randomized block-coordinate descent (RBCD) methods for minimizing the sum of a smooth convex function and a block-separable convex function. In particular, we extend Nesterov’s technique (SIOPT 2012) for analyzing the RBCD method for minimizing a smooth convex function over a block-separable closed convex set to the aforementioned more general … Read more

    Iterative Hard Thresholding Methods for $ Regularized Convex Cone Programming

  • Zhaosong Lu
    • Categories Combinatorial Optimization, Nonsmooth Optimization Tags , , , ,

    In this paper we consider $l_0$ regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving $l_0$ regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer. Also, we establish the iteration complexity of the … Read more