Jiaming Liang

Improved Analysis of Restarted Accelerated Gradient and Augmented Lagrangian Methods via Inexact Proximal Point Frameworks

  • Matthew X. Burns
  • Jiaming Liang
    • Categories Constrained Nonlinear Optimization, Convex and Nonsmooth Optimization, Convex Optimization Tags , , , ,

    This paper studies a class of double-loop (inner-outer) algorithms for convex composite optimization. For unconstrained problems, we develop a restarted accelerated composite gradient method that attains the optimal first-order complexity in both the convex and strongly convex settings. For linearly constrained problems, we introduce inexact augmented Lagrangian methods, including a basic method and an outer-accelerated … Read more

    GFORS: GPU-Accelerated First-Order Method with Randomized Sampling for Binary Integer Programs

  • Ningji Wei
  • Jiaming Liang
    • Categories 0-1 Programming, Constrained Nonlinear Optimization, Parallel Algorithms Tags , ,

    We present GFORS, a GPU-accelerated framework for large binary integer programs. It couples a first-order (PDHG-style) routine that guides the search in the continuous relaxation with a randomized, feasibility-aware sampling module that generates batched binary candidates. Both components are designed to run end-to-end on GPUs with minimal CPU–GPU synchronization. The framework establishes near-stationary-point guarantees for … Read more

    Multi-cut stochastic approximation methods for solving stochastic convex composite optimization

  • Jiaming Liang
  • Honghao Zhang
  • Renato D.C. Monteiro
    • Categories Convex Optimization, Stochastic Programming Tags , , ,

    This paper considers the stochastic convex composite optimization problem and presents multi-cut stochastic approximation (SA) methods for solving it, whose models in expectation overestimate its objective function. The multi-cut model obtained by taking the maximum of a finite number of linearizations of the stochastic objective function provides a biased estimate of the objective function, with … Read more

    Unifying restart accelerated gradient and proximal bundle methods

  • Jiaming Liang
    • Categories Convex and Nonsmooth Optimization, Convex Optimization Tags , , , ,

    This paper presents a novel restarted version of Nesterov’s accelerated gradient method and establishes its optimal iteration-complexity for solving convex smooth composite optimization problems. The proposed restart accelerated gradient method is shown to be a specific instance of the accelerated inexact proximal point framework introduced in “An accelerated hybrid proximal extragradient method for convex optimization … Read more

    Primal-dual proximal bundle and conditional gradient methods for convex problems

  • Jiaming Liang
    • Categories Convex and Nonsmooth Optimization, Convex Optimization, Nonsmooth Optimization Tags , , , , ,

    This paper studies the primal-dual convergence and iteration-complexity of proximal bundle methods for solving nonsmooth problems with convex structures. More specifically, we develop a family of primal-dual proximal bundle methods for solving convex nonsmooth composite optimization problems and establish the iteration-complexity in terms of a primal-dual gap. We also propose a class of proximal bundle … Read more

    Universal subgradient and proximal bundle methods for convex and strongly convex hybrid composite optimization

  • Vincent Guigues
  • Jiaming Liang
  • Renato D.C. Monteiro
    • Categories Convex Optimization, Nonsmooth Optimization Tags , , ,

    This paper develops two parameter-free methods for solving convex and strongly convex hybrid composite optimization problems, namely, a composite subgradient type method and a proximal bundle type method. Both functional and stationary complexity bounds for the two methods are established in terms of the unknown strong convexity parameter. To the best of our knowledge, the … Read more

    Variance Reduction and Low Sample Complexity in Stochastic Optimization via Proximal Point Method

  • Jiaming Liang
    • Categories Convex Optimization, Stochastic Programming Tags , , , , ,

    High-probability guarantees in stochastic optimization are often obtained only under strong noise assumptions such as sub-Gaussian tails. We show that such guarantees can also be achieved under the weaker assumption of bounded variance by developing a stochastic proximal point method. This method combines a proximal subproblem solver, which inherently reduces variance, with a probability booster … Read more

    Proximal bundle methods for hybrid weakly convex composite optimization problems

  • Jiaming Liang
  • Renato D.C. Monteiro
  • Honghao Zhang
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , , ,

    This paper establishes the iteration-complexity of proximal bundle methods for solving hybrid (i.e., a blend of smooth and nonsmooth) weakly convex composite optimization (HWC-CO) problems. This is done in a unified manner by considering a proximal bundle framework (PBF) based on a generic bundle update framework which includes various well-known bundle update schemes. In contrast … Read more

    A single cut proximal bundle method for stochastic convex composite optimization

  • Jiaming Liang
  • Vincent Guigues
  • Renato D.C. Monteiro
    • Categories Convex and Nonsmooth Optimization, Stochastic Programming Tags , , ,

    This paper considers optimization problems where the objective is the sum of a function given by an expectation and a closed convex composite function, and proposes stochastic composite proximal bundle (SCPB) methods for solving it. Complexity guarantees are established for them without requiring knowledge of parameters associated with the problem instance. Moreover, it is shown … Read more

    A unified analysis of a class of proximal bundle methods for solving hybrid convex composite optimization problems

  • Jiaming Liang
  • Renato D.C. Monteiro
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , ,

    This paper presents a proximal bundle (PB) framework based on a generic bundle update scheme for solving the hybrid convex composite optimization (HCCO) problem and establishes a common iteration-complexity bound for any variant belonging to it. As a consequence, iteration-complexity bounds for three PB variants based on different bundle update schemes are obtained in the … Read more