convergence rate

Fast convergence of the primal-dual dynamical system and algorithms for a nonsmooth bilinearly coupled saddle point problem

  • Kewei Ding
  • Jörg Fliege
  • Phan Vuong
    • Categories Game Theory, Nonsmooth Optimization Tags , , , ,

    \(\) This paper is devoted to study the convergence rates of a second-order dynamical system and its corresponding discretizations associated with a nonsmooth bilinearly coupled convex-concave saddle point problem. We derive the convergence rate of the primal-dual gap for the second-order dynamical system with asymptotically vanishing damping term. Based on the implicit discretization, we propose … Read more

    The convergence rate of the Sandwiching algorithm for convex bounded multiobjective optimization

  • Ina Lammel
  • Karl-Heinz Küfer
    • Categories Convex Optimization, Multi-Criteria Optimization Tags , , ,

    Sandwiching algorithms, also known as Benson-type algorithms, approximate the nondominated set of convex bounded multiobjective optimization problems by constructing and iteratively improving polyhedral inner and outer approximations. Using a set-valued metric, an estimate of the approximation quality is determined as the distance between the inner and outer approximation. The convergence of the algorithm is evaluated … Read more

    Goldstein Stationarity in Lipschitz Constrained Optimization

  • Benjamin Grimmer
  • Zhichao Jia
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags , , , ,

    We prove the first convergence guarantees for a subgradient method minimizing a generic Lipschitz function over generic Lipschitz inequality constraints. No smoothness or convexity (or weak convexity) assumptions are made. Instead, we utilize a sequence of recent advances in Lipschitz unconstrained minimization, which showed convergence rates of $O(1/\delta\epsilon^3)$ towards reaching a “Goldstein” stationary point, that … Read more

    Exact convergence rate of the last iterate in subgradient methods

  • Moslem Zamani
  • François Glineur
    • Categories Convex Optimization, Nonsmooth Optimization Tags , , , , , , ,

    \(\) We study the convergence of the last iterate in subgradient methods applied to the minimization of a nonsmooth convex function with bounded subgradients. We first introduce a proof technique that generalizes the standard analysis of subgradient methods. It is based on tracking the distance between the current iterate and a different reference point at … Read more

    On a Frank-Wolfe Approach for Abs-smooth Functions

  • Timo Kreimeier
  • Sebastian Pokutta
  • Andrea Walther
  • Zev Woodstock
    • Categories Constrained Nonlinear Optimization, Nonsmooth Optimization, Other Topics Tags , , , ,

    We propose an algorithm which appears to be the first bridge between the fields of conditional gradient methods and abs-smooth optimization. Our problem setting is motivated by various applications that lead to nonsmoothness, such as $\ell_1$ regularization, phase retrieval problems, or ReLU activation in machine learning. To handle the nonsmoothness in our problem, we propose … Read more

    Faster Lagrangian-based methods: a unified prediction-correction framework

  • Zhang Tao
  • Xia Yong
  • Li Shiru
    • Categories Convex Optimization Tags , , ,

    Motivated by the prediction-correction framework constructed by He and Yuan [SIAM J. Numer. Anal. 50: 700-709, 2012], we propose a unified prediction-correction framework to accelerate Lagrangian-based methods. More precisely, for strongly convex optimization, general linearized Lagrangian method with indefinite proximal term, alternating direction method of multipliers (ADMM) with the step size of Lagrangian multiplier not … Read more

    Optimal Convergence Rates for the Proximal Bundle Method

  • Mateo Díaz
  • Benjamin Grimmer
    • Categories Convex Optimization, Unconstrained Optimization Tags , , , ,

    We study convergence rates of the classic proximal bundle method for a variety of nonsmooth convex optimization problems. We show that, without any modification, this algorithm adapts to converge faster in the presence of smoothness or a Hölder growth condition. Our analysis reveals that with a constant stepsize, the bundle method is adaptive, yet it … Read more

    Smoothing fast iterative hard thresholding algorithm for $\ell_0$ regularized nonsmooth convex regression problem

  • Wei Bian
  • Fan Wu
  • Xiaoping Xue
    • Categories Nonlinear Optimization Tags , , , , ,

    We investigate a class of constrained sparse regression problem with cardinality penalty, where the feasible set is defined by box constraint, and the loss function is convex, but not necessarily smooth. First, we put forward a smoothing fast iterative hard thresholding (SFIHT) algorithm for solving such optimization problems, which combines smoothing approximations, extrapolation techniques and … Read more

    Heteroscedasticity-aware residuals-based contextual stochastic optimization

  • Rohit Kannan
  • Güzin Bayraksan
  • James R. Luedtke
    • Categories Robust Optimization, Stochastic Programming Tags , , , , , ,

    We explore generalizations of some integrated learning and optimization frameworks for data-driven contextual stochastic optimization that can adapt to heteroscedasticity. We identify conditions on the stochastic program, data generation process, and the prediction setup under which these generalizations possess asymptotic and finite sample guarantees for a class of stochastic programs, including two-stage stochastic mixed-integer programs … Read more

    Some Modified Fast Iteration Shrinkage Thresholding Algorithms with a New Adaptive Non-monotone Stepsize Strategy for Nonsmooth and Convex Minimization Problems

  • Hongwei Liu
  • Wang Ting
  • Zexian Liu
    • Categories Convex and Nonsmooth Optimization Tags , , , , ,

    The ” fast iterative shrinkage-thresholding algorithm ” (FISTA) is one of the most famous first order optimization scheme, and the stepsize, which plays an important role in theoretical analysis and numerical experiment, is always determined by a constant related to the Lipschitz constant or by a backtracking strategy. In this paper, we design a new … Read more