Convex and Nonsmooth Optimization

Superiorization vs. Accelerated Convex Optimization: The Superiorized/Regularized Least-Squares Case

  • Yair Censor
  • Stefania Petra
  • Christoph Schnörr
    • Categories Biomedical Applications, Constrained Nonlinear Optimization, Convex Optimization Tags , , , , , ,

    In this paper we conduct a study of both superiorization and optimization approaches for the reconstruction problem of superiorized/regularized solutions to underdetermined systems of linear equations with nonnegativity variable bounds. Specifically, we study a (smoothed) total variation regularized least-squares problem with nonnegativity constraints. We consider two approaches: (a) a superiorization approach that, in contrast to … Read more

    Understanding Limitation of Two Symmetrized Orders by Worst-case Complexity

  • Ruoyu Sun
  • Zhisheng Xiao
  • Peijun Xiao
    • Categories Complementarity and Variational Inequalities, Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , ,

    It was recently found that the standard version of multi-block cyclic ADMM diverges. Interestingly, Gaussian Back Substitution ADMM (GBS-ADMM) and symmetric Gauss-Seidel ADMM (sGS-ADMM) do not have the divergence issue. Therefore, it seems that symmetrization can improve the performance of the classical cyclic order. In another recent work, cyclic CD (Coordinate Descent) was shown to … Read more

    Adaptive Sampling Quasi-Newton Methods for Derivative-Free Stochastic Optimization

  • Raghu Bollapragada
  • Stefan M. Wild
    • Categories Convex Optimization, Nonlinear Optimization, Stochastic Programming Tags , ,

    We consider stochastic zero-order optimization problems, which arise in settings from simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients of a stochastic function using finite differences within a common random number framework. We employ modified versions of a norm test and an inner product quasi-Newton test … Read more

    Fully adaptive proximal extrapolated gradient method for monotone variational inequalities

  • Chang Xiaokai
    • Categories Convex and Nonsmooth Optimization

    The paper presents a fully adaptive proximal extrapolated gradient method for monotone variational inequalities. The proposed method uses fully non-monotonic and adaptive step sizes, that are computed using two previous iterates as an approximation of the locally Lipschitz constant without running a linesearch. Thus, it has almost the same low computational cost as classic proximal … Read more

    Adaptive Gradient Descent without Descent

  • Yura Malitsky
  • Konstantin Mishchenko
    • Categories Convex Optimization Tags , , ,

    We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don’t increase the stepsize too fast and 2) don’t overstep the local curvature. No need for functional values, no line search, no information about the function except for the gradients. By following these rules, you get a method adaptive … Read more

    Calculating Optimistic Likelihoods Using (Geodesically) Convex Optimization

  • Daniel Kuhn
  • Viet Anh Nguyen
  • Soroosh Shafieezadeh-Abadeh
  • Wolfram Wiesemann
  • Man-Chung Yue
    • Categories Generalized Convexity/Monoticity, Semi-definite Programming, Stochastic Programming Tags , , ,

    A fundamental problem arising in many areas of machine learning is the evaluation of the likelihood of a given observation under different nominal distributions. Frequently, these nominal distributions are themselves estimated from data, which makes them susceptible to estimation errors. We thus propose to replace each nominal distribution with an ambiguity set containing all distributions … Read more

    An Oblivious Ellipsoid Algorithm for Solving a System of (In)Feasible Linear Inequalities

  • Robert M. Freund
  • Michael J. Todd
  • Jourdain Lamperski
    • Categories Convex Optimization, Linear Programming, Nonsmooth Optimization Tags , , , ,

    The ellipsoid algorithm is a fundamental algorithm for computing a solution to the system of m linear inequalities in n variables (P) when its set of solutions has positive volume. However, when (P) is infeasible, the ellipsoid algorithm has no mechanism for proving that (P) is infeasible. This is in contrast to the other two … Read more

    Dual-density-based reweighted $\ell_{1}hBcalgorithms for a class of $\ell_{0}hBcminimization problems

  • Yun-Bin Zhao
    • Categories Convex and Nonsmooth Optimization Tags , , , , ,

    The optimization problem with sparsity arises in many areas of science and engineering such as compressed sensing, image processing, statistical learning and data sparse approximation. In this paper, we study the dual-density-based reweighted $\ell_{1}$-algorithms for a class of $\ell_{0}$-minimization models which can be used to model a wide range of practical problems. This class of … Read more

    A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

  • Dunbiao Niu
  • Enbin Song
  • Peipei Tang
  • Qingsong Wang
  • Chengjing Wang
    • Categories Convex and Nonsmooth Optimization, Data-Mining Tags , ,

    Support vector machines (SVMs) are successful modeling and prediction tools with a variety of applications. Previous work has demonstrated the superiority of the SVMs in dealing with the high dimensional, low sample size problems. However, the numerical difficulties of the SVMs will become severe with the increase of the sample size. Although there exist many … Read more

    Objective Selection for Cancer Treatment: An Inverse Optimization Approach

  • Temitayo Ajayi
  • Andrew J. Schaefer
  • Taewoo Lee
    • Categories Biomedical Applications, Convex and Nonsmooth Optimization, Multi-Criteria Optimization Tags , , , , , ,

    In radiation therapy treatment-plan optimization, selecting a set of clinical objectives that are tractable and parsimonious yet effective is a challenging task. In clinical practice, this is typically done by trial and error based on the treatment planner’s subjective assessment, which often makes the planning process inefficient and inconsistent. We develop the objective selection problem … Read more