Convex and Nonsmooth Optimization

Dual-density-based reweighted $\ell_{1}hBcalgorithms for a class of $\ell_{0}hBcminimization problems

  • Yun-Bin Zhao
    • Categories Convex and Nonsmooth Optimization Tags , , , , ,

    The optimization problem with sparsity arises in many areas of science and engineering such as compressed sensing, image processing, statistical learning and data sparse approximation. In this paper, we study the dual-density-based reweighted $\ell_{1}$-algorithms for a class of $\ell_{0}$-minimization models which can be used to model a wide range of practical problems. This class of … Read more

    A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

  • Dunbiao Niu
  • Enbin Song
  • Peipei Tang
  • Qingsong Wang
  • Chengjing Wang
    • Categories Convex and Nonsmooth Optimization, Data-Mining Tags , ,

    Support vector machines (SVMs) are successful modeling and prediction tools with a variety of applications. Previous work has demonstrated the superiority of the SVMs in dealing with the high dimensional, low sample size problems. However, the numerical difficulties of the SVMs will become severe with the increase of the sample size. Although there exist many … Read more

    Continuous selections of solutions for locally Lipschitzian equations

  • A.V. Arutyunov
  • Alexey F. Izmailov
  • S.E. Zhukovskiy
    • Categories Complementarity and Variational Inequalities, Nonlinear Systems and Least-Squares, Nonsmooth Optimization

    This paper answers in affirmative the long-standing question of nonlinear analysis, concerning the existence of a continuous single-valued local selection of the right inverse to a locally Lipschitzian mapping. Moreover, we develop a much more general result, providing conditions for the existence of a continuous single-valued selection not only locally, but rather on any given … Read more

    Objective Selection for Cancer Treatment: An Inverse Optimization Approach

  • Temitayo Ajayi
  • Andrew J. Schaefer
  • Taewoo Lee
    • Categories Biomedical Applications, Convex and Nonsmooth Optimization, Multi-Criteria Optimization Tags , , , , , ,

    In radiation therapy treatment-plan optimization, selecting a set of clinical objectives that are tractable and parsimonious yet effective is a challenging task. In clinical practice, this is typically done by trial and error based on the treatment planner’s subjective assessment, which often makes the planning process inefficient and inconsistent. We develop the objective selection problem … Read more

    Stochastic generalized gradient methods for training nonconvex nonsmooth neural networks

  • Vladimir I. Norkin
    • Categories Nonlinear Optimization, Nonsmooth Optimization, Stochastic Programming Tags , , , , ,

    The paper observes a similarity between the stochastic optimal control of discrete dynamical systems and the learning multilayer neural networks. It focuses on contemporary deep networks with nonconvex nonsmooth loss and activation functions. The machine learning problems are treated as nonconvex nonsmooth stochastic optimization problems. As a model of nonsmooth nonconvex dependences, the so-called generalized … Read more

    An analysis of the superiorization method via the principle of concentration of measure

  • Yair Censor
  • Eliahu Levy
    • Categories Constrained Nonlinear Optimization, Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , , , , , , ,

    The superiorization methodology is intended to work with input data of constrained minimization problems, i.e., a target function and a constraints set. However, it is based on an antipodal way of thinking to the thinking that leads constrained minimization methods. Instead of adapting unconstrained minimization algorithms to handling constraints, it adapts feasibility-seeking algorithms to reduce … Read more

    On Sum of Squares Representation of Convex Forms and Generalized Cauchy-Schwarz Inequalities

  • Bachir El Khadir
    • Categories Convex Optimization, Global Optimization Theory, Semi-definite Programming Tags , ,

    A convex form of degree larger than one is always nonnegative since it vanishes together with its gradient at the origin. In 2007, Parrilo asked if convex forms are always sums of squares. A few years later, Blekherman answered the question in the negative by showing through volume arguments that for high enough number of … Read more

    Substantiation of the Backpropagation Technique via the Hamilton-Pontryagin Formalism for Training Nonconvex Nonsmooth Neural Networks

  • Vladimir I. Norkin
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization, Stochastic Programming Tags , , , , ,

    The paper observes the similarity between the stochastic optimal control of discrete dynamical systems and the training multilayer neural networks. It focuses on contemporary deep networks with nonconvex nonsmooth loss and activation functions. In the paper, the machine learning problems are treated as nonconvex nonsmooth stochastic optimization problems. As a model of nonsmooth nonconvex dependences, … Read more

    Probabilistic guarantees in Robust Optimization

  • Dimitris Bertsimas
  • Dick den Hertog
  • Jean Pauphilet
    • Categories Convex Optimization, Robust Optimization

    We develop a general methodology to derive probabilistic guarantees for solutions of robust optimization problems. Our analysis applies broadly to any convex compact uncertainty set and to any constraint affected by uncertainty in a concave manner, under minimal assumptions on the underlying stochastic process. Namely, we assume that the coordinates of the noise vector are … Read more

    Generalized Gradients in Problems of Dynamic Optimization, Optimal Control, and Machine Learning

  • Vladimir I. Norkin
    • Categories Nonsmooth Optimization, Stochastic Programming Tags , , , , , , ,

    In this work, nonconvex nonsmooth problems of dynamic optimization, optimal control in discrete time (including feedback control), and machine learning are considered from a common point of view. An analogy is observed between tasks of controlling discrete dynamic systems and training multilayer neural networks with nonsmooth target function and connections. Methods for calculating generalized gradients … Read more