regularization

On Generalization and Regularization via Wasserstein Distributionally Robust Optimization

  • Jonathan Yu-Meng Li
    • Categories Robust Optimization, Stochastic Programming Tags , , ,

    Wasserstein distributionally robust optimization (DRO) has found success in operations research and machine learning applications as a powerful means to obtain solutions with favourable out-of-sample performances. Two compelling explanations for the success are the generalization bounds derived from Wasserstein DRO and the equivalency between Wasserstein DRO and the regularization scheme commonly applied in machine learning. … Read more

    Tractable Robust Supervised Learning Models

  • Melvyn Sim
  • Long Zhao
  • Minglong Zhou
    • Categories Robust Optimization, Stochastic Programming Tags , , , ,

    At the heart of supervised learning is a minimization problem with an objective function that evaluates a set of training data over a loss function that penalizes poor fitting and a regularization function that penalizes over-fitting to the training data. More recently, data-driven robust optimization based learning models provide an intuitive robustness perspective of regularization. … Read more

    Cardinality Minimization, Constraints, and Regularization: A Survey

  • Daniel Bienstock
  • Andrea Lodi
  • Andreas M. Tillmann
  • Alexandra Schwartz
    • Categories (Mixed) Integer Linear Programming, (Mixed) Integer Nonlinear Programming, Constrained Nonlinear Optimization Tags , , , , , , ,

    We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and give concrete examples from diverse application fields such as signal and image processing, portfolio selection, or machine learning. The paper discusses general-purpose modeling techniques and … Read more

    New efficient approach in finding a zero of a maximal monotone operator

  • Ba Khiet Le
    • Categories Convex Optimization, Nonsmooth Optimization Tags , , ,

    In the paper, we provide a new efficient approach to find a zero of a maximal monotone operator under very mild assumptions. Using a regularization technique and the proximal point algorithm, we can construct a sequence that converges strongly to a solution with at least linear convergence rate. Article Download View New efficient approach in … Read more

    A Subspace Acceleration Method for Minimization Involving a Group Sparsity-Inducing Regularizer

  • Frank E. Curtis
  • Daniel P. Robinson
  • Yutong Dai
    • Categories Nonsmooth Optimization, Unconstrained Optimization Tags , , , , , , ,

    We consider the problem of minimizing an objective function that is the sum of a convex function and a group sparsity-inducing regularizer. Problems that integrate such regularizers arise in modern machine learning applications, often for the purpose of obtaining models that are easier to interpret and that have higher predictive accuracy. We present a new … Read more

    High-order Evaluation Complexity of a Stochastic Adaptive Regularization Algorithm for Nonconvex Optimization Using Inexact Function Evaluations and Randomly Perturbed Derivatives

  • Stefania Bellavia
  • Benedetta Morini
  • Philippe L. Toint
  • Gianmarco Gurioli
    • Categories Bound-constrained Optimization, Constrained Nonlinear Optimization, Unconstrained Optimization Tags , , ,

    A stochastic adaptive regularization algorithm allowing random noise in derivatives and inexact function values is proposed for computing strong approximate minimizers of any order for inexpensively constrained smooth optimization problems. For an objective function with Lipschitz continuous p-th derivative in a convex neighbourhood of the feasible set and given an arbitrary optimality order q, it … Read more

    A Limiting Analysis on Regularization of Singular SDP and its Implication to Infeasible Interior-point Algorithms

  • Bruno F. Lourenço
  • Masakazu Muramatsu
  • Takashi Tsuchiya
  • Takayuki Okuno
    • Categories Convex Optimization, Linear, Cone and Semidefinite Programming Tags , , ,

    We consider primal-dual pairs of semidefinite programs and assume that they are ill-posed, i.e., both primal and dual are either weakly feasible or weakly infeasible. Under such circumstances, strong duality may break down and the primal and dual might have a nonzero duality gap. Nevertheless, there are arbitrary small perturbations to the problem data which … Read more

    On the Cluster-aware Supervised Learning (CluSL): Frameworks, Convergent Algorithms, and Applications

  • Shutong Chen
  • Weijun Xie
    • Categories Data-Mining Tags , , , , ,

    This paper proposes a cluster-aware supervised learning (CluSL) framework, which integrates the clustering analysis with supervised learning (SL). The objective of CluSL is to simultaneously find the best clusters of the data points and minimize the sum of loss functions within each cluster. This framework has many potential applications in healthcare, operations management, manufacturing, and … Read more

    Objective Selection for Cancer Treatment: An Inverse Optimization Approach

  • Temitayo Ajayi
  • Andrew J. Schaefer
  • Taewoo Lee
    • Categories Biomedical Applications, Convex and Nonsmooth Optimization, Multi-Criteria Optimization Tags , , , , , ,

    In radiation therapy treatment-plan optimization, selecting a set of clinical objectives that are tractable and parsimonious yet effective is a challenging task. In clinical practice, this is typically done by trial and error based on the treatment planner’s subjective assessment, which often makes the planning process inefficient and inconsistent. We develop the objective selection problem … Read more

    Deep Unfolding of a Proximal Interior Point Method for Image Restoration

  • Carla Bertocchi
  • Émilie Chouzenoux
  • Marie-Caroline Corbineau
  • Jean-Christophe Pesquet
  • Marco Prato
    • Categories Convex and Nonsmooth Optimization, Network Optimization, Robust Optimization Tags , , , , ,

    Variational methods are widely applied to ill-posed inverse problems for they have the ability to embed prior knowledge about the solution. However, the level of performance of these methods significantly depends on a set of parameters, which can be estimated through computationally expensive and time-consuming methods. In contrast, deep learning offers very generic and efficient … Read more