machine learning

Newtonian Methods with Wolfe Linesearch in Nonsmooth Optimization and Machine Learning

  • Miantao Chao
  • Boris S. Mordukhovich
  • Zijian Shi
  • Jin Zhang
    • Categories Nonlinear Optimization Tags , , , , , ,

    This paper introduces and develops coderivative-based Newton methods with Wolfe linesearch conditions to solve various classes of problems in nonsmooth optimization and machine learning. We first propose a generalized regularized Newton method with Wolfe linesearch (GRNM-W) for unconstrained $C^{1,1}$ minimization problems (which are second-order nonsmooth) and establish global as well as local superlinear convergence of … Read more

    Variable metric proximal stochastic gradient methods with additional sampling

  • Ilaria Trombini
  • Federica Porta
  • Nataša Krklec Jerinkić
  • Valeria Ruggiero
    • Categories Stochastic Approaches Tags , , ,

    Regularized empirical risk minimization problems arise in a variety of applications, including machine learning, signal processing, and image processing. Proximal stochastic gradient algorithms are a standard approach to solve these problems due to their low computational cost per iteration and a relatively simple implementation. This paper introduces a class of proximal stochastic gradient methods built … Read more

    Multiple Kernel Learning-Aided Column-and-Constraint Generation Method

  • Biao Han
    • Categories Applications - OR and Management Sciences, Optimization in Data Science, Robust Optimization Tags , , , ,

    Two-stage robust optimization (two-stage RO), due to its ability to balance robustness and flexibility, has been widely used in various fields for decision-making under uncertainty. This paper proposes a multiple kernel learning (MKL)-aided column-and-constraint generation (CCG) method to address this issue in the context of data-driven decision optimization, and releases a corresponding registered Julia package, … Read more

    Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis

  • Yifan Yang
  • Hao Ban
  • Minhui Huang
  • Shiqian Ma
  • Kaiyi Ji
    • Categories Convex and Nonsmooth Optimization, Convex Optimization, Other Topics Tags , , , ,

    Bilevel optimization has recently attracted considerable attention due to its abundant applications in machine learning problems. However, existing methods rely on prior knowledge of problem parameters to determine stepsizes, resulting in significant effort in tuning stepsizes when these parameters are unknown. In this paper, we propose two novel tuning-free algorithms, D-TFBO and S-TFBO. D-TFBO employs … Read more

    Forecasting Urban Traffic States with Sparse Data Using Hankel Temporal Matrix Factorization

  • Chun Cheng
    • Categories Data Science Algorithms, Data Science Applications, Transportation Tags , , , ,

    Forecasting urban traffic states is crucial to transportation network monitoring and management, playing an important role in the decision-making process. Despite the substantial progress that has been made in developing accurate, efficient, and reliable algorithms for traffic forecasting, most existing approaches fail to handle sparsity, high-dimensionality, and nonstationarity in traffic time series and seldom consider … Read more

    An Extended Validity Domain for Constraint Learning

  • Yilin Zhu
  • Samuel Burer
    • Categories Data Science Algorithms, Optimization in Data Science Tags , , , ,

    We consider embedding a predictive machine-learning model within a prescriptive optimization problem. In this setting, called constraint learning, we study the concept of a validity domain, i.e., a constraint added to the feasible set, which keeps the optimization close to the training data, thus helping to ensure that the computed optimal solution exhibits less prediction … Read more

    A graph-structured distance for mixed-variable domains with meta variables

  • Edward Hallé-Hannan
  • Charles Audet
  • Youssef Diouane
  • Sébastien Le Digabel
  • Paul Saves
    • Categories Data Science Theory, Optimization in Data Science Tags , , ,

    Heterogeneous datasets emerge in various machine learning and optimization applications that feature different input sources, types or formats. Most models or methods do not natively tackle heterogeneity. Hence, such datasets are often partitioned into smaller and simpler ones, which may limit the generalizability or performance, especially if data is limited. The first main contribution of … Read more

    Stochastic Aspects of Dynamical Low-Rank Approximation in the Context of Machine Learning

  • Arsen Hnatiuk
  • Lisa Kusch
  • Jonas Kusch
  • Nicolas R. Gauger
  • Andrea Walther
    • Categories Data Science Theory, Nonlinear Optimization, Optimization in Data Science Tags , , , ,

    The central challenges of today’s neural network architectures are the prohibitive memory footprint and the training costs associated with determining optimal weights and biases. A large portion of research in machine learning is therefore dedicated to constructing memory-efficient training methods. One promising approach is dynamical low-rank training (DLRT) which represents and trains parameters as a … Read more

    Neur2BiLO: Neural Bilevel Optimization

  • Justin Dumouchelle
  • Esther Julien
  • Jannis Kurtz
  • Elias B. Khalil
    • Categories (Mixed) Integer Linear Programming, (Mixed) Integer Nonlinear Programming Tags , ,

    Bilevel optimization deals with nested problems in which a leader takes the first decision to minimize their objective function while accounting for a follower best-response reaction. Constrained bilevel problems with integer variables are particularly notorious for their hardness.  While exact solvers have been proposed for mixed-integer~linear bilevel optimization, they tend to scale poorly with problem … Read more

    Optimal counterfactual explanations for k-Nearest Neighbors using Mathematical Optimization and Constraint Programming

  • Thibaut Vidal
  • Ricardo Fukasawa
  • Louis-Martin Rousseau
  • Claudio Contardo
    • Categories Applications - Science and Engineering, Data Science Algorithms Tags , , , ,

    Within the topic of explainable AI, counterfactual explanations to classifiers have received significant recent attention. We study counterfactual explanations that try to explain why a data point received an undesirable classification by providing the closest data point that would have received a desirable one. Within the context of one the simplest and most popular classification … Read more