Data Science Algorithms

Complexity of Adagrad and other first-order methods for nonconvex optimization problems with bounds and convex constraints

  • Serge Gratton
  • Sadok Jerad
  • Philippe L. Toint
    • Categories Bound-constrained Optimization, Data Science Algorithms, Nonlinear Optimization Tags , , , , ,

    A parametric class of trust-region algorithms for constrained nonconvex optimization is analyzed, where the objective function is never computed. By defining appropriate first-order stationarity criteria, we are able to extend the Adagrad method to the newly considered problem and retrieve the standard complexity rate of the projected gradient method that uses both the gradient and … Read more

    An Extended Validity Domain for Constraint Learning

  • Yilin Zhu
  • Samuel Burer
    • Categories Data Science Algorithms, Optimization in Data Science Tags , , , ,

    We consider embedding a predictive machine-learning model within a prescriptive optimization problem. In this setting, called constraint learning, we study the concept of a validity domain, i.e., a constraint added to the feasible set, which keeps the optimization close to the training data, thus helping to ensure that the computed optimal solution exhibits less prediction … Read more

    A mathematical introduction to SVMs with self-concordant kernel

  • Florian Jarre
    • Categories Data Science Algorithms, Data-Mining, Quadratic Programming Tags , ,

    A derivation of so-called “soft-margin Support Vector Machines with kernel” is presented which does not rely on concepts from functional analysis such as Mercer’s theorem that is frequently cited in this context, and that leads to a new analysis of the continuity properties of the kernel functions such as a new self-concordance condition for the … Read more

    Optimal counterfactual explanations for k-Nearest Neighbors using Mathematical Optimization and Constraint Programming

  • Thibaut Vidal
  • Ricardo Fukasawa
  • Louis-Martin Rousseau
  • Claudio Contardo
    • Categories Applications - Science and Engineering, Data Science Algorithms Tags , , , ,

    \(\) Within the topic of explainable AI, counterfactual explanations to classifiers have received significant recent attention. We study counterfactual explanations that try to explain why a data point received an undesirable classification by providing the closest data point that would have received a desirable one. Within the context of one the simplest and most popular … Read more

    An Integer Programming Approach To Subspace Clustering With Missing Data

  • Akhilesh Soni
  • Jeff Linderoth
  • James R. Luedtke
  • Daniel Pimentel-Alarcon
    • Categories Data Science Algorithms, Integer Programming Tags , ,

    In the Subspace Clustering with Missing Data (SCMD) problem, we are given a collection of n partially observed d-dimensional vectors. The data points are assumed to be concentrated near a union of low-dimensional subspaces. The goal of SCMD is to cluster the vectors according to their subspace membership and recover the underlying basis, which can … Read more

    Conjecturing-Based Discovery of Patterns in Data

  • J. Paul Brooks
  • David Edwards
  • C. E. Larson
  • Nico Van Cleemput
    • Categories Applications - OR and Management Sciences, Data Science Algorithms, Optimization in Data Science Tags , , , ,

    We propose the use of a conjecturing machine that suggests feature relationships in the form of bounds involving nonlinear terms for numerical features and boolean expressions for categorical features. The proposed Conjecturing framework recovers known nonlinear and boolean relationships among features from data. In both settings, true underlying relationships are revealed. We then compare the … Read more

    Learning the Follower’s Objective Function in Sequential Bilevel Games

  • Ioana Molan
  • Martin Schmidt
  • Johannes Thürauf
    • Categories Data Science Algorithms, Other Topics

    We consider bilevel optimization problems in which the leader has no or only partial knowledge about the objective function of the follower. The studied setting is a sequential one in which the bilevel game is played repeatedly. This allows the leader to learn the objective function of the follower over time. We focus on two … Read more

    Gain Confidence, Reduce Disappointment: A New Approach to Cross-Validation for Sparse Regression

  • Ryan Cory-Wright
  • Andrés Gómez
    • Categories (Mixed) Integer Nonlinear Programming, Data Science Algorithms

    Ridge regularized sparse linear regression involves selecting a subset of features that explains the relationship between a high-dimensional design matrix and an output vector in an interpretable manner. To select the sparsity and robustness of linear regressors, techniques like leave-one-out cross-validation are commonly used for hyperparameter tuning. However, cross-validation typically increases the cost of sparse … Read more

    The Online Shortest Path Problem: Learning Travel Times Using A Multi-Armed Bandit Framework

  • Tomas Lagos
  • Ramon Auad
  • Felipe Lagos
    • Categories Applications - OR and Management Sciences, Data Science Algorithms, Transportation Tags , , , , ,

    In the age of e-commerce, many logistic companies must operate in large road networks without accurate knowledge of travel times for their specific fleet of vehicles. Moreover, millions of dollars are spent on routing services that do not accurately capture the specific characteristics of the companies’ drivers and the types of vehicles they must use. … Read more