Optimization in Data Science – Page 6

Toward Decision-Oriented Prognostics: An Integrated Estimate-Optimize Framework for Predictive Maintenance

Published: 2025/06/18, Updated: 2025/06/19

Optimization in Data Science, Stochastic Programming data-driven optimization, Estimate-then-Optimize, Integrated Estimate-Optimize, machine learning, Predictive maintenance

Recent research increasingly integrates machine learning (ML) into predictive maintenance (PdM) to reduce operational and maintenance costs in data-rich operational settings. However, uncertainty due to model misspecification continues to limit widespread industrial adoption. This paper investigates a PdM framework in which sensor-driven prognostics inform decision-making under economic trade-offs within a finite decision space. We investigate … Read more

Two-way Cutting-plane Algorithm for Best Subset Selection Considering Multicollinearity

Published: 2025/06/05

Yuichi Takano

Global Optimization, Integer Programming, Optimization in Data Science linear regression, logistic regression, mixed integer optimization, multicollinearity, subset selection

When linear dependence exists between some explanatory variables in a regression model, the estimates of regression coefficients become unstable, thereby making the interpretation of the estimation results unreliable. To eliminate such multicollinearity, we propose a high-performance method for selecting the best subset of explanatory variables for linear and logistic regression models. Specifically, we first derive … Read more

Retrospective Approximation Sequential Quadratic Programming for Stochastic Optimization with General Deterministic Nonlinear Constraints

Published: 2025/05/26

Constrained Nonlinear Optimization, Nonlinear Optimization, Optimization in Data Science constrained optimization, nonlinear optimization, retrospective approximation, Sequential Qaudratic Programming, stochastic optimization

In this paper, we propose a framework based on the Retrospective Approximation (RA) paradigm to solve optimization problems with a stochastic objective function and general nonlinear deterministic constraints. This framework sequentially constructs increasingly accurate approximations of the true problems which are solved to a specified accuracy via a deterministic solver, thereby decoupling the uncertainty from … Read more

Responsible Machine Learning via Mixed-Integer Optimization

Published: 2025/05/09, Updated: 2025/08/30

Applications - OR and Management Sciences, Integer Programming, Optimization in Data Science causal inference, fair machine learning, interpretable machine learning, machine learning robust to adversarial attacks, machine learning robust to distribution shifts, mixed integer optimization, robust optimization

In the last few decades, Machine Learning (ML) has achieved significant success across domains ranging from healthcare, sustainability, and the social sciences, to criminal justice and finance. But its deployment in increasingly sophisticated, critical, and sensitive areas affecting individuals, the groups they belong to, and society as a whole raises critical concerns around fairness, transparency … Read more

Fast Stochastic Second-Order Adagrad for Nonconvex Bound-Constrained Optimization

Published: 2025/05/09, Updated: 2026/06/11

Bound-constrained Optimization, Data Science Algorithms, Nonlinear Optimization Adagrad, bound constraints, complexity, objective-function-free optimization (OFFO), second-order information, stochastic nonlinear optimization, stochastic projected gradients

ADAGB2, a generalization of the Adagrad algorithm for stochastic optimization is introduced, which is also applicable to bound-constrained problems and capable of using second-order information when available. It is shown that, given delta in (0,1) and epsilon in (0,1], the ADAGB2 algorithm needs at most O(epsilon^{-2}) iterations to ensure an epsilon-approximate first-order critical point of … Read more

Deterministic global optimization with trained neural networks: What is the benefit of the envelope of single neurons?

Published: 2025/04/28, Updated: 2026/03/30

Global Optimization, Global Optimization Applications, Optimization in Data Science artificial neural network, convex envelope, global optimization, machine learning

Optimization problems containing trained neural networks remain challenging due to their nonconvexity. Deterministic global optimization relies on relaxations that should be tight, quickly convergent, and cheap to evaluate. While envelopes of common activation functions have been established for several years, the envelope of an entire neuron had not. Recently, Carrasco and Mu\~{n}oz (arXiv.2410.23362, 2024) proposed … Read more

Rank-one convexification for convex quadratic optimization with step function penalties

Published: 2025/04/25

(Mixed) Integer Nonlinear Programming, Data Science Applications, Semi-definite Programming convexification, Support Vector Machine

We investigate convexification in convex quadratic optimization with step function penalties. Such problems can be cast as mixed-integer quadratic optimization problems, where binary variables are used to encode the non-convex step function. First, we derive the convex hull for the epigraph of a quadratic function defined by a rank-one matrix. Using this rank-one convexification, we … Read more

A Foundational Perspective for Partitional Clustering on Networks

Published: 2025/04/24, Updated: 2025/04/25

Data Science Theory, Data-Mining, Optimization in Data Science Arc Bottleneck Point, Clustering on Networks, euclidean distance, Partitional Clustering

This study presents a theoretical analysis of partitional clustering on networks. Different versions of the problem are studied considering different assignment schemes (hard and soft) and different objective functions. Cluster centers are not restricted to only the set of nodes, it is assumed that centers can also be at the edges of the network. Four … Read more

Mathematical programs with complementarity constraints and application to hyperparameter tuning

Published: 2025/04/21, Updated: 2026/06/12

Bilevel Optimization, Complementarity and Variational Inequalities, Optimization in Data Science bilevel optimization, hyperparameter optimization, mathematical program with complementarity constraints, penalisation, relaxation, support vector machines

We consider the Mathematical Program with Complementarity Constraints (MPCC). One of the main challenges in solving this problem is the systematic failure of standard Constraint Qualifications (CQs). Carefully accounting for the combinatorial nature of the complementarity constraints, tractable versions of the Mangasarian Fromovitz Constraint Qualification (MFCQ) have been designed and widely studied in the literature. … Read more

Tight Semidefinite Relaxations for Verifying Robustness of Neural Networks

Published: 2025/04/14

Optimization in Data Science, Quadratic Programming, Semi-definite Programming Decomposition into two-stage problem., neural network, Rank-1 solutions, Safety verification, Tight semidefinite relaxations

For verifying the safety of neural networks (NNs), Fazlyab et al. (2019) introduced a semidefinite programming (SDP) approach called DeepSDP. This formulation can be viewed as the dual of the SDP relaxation for a problem formulated as a quadratically constrained quadratic program (QCQP). While SDP relaxations of QCQPs generally provide approximate solutions with some gaps, … Read more