Data Science Applications

Symmetry-Compatible Matrix-Gradient Methods: Equivariant Updates, Spectral Operators, and Convergence

Categories Data Science Algorithms, Data Science Applications, Optimization in Data Science Tags , , , ,

We develop a symmetry-compatible framework for first-order methods on matrix optimization problems. The central principle is that the update rule for a matrix variable should be equivariant with respect to the natural symmetry group acting on that variable. For matrix representations of linear operators, this leads to bi-orthogonal equivariance under left and right orthogonal changes … Read more

Fast and Simple Multiclass Data Segmentation: An Eigendecomposition and Projection-Free Approach

Categories Data Science Algorithms, Data Science Applications, Nonlinear Optimization

Graph-based machine learning has seen an increased interest over the last decade with many connections to other fields of applied mathematics. Learning based on partial differential equations, such as the phase-field Allen-Cahn equation, allows efficient handling of semi-supervised learning approaches on graphs. The numerical solution of the graph Allen-Cahn equation via a convexity splitting or … Read more

A Taxonomy of Multi-Objective Alignment Techniques for Large Language Models

Categories Applications - Science and Engineering, Data Science Applications

Aligning large language models (LLMs) with human preferences has evolved from single-objective reward maximization to sophisticated multi-objective optimization. Real-world deployment requires balancing competing objectiveshelpfulness, harmlessness, honesty, instruction-following, and task-specic capabilitiesthat often conict. This survey provides a systematic taxonomy of multi-objective alignment techniques, organizing the rapidly growing literature into four categories: (1) Reward Decomposition approaches that … Read more

Pareto-optimal trees and Pareto forest: a bi-objective optimization model for binary classification

Categories Data Science Applications, Multi-Criteria Optimization, Optimization in Data Science Tags , ,

As inherently transparent models, classification trees play a central role in interpretable machine learning by providing easily traceable decision paths that allow users to understand how input features contribute to specific predictions. In this work, we introduce a new class of interpretable binary classification models, named Pareto-optimal trees, which aim at combining the complementary strengths … Read more

Rank-one convexification for convex quadratic optimization with step function penalties

Categories (Mixed) Integer Nonlinear Programming, Data Science Applications, Semi-definite Programming Tags ,

We investigate convexification in convex quadratic optimization with step function penalties. Such problems can be cast as mixed-integer quadratic optimization problems, where binary variables are used to encode the non-convex step function. First, we derive the convex hull for the epigraph of a quadratic function defined by a rank-one matrix. Using this rank-one convexification, we … Read more

Decision-focused predictions via pessimistic bilevel optimization: complexity and algorithms

Categories Constrained Nonlinear Optimization, Data Science Applications, Other Topics Tags ,

Dealing with uncertainty in optimization parameters is an important and longstanding challenge. Typically, uncertain parameters are predicted accurately, and then a deterministic optimization problem is solved. However, the decisions produced by this so-called {\it predict-then-optimize} procedure can be highly sensitive to uncertain parameters. In this work, we contribute to recent efforts in producing {\it decision-focused} … Read more

Optimism in the Face of Ambiguity Principle for Multi-Armed Bandits

Categories Convex Optimization, Data Science Algorithms, Data Science Applications Tags , ,

Follow-The-Regularized-Leader (FTRL) algorithms often enjoy optimal regret for adversarial as well as stochastic bandit problems and allow for a streamlined analysis. However, FTRL algorithms require the solution of an optimization problem in every iteration and are thus computationally challenging. In contrast, Follow-The-Perturbed-Leader (FTPL) algorithms achieve computational efficiency by perturbing the estimates of the rewards of … Read more

Forecasting Urban Traffic States with Sparse Data Using Hankel Temporal Matrix Factorization

Categories Data Science Algorithms, Data Science Applications, Transportation Tags , , , ,

Forecasting urban traffic states is crucial to transportation network monitoring and management, playing an important role in the decision-making process. Despite the substantial progress that has been made in developing accurate, efficient, and reliable algorithms for traffic forecasting, most existing approaches fail to handle sparsity, high-dimensionality, and nonstationarity in traffic time series and seldom consider … Read more

Data-Driven Reliable Facility Location Design

Categories Data Science Applications, Network Optimization, Supply Chain Management Tags , , ,

We study the reliable (uncapacitated) facility location (RFL) problem in a data-driven environment where historical observations of random demands and disruptions are available. Owing to the combinatorial optimization nature of the RFL problem and the mixed-binary randomness of parameters therein, the state-of-the-art RFL models applied to the data-driven setting either suggest overly conservative solutions, or … Read more