Statistics

Sharpness and well-conditioning of nonsmooth convex formulations in statistical signal recovery

  • Lijun Ding
  • Alex L. Wang
    • Categories Nonsmooth Optimization, Optimization in Data Science, Statistics Tags , , , , ,

    \(\) We study a sample complexity vs. conditioning tradeoff in modern signal recovery problems where convex optimization problems are built from sampled observations. We begin by introducing a set of condition numbers related to sharpness in \(\ell_p\) or Schatten-p norms (\(p\in[1,2]\)) based on nonsmooth reformulations of a class of convex optimization problems, including sparse recovery, … Read more

    Compressed Sensing: A Discrete Optimization Approach

  • Dimitris Bertsimas
  • Nicholas A. G. Johnson
    • Categories (Mixed) Integer Nonlinear Programming, Second-Order Cone Programming, Statistics Tags , , , ,

    We study the Compressed Sensing (CS) problem, which is the problem of finding the most sparse vector that satisfies a set of linear measurements up to some numerical tolerance. CS is a central problem in Statistics, Operations Research and Machine Learning which arises in applications such as signal processing, data compression and image reconstruction. We … Read more

    Outlier detection in regression: conic quadratic formulations

  • Andrés Gómez
  • José Neto
    • Categories (Mixed) Integer Nonlinear Programming, Statistics Tags , , ,

    In many applications, when building linear regression models, it is important to account for the presence of outliers, i.e., corrupted input data points. Such problems can be formulated as mixed-integer optimization problems involving cubic terms, each given by the product of a binary variable and a quadratic term of the continuous variables. Existing approaches in … Read more

    The min-Knapsack Problem with Compactness Constraints and Applications in Statistics

  • Alberto Santini
  • Enrico Malaguti
    • Categories Branch and Cut Algorithms, Dynamic Programming, Statistics Tags , ,

    In the min-Knapsack problem, one is given a set of items, each having a certain cost and weight. The objective is to select a subset with minimum cost, such that the sum of the weights is not smaller than a given constant. In this paper we introduce an extension of the min-Knapsack problem with additional … Read more

    Computational complexity of decomposing a symmetric matrix as a sum of positive semidefinite and diagonal matrices

  • Levent Tunçel
  • Stephen A. Vavasis
  • jingye xu
    • Categories Finance and Economics, Statistics Tags

    We study several variants of decomposing a symmetric matrix into a sum of a low-rank positive semidefinite matrix and a diagonal matrix. Such decompositions have applications in factor analysis and they have been studied for many decades. On the one hand, we prove that when the rank of the positive semidefinite matrix in the decomposition … Read more

    Cutting-plane algorithm for sparse estimation of the Cox proportional-hazards model

  • Hiroki Saishu
  • Kota Kudo
  • Yuichi Takano
    • Categories (Mixed) Integer Nonlinear Programming, Branch and Cut Algorithms, Statistics Tags , , , , ,

    Survival analysis is a family of statistical methods for analyzing event occurrence times. In this paper, we address the mixed-integer optimization approach to sparse estimation of the Cox proportional-hazards model for survival analysis. Specifically, we propose a high-performance cutting-plane algorithm based on reformulation of bilevel optimization for sparse estimation. This algorithm solves the upper-level problem … Read more

    Blessing of Nonconvexity in Deep Linear Models: Depth Flattens the Optimization Landscape Around the True Solution

  • Jianhao Ma
  • Salar Fattahi
    • Categories Nonlinear Optimization, Statistics Tags , ,

    This work characterizes the effect of depth on the optimization landscape of linear regression, showing that, despite their nonconvexity, deeper models have more desirable optimization landscape. We consider a robust and over-parameterized setting, where a subset of measurements are grossly corrupted with noise and the true linear model is captured via an $N$-layer linear neural … Read more

    Metrizing Fairness

  • Yves Rychener
  • Bahar Tașkesen
  • Daniel Kuhn
    • Categories Statistics, Stochastic Programming, Unconstrained Optimization Tags , , , , ,

    We study supervised learning problems for predicting properties of individuals who belong to one of two demographic groups, and we seek predictors that are fair according to statistical parity. This means that the distributions of the predictions within the two groups should be close with respect to the Kolmogorov distance, and fairness is achieved by … Read more

    Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold

  • Bokun Wang
  • Shiqian Ma
  • Lingzhou Xue
    • Categories Nonlinear Optimization, Statistics

    Riemannian optimization has drawn a lot of attention due to its wide applications in practice. Riemannian stochastic first-order algorithms have been studied in the literature to solve large-scale machine learning problems over Riemannian manifolds. However, most of the existing Riemannian stochastic algorithms require the objective function to be differentiable, and they do not apply to … Read more

    Global Convergence of Sub-gradient Method for Robust Matrix Recovery: Small Initialization, Noisy Measurements, and Over-parameterization

  • Jianhao Ma
  • Salar Fattahi
    • Categories Statistics, Unconstrained Optimization Tags ,

    In this work, we study the performance of sub-gradient method (SubGM) on a natural nonconvex and nonsmooth formulation of low-rank matrix recovery with $\ell_1$-loss, where the goal is to recover a low-rank matrix from a limited number of measurements, a subset of which may be grossly corrupted with noise. We study a scenario where the … Read more