Statistics

Regularization vs. Relaxation: A convexification perspective of statistical variable selection

  • Kun Chen
  • Hongbo Dong
  • Jeff Linderoth
    • Categories (Mixed) Integer Nonlinear Programming, Linear, Cone and Semidefinite Programming, Statistics Tags , , ,

    Variable selection is a fundamental task in statistical data analysis. Sparsity-inducing regularization methods are a popular class of methods that simultaneously perform variable selection and model estimation. The central problem is a quadratic optimization problem with an $\ell_0$-norm penalty. Exactly enforcing the $\ell_0$-norm penalty is computationally intractable for larger scale problems, so different sparsity-inducing penalty … Read more

    A New Perspective on Boosting in Linear Regression via Subgradient Optimization and Relatives

  • Robert M. Freund
  • Rahul Mazumder
  • Paul Grigas
    • Categories Data-Mining, Nonsmooth Optimization, Statistics Tags , , , ,

    In this paper we analyze boosting algorithms in linear regression from a new perspective: that of modern first-order methods in convex optimization. We show that classic boosting algorithms in linear regression, namely the incremental forward stagewise algorithm (FS-epsilon) and least squares boosting (LS-Boost-epsilon), can be viewed as subgradient descent to minimize the loss function defined … Read more

    Robust Testing for Causal Inference in Observational Studies

  • Md. Noor-E-Alam
  • Cynthia Rudin
    • Categories (Mixed) Integer Linear Programming, Applications - OR and Management Sciences, Statistics Tags , , , , ,

    A vast number of causal inference studies use matching techniques, where treatment cases are matched with similar control cases. For observational data in particular, we claim there is a major source of uncertainty that is essentially ignored in these tests, which is the way the assignments of matched pairs are constructed. It is entirely possible, … Read more

    Successive Rank-One Approximations of Nearly Orthogonally Decomposable Symmetric Tensors

  • Donald Goldfarb
  • Daniel Hsu
  • Cun Mu
    • Categories Data-Mining, Global Optimization Applications, Statistics Tags , , ,

    Many idealized problems in signal processing, machine learning and statistics can be reduced to the problem of finding the symmetric canonical decomposition of an underlying symmetric and orthogonally decomposable (SOD) tensor. Drawing inspiration from the matrix case, the successive rank-one approximations (SROA) scheme has been proposed and shown to yield this tensor decomposition exactly, and … Read more

    Exact solutions to Super Resolution on semi-algebraic domains in higher dimensions

  • Yohann De Castro
  • Didier Henrion
  • Jean-Bernard Lasserre
  • Fabrice Gamboa
    • Categories Semi-definite Programming, Statistics

    We investigate the multi-dimensional Super Resolution problem on closed semi-algebraic domains for various sampling schemes such as Fourier or moments. We present a new semidefinite programming (SDP) formulation of the l1-minimization in the space of Radon measures in the multi-dimensional frame on semi-algebraic sets. While standard approaches have focused on SDP relaxations of the dual … Read more

    Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss

  • Lin Xiao
  • Yuchen Zhang
    • Categories Convex Optimization, Statistics Tags , , , ,

    We consider distributed convex optimization problems originated from sample average approximation of stochastic optimization, or empirical risk minimization in machine learning. We assume that each machine in the distributed computing system has access to a local empirical loss function, constructed with i.i.d. data sampled from a common distribution. We propose a communication-efficient distributed algorithm to … Read more

    A Versatile Heuristic Approach for Generalized Hub Location Problems

  • Uwe Clausen
  • J. Fabian Meier
    • Categories Meta Heuristics, Statistics, Transportation

    The usability of hub location models heavily depends on an appropriate modelling approach for the economies of scale. Realistic hub location models require more sophisticated transport cost structures than the traditional flow-independent discount. We develop a general modelling scheme for such problems allowing the definition of complicated (non-linear) costs and constraints; its structure allows an … Read more

    Stochastic Compositional Gradient Descent: Algorithms for Minimizing Compositions of Expected-Value Functions

  • Ethan Fang
  • Mengdi Wang
  • Han Liu
    • Categories Convex Optimization, Statistics, Stochastic Programming Tags , , , ,

    Classical stochastic gradient methods are well suited for minimizing expected-value objective functions. However, they do not apply to the minimization of a nonlinear function involving expected values or a composition of two expected-value functions, i.e., problems of the form $\min_x \E_v\[f_v\big(\E_w [g_w(x)]\big) \]$. In order to solve this stochastic composition problem, we propose a class … Read more

    Fast Algorithms for the Minimum Volume Estimator

  • Selin Damla Ahipasaoglu
    • Categories (Mixed) Integer Nonlinear Programming, Global Optimization, Statistics Tags , ,

    The MVE estimator is an important tool in robust regression and outlier detection in statistics. We develop fast and efficient algorithms for the MVE estimator problem and discuss how they can be implemented efficiently. The novelty of our approach stems from the recent developments in the first-order algorithms for solving the related Minimum Volume Enclosing … Read more

    A Branch-and-Bound Algorithm for Instrumental Variable Quantile Regression

  • Samuel Burer
  • Guanglin Xu
    • Categories Nonlinear Optimization, Quadratic Programming, Statistics

    This paper studies a statistical problem called instrumental variable quantile regres- sion (IVQR). We model IVQR as a convex quadratic program with complementarity constraints and—although this type of program is generally NP-hard—we develop a branch-and-bound algorithm to solve it globally. We also derive bounds on key vari- ables in the problem, which are valid asymptotically … Read more