subspace optimization

Limited Memory Block Krylov Subspace Optimization for Computing Dominant Singular Value Decompositions

  • Xin Liu
  • Zaiwen Wen
  • Yin Zhang
    • Categories Constrained Nonlinear Optimization, Data-Mining Tags , , ,

    In many data-intensive applications, the use of principal component analysis (PCA) and other related techniques is ubiquitous for dimension reduction, data mining or other transformational purposes. Such transformations often require efficiently, reliably and accurately computing dominant singular value decompositions (SVDs) of large unstructured matrices. In this paper, we propose and study a subspace optimization technique … Read more

    A Fast Algorithm for Sparse Reconstruction based on Shrinkage, Subspace Optimization and Continuation

  • Donald Goldfarb
  • Zaiwen Wen
  • Wotao Yin
  • Yin Zhang
    • Categories Convex Optimization, Nonsmooth Optimization Tags , , , , , ,

    We propose a fast algorithm for solving the l1-regularized least squares problem for recovering sparse solutions to an undetermined system of linear equations Ax = b. The algorithm is divided into two stages that are performed repeatedly. In the first stage a first-order iterative method called “shrinkage” yields an estimate of the subset of components … Read more

    Sequential Subspace Optimization Method for Large-Scale Unconstrained Problems

  • Guy Narkiss
  • Michael Zibulevsky
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , ,

    We present the Sequential Subspace Optimization (SESOP) method for large scale smooth unconstrained problems. At each iteration we search for a minimum of the objective function over a subspace spanned by the current gradient and by directions of few previous steps. We also include into this subspace the direction from the starting point to the … Read more

    Support Vector Machine via Sequential Subspace Optimization

  • Guy Narkiss
  • Michael Zibulevsky
    • Categories Data-Mining, Unconstrained Optimization Tags , , , ,

    We present an optimization engine for large scale pattern recognition using Support Vector Machine (SVM). Our treatment is based on conversion of soft-margin SVM constrained optimization problem to an unconstrained form, and solving it using newly developed Sequential Subspace Optimization (SESOP) method. SESOP is a general tool for large-scale smooth unconstrained optimization. At each iteration … Read more