Michael I. Jordan

Understanding the Acceleration Phenomenon via High-Resolution Differential Equations

  • Simon Du
  • Michael I. Jordan
  • Bin Shi
  • Weijie Su
    • Categories Convex Optimization, Global Optimization Theory, Unconstrained Optimization Tags ,

    Gradient-based optimization algorithms can be studied from the perspective of limiting or- dinary differential equations (ODEs). Motivated by the fact that existing ODEs do not distin- guish between two fundamentally different algorithms—Nesterov’s accelerated gradient method for strongly convex functions (NAG-SC) and Polyak’s heavy-ball method—we study an alter- native limiting process that yields high-resolution ODEs. We … Read more

    Gradient Descent only Converges to Minimizers

  • Michael I. Jordan
  • Jason D. Lee
  • Benjamin Recht
  • Max Simchowitz
    • Categories Unconstrained Optimization Tags , , ,

    We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory. Article Download View Gradient Descent only Converges to Minimizers

    A direct formulation for sparse PCA using semidefinite programming

  • Alexandre d'Aspremont
  • Laurent El Ghaoui
  • Michael I. Jordan
  • G. R. G. Lanckriet
    • Categories Finance and Economics, Semi-definite Programming, Statistics Tags , ,

    We examine the problem of approximating, in the Frobenius-norm sense, a positive, semidefinite symmetric matrix by a rank-one matrix, with an upper bound on the cardinality of its eigenvector. The problem arises in the decomposition of a covariance matrix into sparse factors, and has wide applications ranging from biology to finance. We use a modification … Read more