Michael I. Jordan – Optimization Online

Understanding the Acceleration Phenomenon via High-Resolution Differential Equations

Published: 2018/10/21, Updated: 2018/12/02

Gradient-based optimization algorithms can be studied from the perspective of limiting or- dinary differential equations (ODEs). Motivated by the fact that existing ODEs do not distin- guish between two fundamentally different algorithms—Nesterov’s accelerated gradient method for strongly convex functions (NAG-SC) and Polyak’s heavy-ball method—we study an alter- native limiting process that yields high-resolution ODEs. We … Read more

Gradient Descent only Converges to Minimizers

Published: 2016/02/17

Unconstrained Optimization gradient descent, local minimizer, nonconvex optimization, saddle point problem

We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory. ArticleDownload View PDF

A direct formulation for sparse PCA using semidefinite programming

Published: 2004/07/07

Alexandre d'Aspremont

Laurent El Ghaoui

Michael I. Jordan

G. R. G. Lanckriet

Finance and Economics, Semi-definite Programming, Statistics pca, semidefinite programming, sparsity

We examine the problem of approximating, in the Frobenius-norm sense, a positive, semidefinite symmetric matrix by a rank-one matrix, with an upper bound on the cardinality of its eigenvector. The problem arises in the decomposition of a covariance matrix into sparse factors, and has wide applications ranging from biology to finance. We use a modification … Read more