Weijie Su – Optimization Online

Symmetry-Compatible Matrix-Gradient Methods: Equivariant Updates, Spectral Operators, and Convergence

Published: 2026/07/27

Data Science Algorithms, Data Science Applications, Optimization in Data Science Equivariant matrix-gradient methods, first-order methods, matrix optimization, Spectral operators, Symmetry-compatible optimization methods

We develop a symmetry-compatible framework for first-order methods on matrix optimization problems. The central principle is that the update rule for a matrix variable should be equivariant with respect to the natural symmetry group acting on that variable. For matrix representations of linear operators, this leads to bi-orthogonal equivariance under left and right orthogonal changes … Read more

Understanding the Acceleration Phenomenon via High-Resolution Differential Equations

Published: 2018/10/21, Updated: 2018/12/02

Convex Optimization, Global Optimization Theory, Unconstrained Optimization convex optimization, first-order methods

Gradient-based optimization algorithms can be studied from the perspective of limiting or- dinary differential equations (ODEs). Motivated by the fact that existing ODEs do not distin- guish between two fundamentally different algorithms—Nesterov’s accelerated gradient method for strongly convex functions (NAG-SC) and Polyak’s heavy-ball method—we study an alter- native limiting process that yields high-resolution ODEs. We … Read more