Tim Tsz-Kit Lau – Optimization Online

Symmetry-Compatible Matrix-Gradient Methods: Equivariant Updates, Spectral Operators, and Convergence

Published: 2026/07/27

Data Science Algorithms, Data Science Applications, Optimization in Data Science Equivariant matrix-gradient methods, first-order methods, matrix optimization, Spectral operators, Symmetry-compatible optimization methods

We develop a symmetry-compatible framework for first-order methods on matrix optimization problems. The central principle is that the update rule for a matrix variable should be equivariant with respect to the natural symmetry group acting on that variable. For matrix representations of linear operators, this leads to bi-orthogonal equivariance under left and right orthogonal changes … Read more

Global Convergence in Deep Learning with Variable Splitting via the Kurdyka-{\L}ojasiewicz Property

Published: 2018/10/22, Updated: 2019/07/05

Convex and Nonsmooth Optimization, Data-Mining block-coordinate descent, deep learning, global convergence, kurdyka-lojasiewicz inequality

Deep learning has recently attracted a significant amount of attention due to its great empirical success. However, the effectiveness in training deep neural networks (DNNs) remains a mystery in the associated nonconvex optimizations. In this paper, we aim to provide some theoretical understanding on such optimization problems. In particular, the Kurdyka-{\L}ojasiewicz (KL) property is established … Read more