Continuous-time Analysis of a Stochastic ADMM Method for Nonconvex Composite Optimization

In this paper, we focus on nonconvex composite optimization, whose objective is the sum of a smooth but possibly nonconvex function and a composition of a weakly convex function coupled with a linear operator. By leveraging a smoothing technique based on Moreau envelope, we propose a stochastic proximal linearized ADMM algorithm (SPLA). To understand its convergence behavior we consider a stochastic differential equation (SDE) that serves as a continuous-time stochastic model of the discrete scheme SPLA. Under mild conditions, we establish the almost-sure convergence for the smoothed objective function along the SDE’s solution trajectory, and the associated in-expectation convergence rates in the context of \L{}ojasiewicz inequality. We further establish the almost-sure global convergence and the in-expectation convergence rates of the SDE’s solution. Building upon these convergence results, we derive the in-expectation convergence rates of a trajectory derived from the SDE’s solution to some approximate critical point of the original nonsmooth problem. Finally, under certain conditions we obtain the convergence properties of the objective function values along the discrete iterates of SPLA.

Article

Download

View PDF