asymptotic normality – Optimization Online

An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias

Published: 2020/06/14, Updated: 2020/06/15

Statistics, Stochastic Approaches, Stochastic Programming asymptotic normality, constant step-size sgd, stochastic optimization

Structured non-convex learning problems, for which critical points have favorable statistical properties, arise frequently in statistical machine learning. Algorithmic convergence and statistical estimation rates are well-understood for such problems. However, quantifying the uncertainty associated with the underlying training algorithm is not well-studied in the non-convex setting. In order to address this short-coming, in this work, … Read more

Epi-convergence of Sample Averages of a Random Lower Semi-continuous Functional Generated by a Markov Chain and Application to Stochastic Optimization

Published: 2020/04/21

John Birge

Arnab Sur

Stochastic Programming asymptotic normality, consistency, epigraphical convergence, harris recurrent markov chain, measure-preserving ergodic transformation, random lower semi-continuous function, sample average approximation method

The purpose of this article is to establish epigraphical convergence of the sample averages of a random lower semi-continuous functional associated with a Harris recurrent Markov chain with stationary distribution $\pi$. Sample averages associated with an ergodic Markov chain with stationary probability distribution will epigraphically converge from $\pi$-almost all starting points. The property of Harris … Read more