Minibatch Random Reshuffling for Smooth Convex Optimization: Batch-Size Dependence, Finite-Population Variance, and Quadratic Growth
Random Reshuffling (RR) is the default data-ordering strategy in practice, and it is almost always run with minibatches. The recent theory of Liu shows that single-sample RR dominates SGD in smooth convex optimization for every stepsize below \(2/\hat{L}\), but the role of the batch size has not been worked out. In this note we study … Read more