Variance Reduction of Stochastic Gradients Without Full Gradient Evaluation

  • Florian Jarre
  • Felix Lieder
    • Categories Unconstrained Optimization Tags , Short URL: https://optimization-online.org/?p=16303

    A standard concept for reducing the variance of stochastic gradient approximations is based on full gradient evaluations every now and then. In this paper an approach is considered that -- while approximating a local minimizer of a sum of functions -- also generates approximations of the gradient and the function values without relying on full gradient evaluations.

    Citation

    Preprint

    Article

    Download

    View Variance Reduction of Stochastic Gradients Without Full Gradient Evaluation