Shigeng Sun

Stochastic Ratios Tracking Algorithm for Large Scale Machine Learning Problems

  • Shigeng Sun
  • Yuchen Xie
    • Categories Nonlinear Optimization, Stochastic Programming

    Many machine learning applications and tasks rely on the stochastic gradient descent (SGD) algorithm and its variants. Effective step length selection is crucial for the success of these algorithms, which has motivated the development of algorithms such as ADAM or AdaGrad. In this paper, we propose a novel algorithm for adaptive step length selection in … Read more

    A Trust Region Method for the Optimization of Noisy Functions

  • Shigeng Sun
  • Jorge Nocedal
    • Categories Nonlinear Optimization Tags , ,

    Classical trust region methods were designed to solve problems in which function and gradient information are exact. This paper considers the case when there are bounded errors (or noise) in the above computations and proposes a simple modification of the trust region method to cope with these errors. The new algorithm only requires information about … Read more