Shigeng Sun

Noise-Tolerant Optimization Methods for the Solution of a Robust Design Problem

  • Yuchen Lou
  • Shigeng Sun
  • Jorge Nocedal
    • Categories Nonlinear Optimization, Optimization of Systems modeled by PDEs, Stochastic Programming

    The development of nonlinear optimization algorithms capable of performing reliably in the presence of noise has garnered considerable attention lately. This paper advocates for strategies to create noise-tolerant nonlinear optimization algorithms by adapting classical deterministic methods. These adaptations follow certain design guidelines described here, which make use of estimates of the noise level in the … Read more

    Stochastic Ratios Tracking Algorithm for Large Scale Machine Learning Problems

  • Shigeng Sun
  • Yuchen Xie
    • Categories Nonlinear Optimization, Stochastic Programming

    Many machine learning applications and tasks rely on the stochastic gradient descent (SGD) algorithm and its variants. Effective step length selection is crucial for the success of these algorithms, which has motivated the development of algorithms such as ADAM or AdaGrad. In this paper, we propose a novel algorithm for adaptive step length selection in … Read more

    A Trust Region Method for the Optimization of Noisy Functions

  • Shigeng Sun
  • Jorge Nocedal
    • Categories Nonlinear Optimization Tags , ,

    Classical trust region methods were designed to solve problems in which function and gradient information are exact. This paper considers the case when there are bounded errors (or noise) in the above computations and proposes a simple modification of the trust region method to cope with these errors. The new algorithm only requires information about … Read more