Lingjun Guo

Progressively Sampled Equality-Constrained Optimization

  • Frank E. Curtis
  • Lingjun Guo
  • Daniel P. Robinson
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization, Optimization in Data Science Tags , , , ,

    An algorithm is proposed, analyzed, and tested for solving continuous nonlinear-equality-constrained optimization problems where the constraints are defined by an expectation or an average over a large (finite) number of terms. The main idea of the algorithm is to solve a sequence of equality-constrained problems, each involving a finite sample of constraint-function terms, over which … Read more

    Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis Function Decomposition

  • Jianhao Ma
  • Lingjun Guo
  • Salar Fattahi
    • Categories Optimization in Data Science

    This work analyzes the solution trajectory of gradient-based algorithms via a novel basis function decomposition. We show that, although solution trajectories of gradient-based algorithms may vary depending on the learning task, they behave almost monotonically when projected onto an appropriate orthonormal function basis. Such projection gives rise to a basis function decomposition of the solution … Read more