gradient approximation

The cosine measure relative to a subspace

  • Gabriel Jarry-Bolduc
  • Warren Hare
  • Charles Audet
    • Categories Nonlinear Optimization Tags , , , ,

    The cosine measure was introduced in 2003 to quantify the richness of a finite positive spanning sets of directions in the context of derivative-free directional methods. A positive spanning set is a set of vectors whose nonnegative linear combinations span the whole space. The present work extends the definition of cosine measure. In particular, the … Read more

    On the convergence of stochastic bi-level gradient methods

  • Nicolas Couellan
  • Wenjuan Wang
    • Categories Data-Mining, Nonlinear Optimization Tags , , ,

    We analyze the convergence of stochastic gradient methods for bi-level optimization problems. We address two specific cases: first when the outer objective function can be expressed as a finite sum of independent terms, and next when both the outer and inner objective functions can be expressed as finite sums of independent terms. We assume Lipschitz … Read more