first-order oracle complexity

A first-order method for nonconvex-nonconcave minimax problems under a local Kurdyka-Łojasiewicz condition

  • Zhaosong Lu
  • Xiangyuan Wang
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization, Other Topics Tags , , , ,

    We study a class of nonconvex–nonconcave minimax problems in which the inner maximization problem satisfies a local Kurdyka–Łojasiewicz (KL) condition that may vary with the outer minimization variable. In contrast to the global KL or Polyak–Łojasiewicz (PL) conditions commonly assumed in the literature—which are significantly stronger and often too restrictive in practice—this local KL condition … Read more

    Complexity of normalized stochastic first-order methods with momentum under heavy-tailed noise

  • Chuan He
  • Zhaosong Lu
  • Defeng Sun
  • Zhanwang Deng
    • Categories Nonlinear Optimization, Optimization in Data Science, Unconstrained Optimization Tags , , ,

    In this paper, we propose practical normalized stochastic first-order methods with Polyak momentum, multi-extrapolated momentum, and recursive momentum for solving unconstrained optimization problems. These methods employ dynamically updated algorithmic parameters and do not require explicit knowledge of problem-dependent quantities such as the Lipschitz constant or noise bound. We establish first-order oracle complexity results for finding … Read more