Kaiyi Ji

Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis

  • Yifan Yang
  • Hao Ban
  • Minhui Huang
  • Shiqian Ma
  • Kaiyi Ji
    • Categories Convex and Nonsmooth Optimization, Convex Optimization, Other Topics Tags , , , ,

    \(\) Bilevel optimization has recently attracted considerable attention due to its abundant applications in machine learning problems. However, existing methods rely on prior knowledge of problem parameters to determine stepsizes, resulting in significant effort in tuning stepsizes when these parameters are unknown. In this paper, we propose two novel tuning-free algorithms, D-TFBO and S-TFBO. D-TFBO … Read more

    Efficiently Escaping Saddle Points in Bilevel Optimization

  • Minhui Huang
  • Kaiyi Ji
  • Lifeng Lai
  • Shiqian Ma
    • Categories Nonlinear Optimization Tags , , ,

    Bilevel optimization is one of the fundamental problems in machine learning and optimization. Recent theoretical developments in bilevel optimization focus on finding the first-order stationary points for nonconvex-strongly-convex cases. In this paper, we analyze algorithms that can escape saddle points in nonconvex-strongly-convex bilevel optimization. Specifically, we show that the perturbed approximate implicit differentiation (AID) with … Read more