Xiao Wang

Non-convex stochastic compositional optimization under heavy-tailed noise

  • Chunhao Han
  • Xiao Wang
  • pengxiang.xu
  • Jin Zhang
    • Categories Nonlinear Optimization

    This paper investigates non-convex stochastic compositional optimization under heavy-tailed noise, where the stochastic noise exhibits bounded $p$th moment with $p\in(1,2]$. The main challenges arise from biased gradient estimates of the objective and the violation of the standard bounded-variance assumption. To address these issues, we propose a generic algorithm framework of Normalized Stochastic Compositional Gradient methods … Read more

    An Exact Penalty Method for Stochastic Equality-Constrained Optimization

  • Yawen Cui
  • Qiankun Shi
  • Xiao Wang
  • Xiantao Xiao
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization

    In this paper, we study a penalty method for stochastic equality-constrained optimization, where both the objective and constraints are expressed in general expectation form. We introduce a novel adaptive strategy for updating the penalty parameter, guided by iteration progress to balance reductions in the penalty function with improvements in constraint violation, while each penalty subproblem … Read more

    Continuous-time Analysis of a Stochastic ADMM Method for Nonconvex Composite Optimization

  • Jiahong Guo
  • Xiao Wang
  • Xiantao Xiao
    • Categories Nonlinear Optimization, Unconstrained Optimization

    In this paper, we focus on nonconvex composite optimization, whose objective is the sum of a smooth but possibly nonconvex function and a composition of a weakly convex function coupled with a linear operator. By leveraging a smoothing technique based on Moreau envelope, we propose a stochastic proximal linearized ADMM algorithm (SPLA). To understand its … Read more

    An adaptive single-loop stochastic penalty method for nonconvex constrained stochastic optimization

  • Shiji Zuo
  • Xiao Wang
  • Hao Wang
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization

    Adaptive update schemes for penalty parameters are crucial to enhancing robustness and practical applicability of penalty methods for constrained optimization. However, in the context of general constrained stochastic optimization, additional challenges arise due to the randomness introduced by adaptive penalty parameters. To address these challenges, we propose an Adaptive Single-loop Stochastic Penalty method (AdaSSP) in … Read more