worst-case efficiency estimates

Understanding Limitation of Two Symmetrized Orders by Worst-case Complexity

  • Ruoyu Sun
  • Zhisheng Xiao
  • Peijun Xiao
    • Categories Complementarity and Variational Inequalities, Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , ,

    It was recently found that the standard version of multi-block cyclic ADMM diverges. Interestingly, Gaussian Back Substitution ADMM (GBS-ADMM) and symmetric Gauss-Seidel ADMM (sGS-ADMM) do not have the divergence issue. Therefore, it seems that symmetrization can improve the performance of the classical cyclic order. In another recent work, cyclic CD (Coordinate Descent) was shown to … Read more

    Efficiency of coordinate descent methods on huge-scale optimization problems

  • Yurii Nesterov
    • Categories Convex Optimization Tags , , , ,

    In this paper we propose new methods for solving huge-scale optimization problems. For problems of this size, even the simplest full-dimensional vector operations are very expensive. Hence, we propose to apply an optimization technique based on random partial update of decision variables. For these methods, we prove the global estimates for the rate of convergence. … Read more