Vyacheslav Kungurtsev

A Nonmonotone Matrix-Free Algorithm for Nonlinear Equality-Constrained Least-Squares Problems

  • Elhoucine Bergou
  • Youssef Diouane
  • Vyacheslav Kungurtsev
  • Clément W. Royer
    • Categories Constrained Nonlinear Optimization, Nonlinear Systems and Least-Squares

    Least squares form one of the most prominent classes of optimization problems, with numerous applications in scientific computing and data fitting. When such formulations aim at modeling complex systems, the optimization process must account for nonlinear dynamics by incorporating constraints. In addition, these systems often incorporate a large number of variables, which increases the difficulty … Read more

    Complexity iteration analysis for stongly convex multi-objective optimization using a Newton path-following procedure

  • Elhoucine Bergou
  • Youssef Diouane
  • Vyacheslav Kungurtsev
    • Categories Unconstrained Optimization

    In this note we consider the iteration complexity of solving strongly convex multi objective optimization. We discuss the precise meaning of this problem, and indicate it is loosely defined, but the most natural notion is to find a set of Pareto optimal points across a grid of scalarized problems. We derive that in most cases, … Read more

    Zero Order Stochastic Weakly Convex Composite Optimization

  • Vyacheslav Kungurtsev
  • Francesco Rinaldi
    • Categories Nonlinear Optimization, Nonsmooth Optimization, Stochastic Programming Tags , , ,

    In this paper we consider stochastic weakly convex composite problems, however without the existence of a stochastic subgradient oracle. We present a derivative free algorithm that uses a two point approximation for computing a gradient estimate of the smoothed function. We prove convergence at a similar rate as state of the art methods, however with … Read more

    Asynchronous Stochastic Subgradient Methods for General Nonsmooth Nonconvex Optimization

  • Dan Alistarh
  • Bapi Chatterjee
  • Vyacheslav Kungurtsev
  • Malcolm Egan
    • Categories Nonsmooth Optimization

    Asynchronous distributed methods are a popular way to reduce the communication and synchronization costs of large-scale optimization. Yet, for all their success, little is known about their convergence guarantees in the challenging case of general non-smooth, non-convex objectives, beyond cases where closed-form proximal operator solutions are available. This is all the more surprising since these … Read more

    Pathfollowing for Parametric Mathematical Programs with Complementarity Constraints

  • Johannes Jäschke
  • Vyacheslav Kungurtsev
    • Categories Constrained Nonlinear Optimization

    In this paper we study procedures for pathfollowing parametric mathematical pro- grams with complementarity constraints. We present two procedures, one based on the penalty approach to solving standalone MPCCs, and one based on tracing active set bifurcations aris- ing from doubly-active complementarity constraints. We demonstrate the performance of these approaches on a variety of examples … Read more

    A Comparison of Nonsmooth, Nonconvex, Constrained Optimization Solvers for the Design of Time-Delay Compensators

  • Vyacheslav Kungurtsev
  • Tim Mitchell
  • Tomas Vyhlidal
    • Categories Control Applications, Nonsmooth Optimization, Optimization Software Benchmark

    We present a detailed set of performance comparisons of two state-of-the-art solvers for the application of designing time-delay compensators, an important problem in the field of robust control. Formulating such robust control mechanics as constrained optimization problems often involves objective and constraint functions that are both nonconvex and nonsmooth, both of which present significant challenges … Read more

    A Subsampling Line-Search Method with Second-Order Results

  • Elhoucine Bergou
  • Youssef Diouane
  • Vyacheslav Kungurtsev
  • Clément W. Royer
  • V. Kunc
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , ,

    In many contemporary optimization problems such as those arising in machine learning, it can be computationally challenging or even infeasible to evaluate an entire function or its derivatives. This motivates the use of stochastic algorithms that sample problem data, which can jeopardize the guarantees obtained through classical globalization techniques in optimization such as a trust … Read more

    Second-order Guarantees of Distributed gradient Algorithms

  • Amir Daneshmand
  • Vyacheslav Kungurtsev
  • Gesualdo Scutari
    • Categories Nonlinear Optimization Tags , ,

    We consider distributed smooth nonconvex unconstrained optimization over networks, modeled as a connected graph. We examine the behavior of distributed gradient-based algorithms near strict saddle points. Specifically, we establish that (i) the renowned Distributed Gradient Descent (DGD) algorithm likely converges to a neighborhood of a Second-order Stationary (SoS) solution; and (ii) the more recent class … Read more

    A stochastic Levenberg-Marquardt method using random models with complexity results and application to data assimilation

  • Elhoucine Bergou
  • Youssef Diouane
  • Vyacheslav Kungurtsev
  • Clément W. Royer
    • Categories Nonlinear Systems and Least-Squares, Unconstrained Optimization Tags , , , , ,

    Globally convergent variants of the Gauss-Newton algorithm are often the methods of choice to tackle nonlinear least-squares problems. Among such frameworks, Levenberg-Marquardt and trust-region methods are two well-established, similar paradigms. Both schemes have been studied when the Gauss-Newton model is replaced by a random model that is only accurate with a given probability. Trust-region schemes … Read more

    A Shifted Primal-Dual Interior Method for Nonlinear Optimization

  • Philip E. Gill
  • Vyacheslav Kungurtsev
  • Daniel P. Robinson
    • Categories Constrained Nonlinear Optimization Tags , , , , , ,

    Interior methods provide an effective approach for the treatment of inequality constraints in nonlinearly constrained optimization. A new primal-dual interior method is proposed based on minimizing a sequence of shifted primal-dual penalty-barrier functions. Certain global convergence properties are established. In particular, it is shown that every limit point is either an infeasible stationary point, or … Read more