Nonlinear Optimization

Mordukhovich Stationarity for Mathematical Programs with Switching Constraints under Weak Constraint Qualifications

  • Lei Guo
  • Gao-Xi Li
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization

    The mathematical program with switching constraints (MPSC), which is recently introduced, is a difficult class of optimization problems since standard constraint qualifications are very likely to fail at local minimizers. MPSC arises from the discretization of optimal control problems with switching constraints which appears frequently in the field of control. Due to the failure of … Read more

    HyperNOMAD: Hyperparameter optimization of deep neural networks using mesh adaptive direct search

  • Dounia Lakhmiri
  • Sébastien Le Digabel
  • Christophe Tribes
    • Categories Nonlinear Optimization, Optimization Software and Modeling Systems Tags , , , , , ,

    The performance of deep neural networks is highly sensitive to the choice of the hyperparameters that define the structure of the network and the learning process. When facing a new application, tuning a deep neural network is a tedious and time consuming process that is often described as a “dark art”. This explains the necessity … Read more

    Complexity and performance of an Augmented Lagrangian algorithm

  • Ernesto G. Birgin
  • José Mario Martínez
    • Categories Constrained Nonlinear Optimization Tags , , ,

    Algencan is a well established safeguarded Augmented Lagrangian algorithm introduced in [R. Andreani, E. G. Birgin, J. M. Martínez and M. L. Schuverdt, On Augmented Lagrangian methods with general lower-level constraints, SIAM Journal on Optimization 18, pp. 1286-1309, 2008]. Complexity results that report its worst-case behavior in terms of iterations and evaluations of functions and … Read more

    Accelerated Symmetric ADMM and Its Applications in Signal Processing

  • Jianchao Bai
  • Ke Guo
  • Yang Jing
  • Junli Liang
    • Categories Constrained Nonlinear Optimization, Other Topics Tags , , , ,

    The alternating direction method of multipliers (ADMM) were extensively investigated in the past decades for solving separable convex optimization problems. Fewer researchers focused on exploring its convergence properties for the nonconvex case although it performed surprisingly efficient. In this paper, we propose a symmetric ADMM based on different acceleration techniques for a family of potentially … Read more

    Optimal K-Thresholding Algorithms for Sparse Optimization Problems

  • Yun-Bin Zhao
    • Categories Constrained Nonlinear Optimization, Convex Optimization, Data-Mining Tags , , , , ,

    The simulations indicate that the existing hard thresholding technique independent of the residual function may cause a dramatic increase or numerical oscillation of the residual. This inherit drawback of the hard thresholding renders the traditional thresholding algorithms unstable and thus generally inefficient for solving practical sparse optimization problems. How to overcome this weakness and develop … Read more

    Adaptive cubic regularization methods with dynamic inexact Hessian information and applications to finite-sum minimization

  • Stefania Bellavia
  • Benedetta Morini
  • Gianmarco Gurioli
    • Categories Unconstrained Optimization Tags

    Abstract. We consider the Adaptive Regularization with Cubics approach for solving nonconvex optimization problems and propose a new variant based on inexact Hessian information chosen dynamically. The theoretical analysis of the proposed procedure is given. The key property of ARC framework, constituted by optimal worst-case function/derivative evaluation bounds for first- and second-order critical point, is … Read more

    Alternative DC Algorithm for Partial DC programming

  • Le Thi Hoai An
  • Huynh Van Ngai
  • Tao Pham Dinh
    • Categories Nonlinear Optimization Tags , , , ,

    In this paper, we introduce an alternative DC algorithm for solving partial DC programs. This proposed algorithm is an natural extension of the standard DC algorithm. Furthermore, we also consider an inexact version of this alternative DC algorithm. The convergence of these proposed algorithms (both the exact and inexact versions) are investigated. The applications to … Read more

    Spectral properties of Barzilai-Borwein rules in solving singly linearly constrained optimization problems subject to lower and upper bounds

  • Serena Crisci
  • Federica Porta
  • Luca Zanni
  • Valeria Ruggiero
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags , , ,

    In 1988, Barzilai and Borwein published a pioneering paper which opened the way to inexpensively accelerate first-order methods. More in detail, in the framework of unconstrained optimization, Barzilai and Borwein developed two strategies to select the steplength in gradient descent methods with the aim of encoding some second-order information of the problem without computing and/or … Read more

    On the Complexity of an Augmented Lagrangian Method for Nonconvex Optimization

  • Geovani Nunes Grapiglia
  • Ya-xiang Yuan
    • Categories Nonlinear Optimization Tags , , ,

    In this paper we study the worst-case complexity of an inexact Augmented Lagrangian method for nonconvex constrained problems. Assuming that the penalty parameters are bounded, we prove a complexity bound of $\mathcal{O}(|\log(\epsilon)|)$ outer iterations for the referred algorithm to generate an $\epsilon$-approximate KKT point, for $\epsilon\in (0,1)$. When the penalty parameters are unbounded, we prove … Read more

    An accelerated inexact proximal point method for solving nonconvex-concave min-max problems

  • Renato D.C. Monteiro
  • Weiwei Kong
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags , , , , ,

    Abstract This paper presents a quadratic-penalty type method for solving linearly-constrained composite nonconvex-concave min-max problems. The method consists of solving a sequence of penalty subproblems which, due to the min-max structure of the problem, are potentially nonsmooth but can be approximated by smooth composite nonconvex minimization problems. Each of these penalty subproblems is then solved … Read more