Nonlinear Optimization

A Finite-Difference Trust-Region Method for Convexly Constrained Smooth Optimization

  • Dânâ Davar
  • Geovani Nunes Grapiglia
    • Categories Bound-constrained Optimization, Nonlinear Optimization Tags , , , ,

    We propose a derivative-free trust-region method based on finite-difference gradient approximations for smooth optimization problems with convex constraints. The proposed method does not require computing an approximate stationarity measure. For nonconvex problems, we establish a worst-case complexity bound of \(\mathcal{O}\!\left(n\left(\frac{L}{\sigma}\epsilon\right)^{-2}\right)\) function evaluations for the method to reach an \(\left(\frac{L}{\sigma}\epsilon\right)\)-approximate stationary point, where \(n\) is the … Read more

    An Inexact General Descent Method with Applications in Differential Equation-Constrained Optimization

  • Humberto Gimenes Macedo
  • Luís Felipe Bueno
    • Categories Nonlinear Optimization, Systems governed by Differential Equations Optimization Tags , , ,

    In many applications, gradient evaluations are inherently approximate, motivating the development of optimization methods that remain reliable under inexact first-order information. A common strategy in this context is adaptive evaluation, whereby coarse gradients are used in early iterations and refined near a minimizer. This is particularly relevant in differential equation–constrained optimization (DECO), where discrete adjoint … Read more

    A Simple First-Order Algorithm for Full-Rank Equality Constrained Optimization

  • Serge Gratton
  • Philippe L. Toint
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization, Nonlinear Systems and Least-Squares Tags , , , ,

    A very simple first-order algorithm is proposed for solving nonlinear optimization problems with deterministic nonlinear equality constraints. This algorithm adaptively selects steps in the plane tangent to the constraints or steps that reduce infeasibility, without using a merit function or a filter. The tangent steps are based on the AdaGrad method for unconstrained minimization. The … Read more

    Optimization in Theory and Practice

  • Stephen Wright
    • Categories Convex Optimization, Linear Programming, Unconstrained Optimization Tags , ,

    Algorithms for continuous optimization problems have a rich history of design and innovation over the past several decades, in which mathematical analysis of their convergence and complexity properties plays a central role. Besides their theoretical properties, optimization algorithms are interesting also for their practical usefulness as computational tools for solving real-world problems. There are often … Read more

    Adaptive Conditional Gradient Descent

  • Abbas Khademi
  • Antonio Silveti-Falls
    • Categories Nonlinear Optimization, Optimization in Data Science Tags , , , , , , ,

    Selecting an effective step-size is a fundamental challenge in first-order optimization, especially for problems with non-Euclidean geometries. This paper presents a novel adaptive step-size strategy for optimization algorithms that rely on linear minimization oracles, as used in the Conditional Gradient or non-Euclidean Normalized Steepest Descent algorithms. Using a simple heuristic to estimate a local Lipschitz … Read more

    Towards robust optimal control of chromatographic separation processes with controlled flow reversal

  • Dominik H. Cebulla
  • Christian Kirches
  • Andreas Potschka
    • Categories Chemical Engineering, Robust Optimization, Systems governed by Differential Equations Optimization Tags , ,

    Column liquid chromatography is an important technique applied in the production of biopharmaceuticals, specifically for the separation of biological macromolecules such as proteins. When setting up process conditions, it is crucial that the purity of the product is sufficiently high, even in the presence of perturbations in the process conditions, e.g., altered buffer salt concentrations. … Read more

    On the Complexity of Lower-Order Implementations of Higher-Order Methods

  • Nikita Doikov
  • Geovani Nunes Grapiglia
    • Categories Nonlinear Optimization Tags , , , ,

    In this work, we propose a method for minimizing non-convex functions with Lipschitz continuous \(p\)th-order derivatives, starting from \(p \geq 1\). The method, however, only requires derivative information up to order \((p-1)\), since the \(p\)th-order derivatives are approximated via finite differences. To ensure oracle efficiency, instead of computing finite-difference approximations at every iteration, we reuse … Read more

    A Simple Adaptive Proximal Gradient Method for Nonconvex Optimization

  • Ye Zilong
  • Shiqian Ma
  • Junfeng Yang
  • Danqing Zhou
    • Categories Constrained Nonlinear Optimization, Nonsmooth Optimization Tags , , , ,

    Consider composite nonconvex optimization problems where the objective function consists of a smooth nonconvex term (with Lipschitz-continuous gradient) and a convex (possibly nonsmooth) term. Existing parameter-free methods for such problems often rely on complex multi-loop structures, require line searches, or depend on restrictive assumptions (e.g., bounded iterates). To address these limitations, we introduce a novel … Read more