trust-region methods

Aries: A Three-Segment Path for Solving Trust-Region Subproblems

Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , , ,

Trust-region methods are fundamental to nonlinear optimization, but solving the trust-region subproblem exactly can be computationally expensive.  This paper proposes Aries, a three-segment path  approximation that extends the classical dogleg path while retaining comparable per-iteration work.  We state the assumptions under which the usual trust-region sufficient-decrease argument applies and  describe an adaptive rule for selecting … Read more

A particle swarm optimization with a reduced-subspace trust-region safeguard for contact-rich controller tuning

Categories Bound-constrained Optimization, Control Applications, Optimization of Simulated Systems Tags , , , , , ,

Contact-rich robotic insertion is challenging because uncertainty in alignment, friction, and contact conditions can lead to unsuccessful assembly, excessive contact forces,  and jamming. Designing a reliable controller therefore requires balancing insertion performance, robustness, and recovery from unexpected contact. In this paper, we propose PSO+TR, a derivative-free optimization (DFO) method that combines the exploratory search of … Read more

A Momentum Trust-Region Algorithm for Unconstrained Optimization

Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , ,

We introduce a Momentum Trust-Region Algorithm for unconstrained optimization that incorporates Nesterov-type acceleration into the classical trust-region framework. The method builds trust-region models around a momentum-shifted point and uses an Armijo-type backtracking procedure to safeguard progress along the resulting displacement. This design preserves the robustness of trust-region methods while exploiting momentum to improve practical efficiency. … Read more

A derivative-free trust-region approach for Low Order-Value Optimization problems

Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags , , ,

The Low Order-Value Optimization (LOVO) problem involves minimizing the minimum among a finite number of function values within a feasible set. LOVO has several practical applications such as robust parameter estimation, protein alignment, portfolio optimization, among others. In this work, we are interested in the constrained nonlinear optimization LOVO problem of minimizing the minimum between … Read more

Local Acceptance without Global Agreement: Convergence of Decentralized Trust-Region Methods

Categories Parallel Algorithms Tags , ,

We study decentralized smooth nonconvex optimization with a lower-bounded average objective and independent local trust-region acceptance decisions. Successful local trials can increase the average objective, while rejection produces a zero accepted inverse block at a nonzero gradient tracker. Our main contribution is a convergence analysis that permits such genuine local rejections. The method combines gradient … Read more

A Finite-Difference Trust-Region Method for Convexly Constrained Smooth Optimization

Categories Bound-constrained Optimization, Nonlinear Optimization Tags , , , ,

We propose a derivative-free trust-region method based on finite-difference gradient approximations for smooth optimization problems with convex constraints. The proposed method does not require computing an approximate stationarity measure. For nonconvex problems, we establish a worst-case complexity bound of \(\mathcal{O}\!\left(n\left(\frac{L}{\sigma}\epsilon\right)^{-2}\right)\) function evaluations for the method to reach an \(\left(\frac{L}{\sigma}\epsilon\right)\)-approximate stationary point, where \(n\) is the … Read more

Projected proximal gradient trust-region algorithm for nonsmooth optimization

Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

We consider trust-region methods for solving optimization problems where the objective is the sum of a smooth, nonconvex function and a nonsmooth, convex regularizer. We extend the global convergence theory of such methods to include worst-case complexity bounds in the case of unbounded model Hessian growth, and introduce a new, simple nonsmooth trust-region subproblem solver … Read more

TRFD: A Derivative-Free Trust-Region Method Based on Finite Differences for Composite Nonsmooth Optimization

Categories Nonsmooth Optimization Tags , , ,

In this work we present TRFD, a derivative-free trust-region method based on finite differences for minimizing composite functions of the form \(f(x)=h(F(x))\), where \(F\) is a black-box function assumed to have a Lipschitz continuous Jacobian, and \(h\) is a known convex Lipschitz function, possibly nonsmooth. The method approximates the Jacobian of \(F\) via forward finite … Read more

Black-box Optimization Algorithms for Regularized Least-squares Problems

Categories Nonlinear Optimization, Nonsmooth Optimization, Unconstrained Optimization Tags , ,

We consider the problem of optimizing the sum of a smooth, nonconvex function for which derivatives are unavailable, and a convex, nonsmooth function with easy-to-evaluate proximal operator. Of particular focus is the case where the smooth part has a nonlinear least-squares structure. We adapt two existing approaches for derivative-free optimization of nonsmooth compositions of smooth … Read more

Local Convergence Analysis of an Inexact Trust-Region Method for Nonsmooth Optimization

Categories Infinite Dimensional Optimization, Nonlinear Optimization, Nonsmooth Optimization Tags , , , , ,

In [R. J. Baraldi and D. P. Kouri, Mathematical Programming, (2022), pp. 1–40], we introduced an inexact trust-region algorithm for minimizing the sum of a smooth nonconvex function and a nonsmooth convex function in Hilbert space—a class of problems that is ubiquitous in data science, learning, optimal control, and inverse problems. This algorithm has demonstrated … Read more