A robust BFGS algorithm for unconstrained nonlinear optimization problems

The traditional BFGS algorithm has been proved very efficient. It is convergent for convex non linear optimization problems. However, for non-convex nonlinear optimization problems, it is known that the BFGS algorithm may not be convergent. This paper proposes a robust BFGS algorithm in the sense that the algorithm superlinearly converges to a local minimum under … Read more

Aries: A Three-Segment Path for Solving Trust-Region Subproblems

Trust-region methods are fundamental to nonlinear optimization, but solving the trust-region subproblem exactly can be computationally expensive.  This paper proposes Aries, a three-segment path  approximation that extends the classical dogleg path while retaining comparable per-iteration work.  We state the assumptions under which the usual trust-region sufficient-decrease argument applies and  describe an adaptive rule for selecting … Read more

On the BFGS Method for Unconstrained Optimization

Quasi-Newton methods form the basis of many effective methods for unconstrained and constrained optimization. As the iterations proceed, a quasi-Newton method incorporates new curvature information by performing a low-rank update to a matrix that serves as an approximation to a Hessian matrix of second derivatives. In the years following the publication of the Davidon-Fletcher-Powell (DFP) … Read more

Nonconvex stochastic zeroth-order optimization with decision-dependent distributions: from momentum tracking to coupled sampling

In this paper, we study nonconvex stochastic optimization with {decision-dependent distributions}, where the decision variable influences the underlying sampling distribution and only stochastic function-value feedback is available. We address two challenges {induced by decision-dependent distributions}: transport error in momentum-based gradient tracking and variance inflation in zeroth-order estimation. We first develop a Polyak-momentum zeroth-order method that … Read more

Tight Nonasymptotic Local Convergence of Sinkhorn-Knopp

We revisit the Sinkhorn-Knopp (SK) algorithm for the matrix scaling problem. Despite extensive literature on the global convergence of SK and its variants, its local linear convergence behavior remains less understood. We address this gap by providing the first nonasymptotic local analysis of SK that matches the rate obtained from existing asymptotic Jacobian-based arguments. We … Read more

A Minimal-Gradient Subspace Method for Unconstrained Optimization

We propose a minimal-gradient subspace method for unconstrained optimization. For strictly convex quadratics, conjugate gradient can be interpreted as exact minimization over a two-dimensional affine subspace. We use the same reduced subspace in the nonlinear case, but compute a trial step by minimizing a local model of the next gradient norm. For SPD quadratics, every … Read more

A Momentum Trust-Region Algorithm for Unconstrained Optimization

We introduce a Momentum Trust-Region Algorithm for unconstrained optimization that incorporates Nesterov-type acceleration into the classical trust-region framework. The method builds trust-region models around a momentum-shifted point and uses an Armijo-type backtracking procedure to safeguard progress along the resulting displacement. This design preserves the robustness of trust-region methods while exploiting momentum to improve practical efficiency. … Read more

A Quantum Optimization Framework for Data-Assimilation-Augmented Parameter Estimation

Parameter estimation is a fundamental challenge in the calibration of ordinary differential equation (ODE) models, where repeated numerical integration can lead to high computational cost. In this work, we investigate whether quantum algorithms can be leveraged to assist parameter estimation in nonlinear dynamical systems. We develop a hybrid classical–quantum framework that reformulates a data-assimilation-augmented parameter … Read more

bAdag: an adaptive block coordinate gradient method for smooth nonconvex functions

A new Block Coordinate Gradient (BCG) method, dubbed bAdag, for smooth, nonconvex minimization problem is proposed; it falls in the class of Objective Function Free Optimization (OFFO) methods, and it is based on the AdaGrad algorithm. At each iteration, our method computes an adaptive step size based on the cumulative sum of block gradients, instead … Read more

Quadratic Quasi-Newton Optimization – An Interpolated Hybrid Method

We present the Quadratic-Quasi-Newton (QQN) algorithm, a novel optimization method that combines gradient descent and quasi-Newton directions through quadratic interpolation. QQN constructs a parametric path d(t) = t(1 − t)(−∇f) + t 2 d L-BFGS and performs univariate optimization along this path, creating an adaptive interpolation that requires no additional hyperparameters beyond those of its … Read more