Novel stepsize for some accelerated and stochastic optimization methods

New first-order methods now need to be improved to keep up with the constant developments in machine learning and mathematics. They are commonly used methods to solve optimization problems. Among them, the algorithm branch based on gradient descent has developed rapidly with good results achieved. Not out of that trend, in this article, we research … Read more

Safeguarded augmented Lagrangian algorithms with scaled stopping criterion for the subproblems

At each iteration of the Safeguarded Augmented Lagrangian algorithm Algencan, a bound-constrained subproblem consisting of the minimization of the Powell-Hestenes-Rockafellar augmented Lagrangian function is considered, for which a minimizer with tolerance tending to zero is sought. More precisely, a point that satisfies a subproblem first-order necessary optimality condition with tolerance tending to zero is required. … Read more

An optimally fast objective-function-free minimization algorithm using random subspaces

Article Download View An optimally fast objective-function-free minimization algorithm using random subspaces

Sequential Quadratic Optimization for Stochastic Optimization with Deterministic Nonlinear Inequality and Equality Constraints

A sequential quadratic optimization algorithm for minimizing an objective function defined by an expectation subject to nonlinear inequality and equality constraints is proposed, analyzed, and tested. The context of interest is when it is tractable to evaluate constraint function and derivative values in each iteration, but it is intractable to evaluate the objective function or … Read more

On the paper “Augmented Lagrangian algorithms for solving the continuous nonlinear resource allocation problem”

In the paper [Torrealba, E.M.R. et al. Augmented Lagrangian algorithms for solving the continuous nonlinear resource allocation problem. EJOR, 299(1) 46–59, 2021] an augmented Lagrangian algorithm was proposed for resource allocation problems with the intriguing characteristic that instead of solving the box-constrained augmented Lagrangian subproblem, they propose projecting the solution of the unconstrained subproblem onto … Read more

Gas Transport Network Optimization: Mixed-Integer Nonlinear Models

Although modern societies strive towards energy systems that are entirely based on renewable energy carriers, natural gas is still one of the most important energy sources. This became even more obvious in Europe with Russia’s 2022 war against the Ukraine and the resulting stop of gas supplies from Russia. Besides that it is very important … Read more

Shape-Changing Trust-Region Methods Using Multipoint Symmetric Secant Matrices

In this work, we consider methods for large-scale and nonconvex unconstrained optimization. We propose a new trust-region method whose subproblem is defined using a so-called “shape-changing” norm together with densely-initialized multipoint symmetric secant (MSS) matrices to approximate the Hessian. Shape-changing norms and dense initializations have been successfully used in the context of traditional quasi Newton … Read more

A Quadratically Convergent Sequential Programming Method for Second-Order Cone Programs Capable of Warm Starts

We propose a new method for linear second-order cone programs. It is based on the sequential quadratic programming framework for nonlinear programming. In contrast to interior point methods, it can capitalize on the warm-start capabilities of active-set quadratic programming subproblem solvers and achieve a local quadratic rate of convergence. In order to overcome the non-differentiability … Read more

Convergence to a second-order critical point of composite nonsmooth problems by a trust region method

An algorithm for finding a first-order and second-order critical point of composite nonsmooth problems is proposed in this paper. For smooth problems, algorithms for searching such a point usually utilize the so called negative-curvature directions. In this paper, the method recently proposed for nonlinear semidefinite problems by the current author is extended for solving general … Read more

Worst-Case Complexity of TRACE with Inexact Subproblem Solutions for Nonconvex Smooth Optimization

An algorithm for solving nonconvex smooth optimization problems is proposed, analyzed, and tested. The algorithm is an extension of the Trust Region Algorithm with Contractions and Expansions (TRACE) [Math. Prog. 162(1):132, 2017]. In particular, the extension allows the algorithm to use inexact solutions of the arising subproblems, which is an important feature for solving large-scale … Read more