A line search based proximal stochastic gradient algorithm with dynamical variance reduction

Many optimization problems arising from machine learning applications can be cast as the minimization of the sum of two functions: the first one typically represents the expected risk, and in practice it is replaced by the empirical risk, and the other one imposes a priori information on the solution. Since in general the first term … Read more

Adaptive Third-Order Methods for Composite Convex Optimization

In this paper we propose third-order methods for composite convex optimization problems in which the smooth part is a three-times continuously differentiable function with Lipschitz continuous third-order derivatives. The methods are adaptive in the sense that they do not require the knowledge of the Lipschitz constant. Trial points are computed by the inexact minimization of … Read more

Exploiting Prior Function Evaluations in Derivative-Free Optimization

A derivative-free optimization (DFO) algorithm is presented. The distinguishing feature of the algorithm is that it allows for the use of function values that have been made available through prior runs of a DFO algorithm for solving prior related optimization problems. Applications in which sequences of related optimization problems are solved such that the proposed … Read more

A Novel Model for Transfer Synchronization in Transit Networks and a Lagrangian-based Heuristic Solution Method

To realize the benefits of network connectivity in transfer-based transit networks, it is critical to minimize transfer disutility for passengers by synchronizing timetables of intersecting routes. We propose a mixed-integer linear programming timetable synchronization model that incorporates new features, such as dwell time determination and vehicle capacity limit consideration, which have been largely overlooked in … Read more

A weak tail-bound probabilistic condition for function estimation in stochastic derivative-free optimization

In this paper, we use tail bounds to define a tailored probabilistic condition for function estimation that eases the theoretical analysis of stochastic derivative-free optimization methods. In particular, we focus on the unconstrained minimization of a potentially non-smooth function, whose values can only be estimated via stochastic observations, and give a simplified convergence proof for … Read more

Semi-infinite models for equilibrium selection

In their seminal work `A General Theory of Equilibrium Selection in Games’ (The MIT Press, 1988) Harsanyi and Selten introduce the notion of payoff dominance to explain how players select some solution of a Nash equilibrium problem from a set of nonunique equilibria. We formulate this concept for generalized Nash equilibrium problems, relax payoff dominance … Read more

Retraction based Direct Search Methods for Derivative Free Riemannian Optimization

Direct search methods represent a robust and reliable class of algorithms for solving black-box optimization problems. In this paper, we explore the application of those strategies to Riemannian optimization, wherein minimization is to be performed with respect to variables restricted to lie on a manifold. More specifically, we consider classic and line search extrapolated variants … Read more

A harmonic framework for stepsize selection in gradient methods

We study the use of inverse harmonic Rayleigh quotients with target for the stepsize selection in gradient methods for nonlinear unconstrained optimization problems. This provides not only an elegant and flexible framework to parametrize and reinterpret existing stepsize schemes, but also gives inspiration for new flexible and tunable families of steplengths. In particular, we analyze … Read more

Global convergence and acceleration of fixed point iterations of union upper semicontinuous operators: proximal algorithms, alternating and averaged nonconvex projections, and linear complementarity problems

We propose a unified framework to analyze fixed point iterations of a set-valued operator that is the union of a finite number of upper semicontinuous maps, each with a nonempty closed domain and compact values. We discuss global convergence, local linear convergence under a calmness condition, and component identification, and further propose acceleration strategies that … Read more

A Branch-and-Price Algorithm for the Vehicle Routing Problem with Drones

This paper considers a new variant of the vehicle routing problem with drones (VRPD), where multiple vehicles and drones work collaboratively to serve customers. Several practical constraints such as customers’ delivery deadlines and drones’ energy capacity are considered. Different from existing studies, we treat the number of drones taken by each vehicle as a decision … Read more