Federica Porta – Optimization Online

On the first order optimization methods in Deep Image Prior

Published: 2022/08/26, Updated: 2022/08/31

Network Optimization, Nonsmooth Optimization, Stochastic Approaches

Deep learning methods have state-of-the-art performances in many image restoration tasks. Their effectiveness is mostly related to the size of the dataset used for the training. Deep Image Prior (DIP) is an energy function framework which eliminates the dependency on the training set, by considering the structure of a neural network as an handcrafted prior … Read more

A line search based proximal stochastic gradient algorithm with dynamical variance reduction

Published: 2022/02/27, Updated: 2022/08/25

Convex Optimization First order stochastic methods, Green Artificial Intelligence, machine learning, Stochastic proximal methods

Many optimization problems arising from machine learning applications can be cast as the minimization of the sum of two functions: the first one typically represents the expected risk, and in practice it is replaced by the empirical risk, and the other one imposes a priori information on the solution. Since in general the first term … Read more

Spectral properties of Barzilai-Borwein rules in solving singly linearly constrained optimization problems subject to lower and upper bounds

Published: 2019/06/14, Updated: 2020/01/05

Constrained Nonlinear Optimization, Nonlinear Optimization gradient projection methods, hessian spectral properties, singly linearly and bound constrained optimization, steplength rules

In 1988, Barzilai and Borwein published a pioneering paper which opened the way to inexpensively accelerate first-order methods. More in detail, in the framework of unconstrained optimization, Barzilai and Borwein developed two strategies to select the steplength in gradient descent methods with the aim of encoding some second-order information of the problem without computing and/or … Read more