Andrea Walther

On regularized structure exploiting Quasi-Newton methods for ill-posed problems

  • Raphael Kuess
  • Andrea Walther
    • Categories Infinite Dimensional Optimization, Nonlinear Systems and Least-Squares Tags , , , ,

    Inverse problems are inherently ill-posed, leading standard optimization techniques to fail and necessitating the use of regularization. This paper introduces a regularized, structure-exploiting Powell-Symmetric-Broyden method under modified secant conditions for solving ill-posed inverse problems in both infinite dimensional and finite dimensional settings. Our approach integrates regularization and structure exploitation directly within the Quasi-Newton framework, leveraging … Read more

    How Stringent is the Linear Independence Kink Qualification in Abs-Smooth Optimization?

  • Franz Bethke
  • Andrea Walther
    • Categories Nonsmooth Optimization Tags

    Abs-smooth functions are given by compositions of smooth functions and the evaluation of the absolute value. The linear independence kink qualification (LIKQ) is a fundamental assumption in optimization problems governed by these abs-smooth functions, generalizing the well-known LICQ from smooth optimization. In particular, provided that LIKQ holds it is possible to derive optimality conditions for … Read more

    Stochastic Aspects of Dynamical Low-Rank Approximation in the Context of Machine Learning

  • Arsen Hnatiuk
  • Lisa Kusch
  • Jonas Kusch
  • Nicolas R. Gauger
  • Andrea Walther
    • Categories Data Science Theory, Nonlinear Optimization, Optimization in Data Science Tags , , , ,

    The central challenges of today’s neural network architectures are the prohibitive memory footprint and the training costs associated with determining optimal weights and biases. A large portion of research in machine learning is therefore dedicated to constructing memory-efficient training methods. One promising approach is dynamical low-rank training (DLRT) which represents and trains parameters as a … Read more

    On a Frank-Wolfe Approach for Abs-smooth Functions

  • Timo Kreimeier
  • Sebastian Pokutta
  • Andrea Walther
  • Zev Woodstock
    • Categories Constrained Nonlinear Optimization, Nonsmooth Optimization, Other Topics Tags , , , ,

    We propose an algorithm which appears to be the first bridge between the fields of conditional gradient methods and abs-smooth optimization. Our problem setting is motivated by various applications that lead to nonsmoothness, such as $\ell_1$ regularization, phase retrieval problems, or ReLU activation in machine learning. To handle the nonsmoothness in our problem, we propose … Read more

    A Semismooth Conjugate Gradients Method — Theoretical Analysis

  • Franz Bethke
  • Andreas Griewank
  • Andrea Walther
    • Categories Nonsmooth Optimization Tags , , ,

    In large scale applications, deterministic and stochastic variants of Cauchy’s steepest descent method are widely used for the minimization of objectives that are only piecewise smooth. In this paper we analyse a  deterministic descent method based on the generalization of rescaled conjugate gradients proposed by Philip Wolfe in 1975 for objectives that are convex. Without … Read more

    An active signature method for constrained abs-linear minimization

  • Timo Kreimeier
  • Andrea Walther
  • Andreas Griewank
    • Categories Nonsmooth Optimization Tags , , , , ,

    In this paper we consider the solution of optimization tasks with a piecewise linear objective function and piecewise linear constraints. First, we state optimality conditions for that class of problems using the abs-linearization approach and prove that they can be verified in polynomial time. Subsequently, we propose an algorithm called Constrained Active Signature Method that … Read more

    A Structure Exploiting Algorithm for Non-Smooth Semi-Linear Elliptic Optimal Control Problems

  • Stephan Schmidt
  • Olga Weiß
  • Andrea Walther
    • Categories Infinite Dimensional Optimization, Nonsmooth Optimization

    We investigate optimization problems with a non-smooth partial differential equation as constraint, where the non-smoothness is assumed to be caused by Nemytzkii operators generated by the functions abs, min and max. For the efficient as well as robust solution of such problems, we propose a new optimization method based on abs-linearization, i.e., a special handling … Read more

    An Algorithm for Piecewise Linear Optimization of Objective Functions in Abs-normal Form

  • Andreas Griewank
  • Andrea Walther
    • Categories Nonsmooth Optimization Tags , , , , , , ,

    In the paper [11] we derived first order (KKT) and second order (SSC) optimality conditions for functions defined by evaluation programs involving smooth elementals and absolute values. For this class of problems we showed in [12] that the natural algorithm of successive piecewise linear optimization with a proximal term (SPLOP) achieves a linear or even … Read more

    Relaxing kink qualifications and proving convergence rates in piecewise smooth optimization

  • Andreas Griewank
  • Andrea Walther
    • Categories Nonsmooth Optimization Tags , , , , , , ,

    Abstract. In the paper [9] we derived first order (KKT) and second order (SSC) optimality conditions for functions defined by evaluation programs involving smooth elementals and absolute values. In that analysis, a key assumption on the local piecewise linearization was the Linear Independence Kink Qualification (LIKQ), a generalization of the Linear Independence Constraint Qualification (LICQ) … Read more

    Characterizing and testing subdifferential regularity for piecewise smooth objective functions

  • Andreas Griewank
  • Andrea Walther
    • Categories Nonsmooth Optimization Tags , , , , , ,

    Functions defined by evaluation programs involving smooth elementals and absolute values as well as the max- and min-operator are piecewise smooth. Using piecewise linearization we derived in [7] for this class of nonsmooth functions first and second order conditions for local optimality (MIN). They are necessary and sufficient, eespectively. These generalizations of the classical KKT … Read more