Seminar: Optimization, Control and Inverse Problems


This seminar serves as a knowledge exchange and networking platform for the broad area of mathematical optimization and related applications within Berlin.

  • Access to the seminar: A zoom link will be announced here shortly before the seminar
  • Place: Temporarily: Zoom platform
    Place: Weierstrass Institute for Applied Analysis and Stochastics
    Place: Mohrenstraße 39, 10117 Berlin
    Place: Erhard Schmidt Lecture Room (ESH)
  • Time: Mondays (biweekly), 15:00PM - 16:00PM
  • Organizers: René Henrion
    Organizers: Michael Hintermüller
    Organizers: Dietmar Hömberg
    Organizers: Kostas Papafitsoros
29.03.2021 Serge Gratton (ENSEEIHT, Toulouse, France)

On a multilevel Levenberg-Marquardt method for the training of artificial neural networks and its application to the solution of partial differential equations

We propose a new multilevel Levenberg-Marquardt optimizer for the training of artificial neural networks with quadratic loss function. When the least-squares problem arises from the training of artificial neural networks, the variables subject to optimization are not related by any geometrical constraints and the standard interpolation and restriction operators cannot be employed any longer. A heuristic, inspired by algebraic multigrid methods, is then proposed to construct the multilevel transfer operators. We test the new optimizer on an important application: the approximate solution of partial differential equations by means of artificial neural networks. The learning problem is formulated as a least squares problem, choosing the nonlinear residual of the equation as a loss function, whereas the multilevel method is employed as a training method. Numerical experiments show encouraging results related to the efficiency of the new multilevel optimization method compared to the corresponding one-level procedure in this context.