A multilevel proximal trust--region method for nonsmooth optimization with applications
Authors
- Baraldi, Robert
- Hintermüller, Michael
ORCID: 0000-0001-9471-2479 - Wang, Qi
ORCID: 0009-0000-9138-1723
2020 Mathematics Subject Classification
- 35Q93 68T07 49M37 65K05 90C06
Keywords
- Multilevel methods, nonsmooth optimization, global convergence, scientific machine learning, trust-region methods
DOI
Abstract
Many large-scale optimization problems arising in science and engineering are naturally defined at multiple levels of discretization or model fidelity. Multilevel methods exploit this hierarchy to accelerate convergence by combining coarse- and fine-level information, a strategy that has proven highly effective in the numerical solution of partial differential equations and related optimization problems. It turns out that many applications in PDE-constrained optimization and data science require minimizing the sum of smooth and nonsmooth functions. For example, training neural networks may require minimizing a mean squared error plus an L1-regularization to induce sparsity in the weights. Correspondingly, we introduce a multilevel proximal trust-region method to minimize the sum of a non-convex, smooth and a convex, nonsmooth function. Exploiting ideas from the multilevel literature allows us to reduce the cost of the step computation, which is a major bottleneck in single level procedures. Our work unifies theory behind the proximal trust-region methods and multilevel recursive strategies. We prove global convergence of our method in finite dimensional space and provide an efficient non-smooth subproblem solver. We show the efficiency and robustness of our algorithm by means of numerical examples in PDE constrained optimization and machine-learning.
Download Documents