Publications

Monographs

  • M. Danilova, P. Dvurechensky, A. Gasnikov, E. Gorbunov, S. Guminov, D. Kamzolov, I. Shibaev, Recent theoretical advances in non-convex optimization, A. Nikeghbali, P.M. Pardalos, A.M. Raigorodskii, M.Th. Rassias, eds., 191 of Springer Optimization and Its Applications, Springer, Cham, 2022, pp. 79--163, (Chapter Published), DOI 10.1007/978-3-031-00832-0_3 .

  • L. Starke, K. Tabelow, Th. Niendorf, A. Pohlmann, Chapter 34: Denoising for Improved Parametric MRI of the Kidney: Protocol for Nonlocal Means Filtering, in: Preclinical MRI of the Kidney: Methods and Protocols, A. Pohlmann, Th. Niendorf, eds., 2216 of Methods in Molecular Biology, Springer Nature Switzerland AG, Cham, 2021, pp. 565--576, (Chapter Published), DOI 10.1007/978-1-0716-0978-1_34 .

Articles in Refereed Journals

  • S.A. Alves, J. Polzehl, N.M. Brisson, A. Bender, A.N. Agres, P. Damm, G.N. Duda, Ground reaction forces and external hip joint moments predict in vivo hip contact forces during gait, Journal of Biomechanics, 135 (2022), pp. 111037/1--111037/6, DOI 10.1016/j.jbiomech.2022.111037 .

  • M. Coghi, W. Dreyer, P. Gajewski, C. Guhlke, P. Friz, M. Maurelli, A McKean--Vlasov SDE and particle system with interaction from reflecting boundaries, SIAM Journal on Mathematical Analysis, 54 (2022), pp. 2251--2294, DOI 10.1137/21M1409421 .

  • A. Ivanova, A. Gasnikov, P. Dvurechensky, D. Dvinskikh, A. Tyurin, E. Vorontsova, D. Pasechnyuk, Oracle complexity separation in convex optimization, Journal of Optimization Theory and Applications, 193 (2022), pp. 462--490, DOI 10.1007/s10957-022-02038-7 .
    Abstract
    Ubiquitous in machine learning regularized empirical risk minimization problems are often composed of several blocks which can be treated using different types of oracles, e.g., full gradient, stochastic gradient or coordinate derivative. Optimal oracle complexity is known and achievable separately for the full gradient case, the stochastic gradient case, etc. We propose a generic framework to combine optimal algorithms for different types of oracles in order to achieve separate optimal oracle complexity for each block, i.e. for each block the corresponding oracle is called the optimal number of times for a given accuracy. As a particular example, we demonstrate that for a combination of a full gradient oracle and either a stochastic gradient oracle or a coordinate descent oracle our approach leads to the optimal number of oracle calls separately for the full gradient part and the stochastic/coordinate descent part.

  • J. Diehl, R. Preiss, M. Ruddy, N. Tapia, The moving-frame method for iterated-integrals: Orthogonal invariants, Foundations of Computational Mathematics. The Journal of the Society for the Foundations of Computational Mathematics, (2022), published online on 01.06.2022, DOI 10.1007/s10208-022-09569-5 .

  • P. Hager, P. Friz, N. Tapia, Unified signature cumulants and generalized magnus expansions, Forum of Mathematics. Sigma, 10 (2022), published online on 09.06.2022, DOI 10.1017/fms.2022.20 .
    Abstract
    The signature of a path can be described as its full non-commutative exponential. Following T. Lyons we regard its expectation, the expected signature, as path space analogue of the classical moment generating function. The logarithm thereof, taken in the tensor algebra, defines the signature cumulant. We establish a universal functional relation in a general semimartingale context. Our work exhibits the importance of Magnus expansions in the algorithmic problem of computing expected signature cumulants, and further offers a far-reaching generalization of recent results on characteristic exponents dubbed diamond and cumulant expansions; with motivation ranging from financial mathematics to statistical physics. From an affine process perspective, the functional relation may be interpreted as infinite-dimensional, non-commutative ("Hausdorff") variation of Riccati's equation. Many examples are given.

  • S. Mohammadi, T. Streubel, L. Klock, A. Lutti, K. Pine, S. Weber, L. Edwards, P. Scheibe, G. Ziegler, J. Gallinat, S. Kuhn, M. Callaghan, N. Weiskopf, K. Tabelow, Error quantification in multi-parameter mapping facilitates robust estimation and enhanced group level sensitivity, NeuroImage, (2022), published online on 01.08.2022, DOI 10.1016/j.neuroimage.2022.119529 .
    Abstract
    Multi-Parameter Mapping (MPM) is a comprehensive quantitative neuroimaging protocol that enables estimation of four physical parameters (longitudinal and effective transverse relaxation rates R1 and R2*, proton density PD, and magnetization transfer saturation MTsat) that are sensitive to microstructural tissue properties such as iron and myelin content. Their capability to reveal microstructural brain differences, however, is tightly bound to controlling random noise and artefacts (e.g. caused by head motion) in the signal. Here, we introduced a method to estimate the local error of PD, R1, and MTsat maps that captures both noise and artefacts on a routine basis without requiring additional data. To investigate the method's sensitivity to random noise, we calculated the model-based signal-to-noise ratio (mSNR) and showed in measurements and simulations that it correlated linearly with an experimental raw-image-based SNR map. We found that the mSNR varied with MPM protocols, magnetic field strength (3T vs. 7T) and MPM parameters: it halved from PD to R1 and decreased from PD to MT_sat by a factor of 3-4. Exploring the artefact-sensitivity of the error maps, we generated robust MPM parameters using two successive acquisitions of each contrast and the acquisition-specific errors to down-weight erroneous regions. The resulting robust MPM parameters showed reduced variability at the group level as compared to their single-repeat or averaged counterparts. The error and mSNR maps may better inform power-calculations by accounting for local data quality variations across measurements. Code to compute the mSNR maps and robustly combined MPM maps is available in the open-source hMRI toolbox.

  • N. Puchkin, V. Spokoiny, Structure-adaptive manifold estimation, Journal of Machine Learning Research (JMLR). MIT Press, Cambridge, MA. English, English abstracts., 23 (2022), pp. 1--62.
    Abstract
    We consider a problem of manifold estimation from noisy observations. Many manifold learning procedures locally approximate a manifold by a weighted average over a small neighborhood. However, in the presence of large noise, the assigned weights become so corrupted that the averaged estimate shows very poor performance. We suggest a novel computationally efficient structure-adaptive procedure, which simultaneously reconstructs a smooth manifold and estimates projections of the point cloud onto this manifold. The proposed approach iteratively refines the weights on each step, using the structural information obtained at previous steps. After several iterations, we obtain nearly öracle" weights, so that the final estimates are nearly efficient even in the presence of relatively large noise. In our theoretical study we establish tight lower and upper bounds proving asymptotic optimality of the method for manifold estimation under the Hausdorff loss. Our finite sample study confirms a very reasonable performance of the procedure in comparison with the other methods of manifold estimation.

  • P. Dvurechensky, K. Safin, S. Shtern, M. Staudigl, Generalized self-concordant analysis of Frank--Wolfe algorithms, Mathematical Programming. A Publication of the Mathematical Programming Society, (2022), published online on 29.01.2022, DOI 10.1007/s10107-022-01771-1 .
    Abstract
    Projection-free optimization via different variants of the Frank--Wolfe method has become one of the cornerstones of large scale optimization for machine learning and computational statistics. Numerous applications within these fields involve the minimization of functions with self-concordance like properties. Such generalized self-concordant functions do not necessarily feature a Lipschitz continuous gradient, nor are they strongly convex, making them a challenging class of functions for first-order methods. Indeed, in a number of applications, such as inverse covariance estimation or distance-weighted discrimination problems in binary classification, the loss is given by a generalized self-concordant function having potentially unbounded curvature. For such problems projection-free minimization methods have no theoretical convergence guarantee. This paper closes this apparent gap in the literature by developing provably convergent Frank?Wolfe algorithms with standard O(1/k) convergence rate guarantees. Based on these new insights, we show how these sublinearly convergent methods can be accelerated to yield linearly convergent projection-free methods, by either relying on the availability of a local liner minimization oracle, or a suitable modification of the away-step Frank--Wolfe method.

  • P. Friz, T. Klose, Precise Laplace asymptotics for singular stochastic PDEs: The case of 2D gPAM, Journal of Functional Analysis, 283 (2022), pp. 109446/1--109446/86, DOI 10.1016/j.jfa.2022.109446 .

  • O. Butkovsky, K. Dareiotis, M. Geréncser, Approximation of SDEs: A stochastic sewing approach, Probability Theory and Related Fields, 181 (2021), pp. 975--1034, DOI 10.1007/s00440-021-01080-2 .

  • E. Vorontsova, A. Gasnikov, P. Dvurechensky, A. Ivanova, D. Pasechnyuk, Numerical methods for the resource allocation problem in a computer network, Computational Mathematics and Mathematical Physics, 61 (2021), pp. 297--328, DOI 10.1134/S0965542521020135 .

  • C. Bellingeri, A. Djurdjevac, P. Friz, N. Tapia, Transport and continuity equations with (very) rough noise, SN Partial Differential Equations and Applications, 2 (2021), pp. 2--26, DOI 10.1007/s42985-021-00101-y .
    Abstract
    Existence and uniqueness for rough flows, transport and continuity equations driven by general geometric rough paths are established.

  • C. Bellingeri, P. Friz, M. Gerencsér, Singular paths spaces and applications, Stochastic Analysis and Applications, published online on 29.10.2021, DOI 10.1080/07362994.2021.1988641 .

  • K. Ebrahimi-Fard, F. Patras, N. Tapia, L. Zambotti, Wick polynomials in non-commutative probability: A group-theoretical approach, Canadian Journal of Mathematics. Journal Canadien de Mathematiques, published online on 25.08.2021, DOI 10.4153/S0008414X21000407 .
    Abstract
    Wick polynomials and Wick products are studied in the context of non-commutative probability theory. It is shown that free, boolean and conditionally free Wick polynomials can be defined and related through the action of the group of characters over a particular Hopf algebra. These results generalize our previous developments of a Hopf algebraic approach to cumulants and Wick products in classical probability theory.

  • A. Gasnikov, D. Dvinskikh, P. Dvurechensky, D. Kamzolov, V. Matyukhin, D. Pasechnyuk, N. Tupitsa, A. Chernov, Accelerated meta-algorithm for convex optimization, Computational Mathematics and Mathematical Physics, 61 (2021), pp. 17--28, DOI 10.1134/S096554252101005X .

  • A. Kroshnin, V. Spokoiny, A. Suvorikova, Statistical inference for Bures--Wasserstein barycenters, The Annals of Applied Probability, 31 (2021), pp. 1264--1298, DOI 10.1214/20-AAP1618 .

  • L.-Ch. Lin, Y. Chen, G. Pan, V. Spokoiny, Efficient and positive semidefinite pre-averaging realized covariance estimator, Statistica Sinica, 31 (2021), pp. 1441--1462, DOI 10.5705/ss.202017.0489 .

  • M. Redmann, Ch. Bayer, P. Goyal, Low-dimensional approximations of high-dimensional asset price models, SIAM Journal on Financial Mathematics, ISSN 1945-497X, 12 (2021), pp. 1--28, DOI 10.1137/20M1325666 .
    Abstract
    We consider high-dimensional asset price models that are reduced in their dimension in order to reduce the complexity of the problem or the effect of the curse of dimensionality in the context of option pricing. We apply model order reduction (MOR) to obtain a reduced system. MOR has been previously studied for asymptotically stable controlled stochastic systems with zero initial conditions. However, stochastic differential equations modeling price processes are uncontrolled, have non-zero initial states and are often unstable. Therefore, we extend MOR schemes and combine ideas of techniques known for deterministic systems. This leads to a method providing a good pathwise approximation. After explaining the reduction procedure, the error of the approximation is analyzed and the performance of the algorithm is shown conducting several numerical experiments. Within the numerics section, the benefit of the algorithm in the context of option pricing is pointed out.

  • I. Shibaev, P. Dvurechensky, A. Gasnikov, Zeroth-order methods for noisy Hölder-gradient functions, Optimization Letters, published online in April 2021, DOI 10.1007/s11590-021-01742-z .

  • F. Stonyakin, A. Tyurin, A. Gasnikov, P. Dvurechensky, A. Agafonov, D. Dvinskikh, M. Alkousa, D. Pasechnyuk, S. Artamonov, V. Piskunova, Inexact model: A framework for optimization and variational inequalities, Optimization Methods & Software, published online in July 2021, DOI 10.1080/10556788.2021.1924714 .
    Abstract
    In this paper we propose a general algorithmic framework for first-order methods in optimization in a broad sense, including minimization problems, saddle-point problems and variational inequalities. This framework allows to obtain many known methods as a special case, the list including accelerated gradient method, composite optimization methods, level-set methods, proximal methods. The idea of the framework is based on constructing an inexact model of the main problem component, i.e. objective function in optimization or operator in variational inequalities. Besides reproducing known results, our framework allows to construct new methods, which we illustrate by constructing a universal method for variational inequalities with composite structure. This method works for smooth and non-smooth problems with optimal complexity without a priori knowledge of the problem smoothness. We also generalize our framework for strongly convex objectives and strongly monotone variational inequalities.

  • N. Tupitsa, P. Dvurechensky, A. Gasnikov, S. Guminov, Alternating minimization methods for strongly convex optimization, Journal of Inverse and Ill-Posed Problems, 29 (2021), pp. 721--739, DOI 10.1515/jiip-2020-0074 .
    Abstract
    We consider alternating minimization procedures for convex optimization problems with variable divided in many block, each block being amenable for minimization with respect to its variable with freezed other variables blocks. In the case of two blocks, we prove a linear convergence rate for alternating minimization procedure under Polyak-Łojasiewicz condition, which can be seen as a relaxation of the strong convexity assumption. Under strong convexity assumption in many-blocks setting we provide an accelerated alternating minimization procedure with linear rate depending on the square root of the condition number as opposed to condition number for the non-accelerated method.

  • D. Dvinskikh, A. Gasnikov, Decentralized and parallel primal and dual accelerated methods for stochastic convex programming problems, Journal of Inverse and Ill-Posed Problems, 29 (2021), pp. 385--405, DOI 10.1515/jiip-2020-0068 .
    Abstract
    We introduce primal and dual stochastic gradient oracle methods for decentralized convex optimization problems. Both for primal and dual oracles, the proposed methods are optimal in terms of the number of communication steps. However, for all classes of the objective, the optimality in terms of the number of oracle calls per node takes place only up to a logarithmic factor and the notion of smoothness. By using mini-batching technique, we show that the proposed methods with stochastic oracle can be additionally parallelized at each node. The considered algorithms can be applied to many data science problems and inverse problems.

  • D. Dvinskikh, Stochastic approximation versus sample average approximation for Wasserstein barycenters, Optimization Methods & Software, (2021), published online on 09.09.2021, DOI 10.1080/10556788.2021.1965600 .
    Abstract
    In the machine learning and optimization community, there are two main approaches for the convex risk minimization problem, namely the Stochastic Approximation (SA) and the Sample Average Approximation (SAA). In terms of the oracle complexity (required number of stochastic gradient evaluations), both approaches are considered equivalent on average (up to a logarithmic factor). The total complexity depends on a specific problem, however, starting from the work [A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro, Robust stochastic approximation approach to stochastic programming, SIAM. J. Opt. 19 (2009), pp. 1574?1609] it was generally accepted that the SA is better than the SAA. We show that for the Wasserstein barycenter problem, this superiority can be inverted. We provide a detailed comparison by stating the complexity bounds for the SA and SAA implementations calculating barycenters defined with respect to optimal transport distances and entropy-regularized optimal transport distances. As a byproduct, we also construct confidence intervals for the barycenter defined with respect to entropy-regularized optimal transport distances in the ?2-norm. The preliminary results are derived for a general convex optimization problem given by the expectation to have other applications besides the Wasserstein barycenter problem.

  • CH. Bayer, D. Belomestny, P. Hager, P. Pigato, J.G.M. Schoenmakers, Randomized optimal stopping algorithms and their convergence analysis, SIAM Journal on Financial Mathematics, ISSN 1945-497X, 12 (2021), pp. 1201--1225, DOI 10.1137/20M1373876 .
    Abstract
    In this paper we study randomized optimal stopping problems and consider corresponding forward and backward Monte Carlo based optimization algorithms. In particular we prove the convergence of the proposed algorithms and derive the corresponding convergence rates.

  • CH. Bayer, F. Harang, P. Pigato, Log-modulated rough stochastic volatility models, SIAM Journal on Financial Mathematics, ISSN 1945-497X, 12 (2021), pp. 1257--1284, DOI 10.1137/20M135902X .
    Abstract
    We propose a new class of rough stochastic volatility models obtained by modulating the power-law kernel defining the fractional Brownian motion (fBm) by a logarithmic term, such that the kernel retains square integrability even in the limit case of vanishing Hurst index H. The so-obtained log-modulated fractional Brownian motion (log-fBm) is a continuous Gaussian process even for H = 0. As a consequence, the resulting super-rough stochastic volatility models can be analysed over the whole range of Hurst indices between 0 and 1/2, including H = 0, without the need of further normalization. We obtain the usual power law explosion of the skew as maturity T goes to 0, modulated by a logarithmic term, so no flattening of the skew occurs as H goes to 0.

  • P. Dvurechensky, M. Staudigl, S. Shtern, First-order methods for convex optimization, EURO Journal on Computational Optimization, 9 (2021), pp. 100015/1--100015/27, DOI 10.1016/j.ejco.2021.100015 .
    Abstract
    First-order methods for solving convex optimization problems have been at the forefront of mathematical optimization in the last 20 years. The rapid development of this important class of algorithms is motivated by the success stories reported in various applications, including most importantly machine learning, signal processing, imaging and control theory. First-order methods have the potential to provide low accuracy solutions at low computational complexity which makes them an attractive set of tools in large-scale optimization problems. In this survey we cover a number of key developments in gradient-based optimization methods. This includes non-Euclidean extensions of the classical proximal gradient method, and its accelerated versions. Additionally we survey recent developments within the class of projection-free methods, and proximal versions of primal-dual schemes. We give complete proofs for various key results, and highlight the unifying aspects of several optimization algorithms.

  • P. Friz, P. Gassiat, P. Pigato, Precise asymptotics: Robust stochastic volatility models, The Annals of Applied Probability, 31 (2021), pp. 896--940, DOI 10.1214/20-AAP1608 .
    Abstract
    We present a new methodology to analyze large classes of (classical and rough) stochastic volatility models, with special regard to short-time and small noise formulae for option prices. Our main tool is the theory of regularity structures, which we use in the form of Bayer et al. (Math. Finance30 (2020) 782--832) In essence, we implement a Laplace method on the space of models (in the sense of Hairer), which generalizes classical works of Azencott and Ben Arous on path space and then Aida, Inahama--Kawabi on rough path space. When applied to rough volatility models, for example, in the setting of Bayer, Friz and Gatheral (Quant. Finance16 (2016) 887--904) and Forde--Zhang (SIAM J. Financial Math.8 (2017) 114--145), one obtains precise asymptotics for European options which refine known large deviation asymptotics.

  • P. Friz, P. Gassiat, P. Pigato, Short-dated smile under rough volatility: Asymptotics and numerics, Quantitative Finance, published online on 07.12.2021, DOI 10.1080/14697688.2021.1999486 .
    Abstract
    In Friz et al. [Precise asymptotics for robust stochastic volatility models. Ann. Appl. Probab, 2021, 31(2), 896?940], we introduce a new methodology to analyze large classes of (classical and rough) stochastic volatility models, with special regard to short-time and small-noise formulae for option prices, using the framework [Bayer et al., A regularity structure for rough volatility. Math. Finance, 2020, 30(3), 782?832]. We investigate here the fine structure of this expansion in large deviations and moderate deviations regimes, together with consequences for implied volatility. We discuss computational aspects relevant for the practical application of these formulas. We specialize such expansions to prototypical rough volatility examples and discuss numerical evidence.

  • P. Friz, H. Tran, Y. Yuan , Regularity of SLE in (t,k) and refined GRR estimates, Probability Theory and Related Fields, 180 (2021), pp. 71--112, DOI 10.1007/s00440-021-01058-0 .

Contributions to Collected Editions

  • F. Galarce Marín, K. Tabelow, J. Polzehl, Ch. Panagiotis, V. Vavourakis, I. Sack, A. Caiazzo, Assimilation of magnetic resonance elastography displacement data in brain tissues, in: Proceedings of the 7th International Conference on Computational and Mathematical Biomedical Engineering (CMBE22), P. Nithiarasu, C. Vergara, eds., 2, pp. 648--651.

  • D. Yarmoshik, A. Rogozin, O.O. Khamisov, P. Dvurechensky, A. Gasnikov, Decentralized convex optimization under affine constraints for power systems control, in: Mathematical Optimization Theory and Operations Research. MOTOR 2022., P. Pardalos, M. Khachay, V. Mazalov, eds., 13367 of Lecture Notes in Computer Science book series (LNCS), Springer International Publishing AG, Basel, 2020, pp. 62--75.

  • A. Agafonov, P. Dvurechensky, G. Scutari, A. Gasnikov, D. Kamzolov, A. Lukashevich, A. Daneshmand, An accelerated second-order method for distributed stochastic optimization, in: 60th IEEE Conference on Decision and Control (CDC), IEEE, 2021, pp. 2407--2413, DOI 10.1109/CDC45484.2021.9683400 .

  • A. Daneshmand, G. Scutari, P. Dvurechensky, A. Gasnikov, Newton method over networks is fast up to the statistical precision, in: Proceedings of the 38th International Conference on Machine Learning, 139 of Proceedings of Machine Learning Research, 2021, pp. 2398--2409.

  • E. Gladin, A. Sadiev, A. Gasnikov, P. Dvurechensky, A. Beznosikov, M. Alkousa, Solving smooth min-min and min-max problems by mixed oracle algorithms, in: Mathematical Optimization Theory and Operations Research: Recent Trends, A. Strekalovsky, Y. Kochetov, T. Gruzdeva, A. Orlov , eds., 1476 of Communications in Computer and Information Science book series (CCIS), Springer International Publishing, Basel, 2021, pp. 19--40, DOI 10.1007/978-3-030-86433-0_2 .
    Abstract
    In this paper, we consider two types of problems that have some similarity in their structure, namely, min-min problems and min-max saddle-point problems. Our approach is based on considering the outer minimization problem as a minimization problem with an inexact oracle. This inexact oracle is calculated via an inexact solution of the inner problem, which is either minimization or maximization problem. Our main assumption is that the available oracle is mixed: it is only possible to evaluate the gradient w.r.t. the outer block of variables which corresponds to the outer minimization problem, whereas for the inner problem, only zeroth-order oracle is available. To solve the inner problem, we use the accelerated gradient-free method with zeroth-order oracle. To solve the outer problem, we use either an inexact variant of Vaidya's cutting-plane method or a variant of the accelerated gradient method. As a result, we propose a framework that leads to non-asymptotic complexity bounds for both min-min and min-max problems. Moreover, we estimate separately the number of first- and zeroth-order oracle calls, which are sufficient to reach any desired accuracy.

  • S. Guminov, P. Dvurechensky, N. Tupitsa, A. Gasnikov, On a combination of alternating minimization and Nesterov's momentum, in: Proceedings of the 38th International Conference on Machine Learning, 139 of Proceedings of Machine Learning Research, 2021, pp. 3886--3898.
    Abstract
    Alternating minimization (AM) optimization algorithms have been known for a long time and are of importance in machine learning problems, among which we are mostly motivated by approximating optimal transport distances. AM algorithms assume that the decision variable is divided into several blocks and minimization in each block can be done explicitly or cheaply with high accuracy. The ubiquitous Sinkhorn's algorithm can be seen as an alternating minimization algorithm for the dual to the entropy-regularized optimal transport problem. We introduce an accelerated alternating minimization method with a $1/k^2$ convergence rate, where $k$ is the iteration counter. This improves over known bound $1/k$ for general AM methods and for the Sinkhorn's algorithm. Moreover, our algorithm converges faster than gradient-type methods in practice as it is free of the choice of the step-size and is adaptive to the local smoothness of the problem. We show that the proposed method is primal-dual, meaning that if we apply it to a dual problem, we can reconstruct the solution of the primal problem with the same convergence rate. We apply our method to the entropy regularized optimal transport problem and show experimentally, that it outperforms Sinkhorn's algorithm.

  • D. Pasechnyuk, P. Dvurechensky, S. Omelchenko, A. Gasnikov, Stochastic optimization for dynamic pricing, in: Advances in Optimization and Applications, N.N. Olenev, Y.G. Evtushenko, M. Jaćimović, M. Khachay, eds., 1514 of Communications in Computer and Information Science, Springer Nature Switzerland AG, Cham, 2021, pp. 82--94, DOI 10.1007/978-3-030-92711-0 .

  • A. Rogozin, M. Bochko, P. Dvurechensky, A. Gasnikov, V. Lukoshkin, An accelerated method for decentralized distributed stochastic optimization over time-varying graphs, in: 2021 IEEE 60th Annual Conference on Decision and Control (CDC), IEEE, 2021, pp. 3367--3373, DOI 10.1109/CDC45484.2021.9683400 .

  • A. Sadiev , A. Beznosikov, P. Dvurechensky, A. Gasnikov, Zeroth-order algorithms for smooth saddle-point problems, in: Mathematical Optimization Theory and Operations Research: Recent Trends, A. Strekalovsky, Y. Kochetov, T. Gruzdeva, A. Orlov , eds., 1476 of Communications in Computer and Information Science book series (CCIS), Springer International Publishing, Basel, 2021, pp. 71--85, DOI 10.1007/978-3-030-86433-0_5 .
    Abstract
    Saddle-point problems have recently gained an increased attention from the machine learning community, mainly due to applications in training Generative Adversarial Networks using stochastic gradients. At the same time, in some applications only a zeroth-order oracle is available. In this paper, we propose several algorithms to solve stochastic smooth (strongly) convex-concave saddle-point problems using zeroth-order oracles, and estimate their convergence rate and its dependence on the dimension n of the variable. In particular, our analysis shows that in the case when the feasible set is a direct product of two simplices, our convergence rate for the stochastic term is only by a factor worse than for the first-order methods. Finally, we demonstrate the practical performance of our zeroth-order methods on practical problems.

  • K. Safin, P. Dvurechensky, A. Gasnikov, Adaptive gradient-free method for stochastic optimization, in: Advances in Optimization and Applications, N.N. Olenev, Y.G. Evtushenko, M. Jaćimović, M. Khachay, eds., 1514 of Communications in Computer and Information Science, Springer Nature Switzerland AG, Cham, 2021, pp. 95--108, DOI 10.1007/978-3-030-92711-0_7 .

  • D. Dvinskikh, D. Tiapkin, Improved complexity bounds in Wasserstein barycenter problem, in: 24th International Conference on Artificial Intelligence and Statistics (AISTATS), A. Banerjee, K. Fukumizu, eds., 130 of Proceedings of Machine Learning Research, Microtome Publishing, Brookline, MA, USA, 2021, pp. 1738--1746.

Preprints, Reports, Technical Reports

  • K. Ebrahimi-Fard, F. Patras, N. Tapia, L. Zambotti, Shifted substitution in non-commutative multivariate power series with a view towards free probability, Preprint no. 2945, WIAS, Berlin, 2022, DOI 10.20347/WIAS.PREPRINT.2945 .
    Abstract, PDF (341 kByte)
    We study a particular group law on formal power series in non-commuting parameters induced by their interpretation as linear forms on a suitable non-commutative and non- cocommutative graded connected word Hopf algebra. This group law is left-linear and is therefore associated to a pre-Lie structure on formal power series. We study these structures and show how they can be used to recast in a group theoretic form various identities and transformations on formal power series that have been central in the context of non-commutative probability theory, in particular in Voiculescu?s theory of free probability.

  • F. Galarce Marín, K. Tabelow, J. Polzehl, Ch.P. Papanikas, V. Vavourakis, L. Lilaj, I. Sack, A. Caiazzo, Displacement and pressure reconstruction from magnetic resonance elastography images: Application to an in silico brain model, Preprint no. 2933, WIAS, Berlin, 2022, DOI 10.20347/WIAS.PREPRINT.2933 .
    Abstract, PDF (9978 kByte)
    This paper investigates a data assimilation approach for non-invasive quantification of intracranial pressure from partial displacement data, acquired through magnetic resonance elastography. Data assimilation is based on a parametrized-background data weak methodology, in which the state of the physical system tissue displacements and pressure fields is reconstructed from partially available data assuming an underlying poroelastic biomechanics model. For this purpose, a physics-informed manifold is built by sampling the space of parameters describing the tissue model close to their physiological ranges, to simulate the corresponding poroelastic problem, and compute a reduced basis. Displacements and pressure reconstruction is sought in a reduced space after solving a minimization problem that encompasses both the structure of the reduced-order model and the available measurements. The proposed pipeline is validated using synthetic data obtained after simulating the poroelastic mechanics on a physiological brain. The numerical experiments demonstrate that the framework can exhibit accurate joint reconstructions of both displacement and pressure fields. The methodology can be formulated for an arbitrary resolution of available displacement data from pertinent images. It can also inherently handle uncertainty on the physical parameters of the mechanical model by enlarging the physics-informed manifold accordingly. Moreover, the framework can be used to characterize, in silico, biomarkers for pathological conditions, by appropriately training the reduced-order model. A first application for the estimation of ventricular pressure as an indicator of abnormal intracranial pressure is shown in this contribution.

  • M.G. Varzaneh, S. Riedel, A. Schmeding, N. Tapia, The geometry of controlled rough paths, Preprint no. 2926, WIAS, Berlin, 2022, DOI 10.20347/WIAS.PREPRINT.2926 .
    Abstract, PDF (472 kByte)
    We prove that the spaces of controlled (branched) rough paths of arbitrary order form a continuous field of Banach spaces. This structure has many similarities to an (infinite-dimensional) vector bundle and allows to define a topology on the total space, the collection of all controlled path spaces, which turns out to be Polish in the geometric case. The construction is intrinsic and based on a new approximation result for controlled rough paths. This framework turns well-known maps such as the rough integration map and the Itô-Lyons map into continuous (structure preserving) mappings. Moreover, it is compatible with previous constructions of interest in the stability theory for rough integration.

  • CH. Bayer, D. Belomestny, O. Butkovsky, J.G.M. Schoenmakers, RKHS regularization of singular local stochastic volatility McKean--Vlasov models, Preprint no. 2921, WIAS, Berlin, 2022, DOI 10.20347/WIAS.PREPRINT.2921 .
    Abstract, PDF (504 kByte)
    Motivated by the challenges related to the calibration of financial models, we consider the problem of solving numerically a singular McKean-Vlasov equation, which represents a singular local stochastic volatility model. Whilst such models are quite popular among practitioners, unfortunately, its well-posedness has not been fully understood yet and, in general, is possibly not guaranteed at all. We develop a novel regularization approach based on the reproducing kernel Hilbert space (RKHS) technique and show that the regularized model is well-posed. Furthermore, we prove propagation of chaos. We demonstrate numerically that a thus regularized model is able to perfectly replicate option prices due to typical local volatility models. Our results are also applicable to more general McKean--Vlasov equations.

  • CH. Bayer, Ch. Ben Hammouda, R.F. Tempone, Numerical smoothing with hierarchical adaptive sparse grids and quasi-Monte Carlo methods for efficient option pricing, Preprint no. 2917, WIAS, Berlin, 2022.
    Abstract, PDF (654 kByte)
    When approximating the expectation of a functional of a stochastic process, the efficiency and performance of deterministic quadrature methods, such as sparse grid quadrature and quasi-Monte Carlo (QMC) methods, may critically depend on the regularity of the integrand. To overcome this issue and reveal the available regularity, we consider cases in which analytic smoothing cannot be performed, and introduce a novel numerical smoothing approach by combining a root finding algorithm with one-dimensional integration with respect to a single well-selected variable. We prove that under appropriate conditions, the resulting function of the remaining variables is a highly smooth function, potentially affording the improved efficiency of adaptive sparse grid quadrature (ASGQ) and QMC methods, particularly when combined with hierarchical transformations (i.e., Brownian bridge and Richardson extrapolation on the weak error). This approach facilitates the effective treatment of high dimensionality. Our study is motivated by option pricing problems, and our focus is on dynamics where the discretization of the asset price is necessary. Based on our analysis and numerical experiments, we show the advantages of combining numerical smoothing with the ASGQ and QMC methods over ASGQ and QMC methods without smoothing and the Monte Carlo approach.

  • CH. Bayer, E. Hall, R.F. Tempone, Weak error rates for option pricing under linear rough volatility, Preprint no. 2916, WIAS, Berlin, 2022, DOI 10.20347/WIAS.PREPRINT.2916 .
    Abstract, PDF (1685 kByte)
    In quantitative finance, modeling the volatility structure of underlying assets is vital to pricing options. Rough stochastic volatility models, such as the rough Bergomi model [Bayer, Friz, Gatheral, Quantitative Finance 16(6), 887-904, 2016], seek to fit observed market data based on the observation that the log-realized variance behaves like a fractional Brownian motion with small Hurst parameter, H < 1/2, over reasonable timescales. Both time series of asset prices and option-derived price data indicate that H often takes values close to 0.1 or less, i.e., rougher than Brownian motion. This change improves the fit to both option prices and time series of underlying asset prices while maintaining parsimoniousness. However, the non-Markovian nature of the driving fractional Brownian motion in rough volatility models poses severe challenges for theoretical and numerical analyses and for computational practice. While the explicit Euler method is known to converge to the solution of the rough Bergomi and similar models, its strong rate of convergence is only H. We prove rate H + 1/2 for the weak convergence of the Euler method for the rough Stein--Stein model, which treats the volatility as a linear function of the driving fractional Brownian motion, and, surprisingly, we prove rate one for the case of quadratic payoff functions. Indeed, the problem of weak convergence for rough volatility models is very subtle; we provide examples demonstrating the rate of convergence for payoff functions that are well approximated by second-order polynomials, as weighted by the law of the fractional Brownian motion, may be hard to distinguish from rate one empirically. Our proof uses Talay--Tubaro expansions and an affine Markovian representation of the underlying and is further supported by numerical experiments. These convergence results provide a first step toward deriving weak rates for the rough Bergomi model, which treats the volatility as a nonlinear function of the driving fractional Brownian motion.

  • D. Belomestny, Ch. Bender, J.G.M. Schoenmakers, Solving optimal stopping problems via randomization and empirical dual optimization, Preprint no. 2884, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2884 .
    Abstract, PDF (706 kByte)
    In this paper we consider optimal stopping problems in their dual form. In this way we reformulate the optimal stopping problem as a problem of stochastic average approximation (SAA) which can be solved via linear programming. By randomizing the initial value of the underlying process, we enforce solutions with zero variance while preserving the linear programming structure of the problem. A careful analysis of the randomized SAA algorithm shows that it enjoys favorable properties such as faster convergence rates and reduced complexity as compared to the non randomized procedure. We illustrate the performance of our algorithm on several benchmark examples.

  • CH. Bayer, S. Breneis, Markovian approximations of stochastic Volterra equations with the fractional kernel, Preprint no. 2868, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2868 .
    Abstract, PDF (525 kByte)
    We consider rough stochastic volatility models where the variance process satisfies a stochastic Volterra equation with the fractional kernel, as in the rough Bergomi and the rough Heston model. In particular, the variance process is therefore not a Markov process or semimartingale, and has quite low Hölder-regularity. In practice, simulating such rough processes thus often results in high computational cost. To remedy this, we study approximations of stochastic Volterra equations using an N-dimensional diffusion process defined as solution to a system of ordinary stochastic differential equation. If the coefficients of the stochastic Volterra equation are Lipschitz continuous, we show that these approximations converge strongly with superpolynomial rate in N. Finally, we apply this approximation to compute the implied volatility smile of a European call option under the rough Bergomi and the rough Heston model.

  • A. Sadiev, A. Beznosikov, P. Dvurechensky, A. Gasnikov, Zeroth-order algorithms for smooth saddle-point problems, Preprint no. 2827, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2827 .
    Abstract, PDF (564 kByte)
    Saddle-point problems have recently gained an increased attention from the machine learning community, mainly due to applications in training Generative Adversarial Networks using stochastic gradients. At the same time, in some applications only a zeroth-order oracle is available. In this paper, we propose several algorithms to solve stochastic smooth (strongly) convex-concave saddle- point problems using zeroth-order oracles, and estimate their convergence rate and its dependence on the dimension n of the variable. In particular, our analysis shows that in the case when the feasible set is a direct product of two simplices, our convergence rate for the stochastic term is only by a log n factor worse than for the first-order methods. We also consider a mixed setup and develop 1/2th-order methods which use zeroth-order oracle for the minimization part and first-order oracle for the maximization part. Finally, we demonstrate the practical performance of our zeroth-order and 1/2th-order methods on practical problems.

  • CH. Bayer, M. Eigel, L. Sallandt, P. Trunschke, Pricing high-dimensional Bermudan options with hierarchical tensor formats, Preprint no. 2821, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2821 .
    Abstract, PDF (321 kByte)
    An efficient compression technique based on hierarchical tensors for popular option pricing methods is presented. It is shown that the “curse of dimensionality" can be alleviated for the computation of Bermudan option prices with the Monte Carlo least-squares approach as well as the dual martingale method, both using high-dimensional tensorized polynomial expansions. This discretization allows for a simple and computationally cheap evaluation of conditional expectations. Complexity estimates are provided as well as a description of the optimization procedures in the tensor train format. Numerical experiments illustrate the favourable accuracy of the proposed methods. The dynamical programming method yields results comparable to recent Neural Network based methods.

  • P. Ostroukhov, R. Kamalov, P. Dvurechensky, A. Gasnikov, Tensor methods for strongly convex strongly concave saddle point problems and strongly monotone variational inequalities, Preprint no. 2820, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2820 .
    Abstract, PDF (302 kByte)
    In this paper we propose three tensor methods for strongly-convex-strongly-concave saddle point problems (SPP). The first method is based on the assumption of higher-order smoothness (the derivative of the order higher than 2 is Lipschitz-continuous) and achieves linear convergence rate. Under additional assumptions of first and second order smoothness of the objective we connect the first method with a locally superlinear converging algorithm in the literature and develop a second method with global convergence and local superlinear convergence. The third method is a modified version of the second method, but with the focus on making the gradient of the objective small. Since we treat SPP as a particular case of variational inequalities, we also propose two methods for strongly monotone variational inequalities with the same complexity as the described above.

  • A. Agafonov, D. Kamzolov, P. Dvurechensky, A. Gasnikov, Inexact tensor methods and their application to stochastic convex optimization, Preprint no. 2818, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2818 .
    Abstract, PDF (333 kByte)
    We propose a general non-accelerated tensor method under inexact information on higher- order derivatives, analyze its convergence rate, and provide sufficient conditions for this method to have similar complexity as the exact tensor method. As a corollary, we propose the first stochastic tensor method for convex optimization and obtain sufficient mini-batch sizes for each derivative.

  • V. Matyukhin, S. Kabanikhin, M. Shishlenin, N. Novikov, A. Vasin, A. Gasnikov, Convex optimization with inexact gradients in Hilbert space and applications to elliptic inverse problems, Preprint no. 2815, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2815 .
    Abstract, PDF (1346 kByte)
    In this paper we propose the gradient descent type methods to solve convex optimization problems in Hilbert space. We apply it to solve ill-posed Cauchy problem for Poisson equation and make a comparative analysis with Landweber iteration and steepest descent method. The theoretical novelty of the paper consists in the developing of new stopping rule for accelerated gradient methods with inexact gradient (additive noise). Note that up to the moment of stopping the method “doesn't feel the noise”. But after this moment the noise start to accumulate and the quality of the solution becomes worse for further iterations.

  • P. Friz, P. Hager, N. Tapia, Unified signature cumulants and generalized Magnus expansions, Preprint no. 2814, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2814 .
    Abstract, PDF (388 kByte)
    The signature of a path can be described as its full non-commutative exponential. Following T. Lyons we regard its expectation, the expected signature, as path space analogue of the classical moment generating function. The logarithm thereof, taken in the tensor algebra, defines the signature cumulant. We establish a universal functional relation in a general semimartingale context. Our work exhibits the importance of Magnus expansions in the algorithmic problem of computing expected signature cumulants, and further offers a far-reaching generalization of recent results on characteristic exponents dubbed diamond and cumulant expansions; with motivation ranging from financial mathematics to statistical physics. From an affine process perspective, the functional relation may be interpreted as infinite-dimensional, non-commutative (“Hausdorff") variation of Riccati's equation. Many examples are given.

  • N. Yudin, A. Gasnikov, Flexible modification of Gauss--Newton method and its stochastic extension, Preprint no. 2813, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2813 .
    Abstract, PDF (8261 kByte)
    This work presents a novel version of recently developed Gauss--Newton method for solving systems of nonlinear equations, based on upper bound of solution residual and quadratic regularization ideas. We obtained for such method global convergence bounds and under natural non-degeneracy assumptions we present local quadratic convergence results. We developed stochastic optimization algorithms for presented Gauss--Newton method and justified sub-linear and linear convergence rates for these algorithms using weak growth condition (WGC) and Polyak--Lojasiewicz (PL) inequality. We show that Gauss--Newton method in stochastic setting can effectively find solution under WGC and PL condition matching convergence rate of the deterministic optimization method. The suggested method unifies most practically used Gauss--Newton method modifications and can easily interpolate between them providing flexible and convenient method easily implementable using standard techniques of convex optimization.

  • A. Vasin, A. Gasnikov, V. Spokoiny, Stopping rules for accelerated gradient methods with additive noise in gradient, Preprint no. 2812, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2812 .
    Abstract, PDF (1129 kByte)
    In this article, we investigate an accelerated first-order method, namely, the method of similar triangles, which is optimal in the class of convex (strongly convex) problems with a Lipschitz gradient. The paper considers a model of additive noise in a gradient and a Euclidean prox- structure for not necessarily bounded sets. Convergence estimates are obtained in the case of strong convexity and its absence, and a stopping criterion is proposed for not strongly convex problems.

  • D. Belomestny, J.G.M. Schoenmakers, From optimal martingales to randomized dual optimal stopping, Preprint no. 2810, WIAS, Berlin, 2021.
    Abstract, PDF (571 kByte)
    In this article we study and classify optimal martingales in the dual formulation of optimal stopping problems. In this respect we distinguish between weakly optimal and surely optimal martingales. It is shown that the family of weakly optimal and surely optimal martingales may be quite large. On the other hand it is shown that the Doob-martingale, that is, the martingale part of the Snell envelope, is in a certain sense the most robust surely optimal martingale under random perturbations. This new insight leads to a novel randomized dual martingale minimization algorithm that does`nt require nested simulation. As a main feature, in a possibly large family of optimal martingales the algorithm efficiently selects a martingale that is as close as possible to the Doob martingale. As a result, one obtains the dual upper bound for the optimal stopping problem with low variance.

  • A. Neumann, N. Peitek, A. Brechmann, K. Tabelow, Th. Dickhaus, Utilizing anatomical information for signal detection in functional magnetic resonance imaging, Preprint no. 2806, WIAS, Berlin, 2021, DOI 10.20347/WIAS.PREPRINT.2806 .
    Abstract, PDF (2995 kByte)
    We are considering the statistical analysis of functional magnetic resonance imaging (fMRI) data. As demonstrated in previous work, grouping voxels into regions (of interest) and carrying out a multiple test for signal detection on the basis of these regions typically leads to a higher sensitivity when compared with voxel-wise multiple testing approaches. In the case of a multi-subject study, we propose to define the regions for each subject separately based on their individual brain anatomy, represented, e.g., by so-called Aparc labels. The aggregation of the subject-specific evidence for the presence of signals in the different regions is then performed by means of a combination function for p-values. We apply the proposed methodology to real fMRI data and demonstrate that our approach can perform comparably to a two-stage approach for which two independent experiments are needed, one for defining the regions and one for actual signal detection.

Talks, Poster

  • S. Breneis, An error representation formula for the log-ode method, 15th Berlin-Oxford Young Researcher's Meeting on Applied Stochastic Analysis, May 12 - 14, 2022, WIAS & TU Berlin, May 14, 2022.

  • S. Breneis, Markovian approximations for rough volatility models, Seminar der Stochastic Numerics Research Group, King Abdullah University of Science and Technology, Stochastic Numerics Research Group, Thuval, Saudi Arabia, January 26, 2022.

  • O. Butkovsky, Regularisation by noise for SDEs: State of the art & open problems, Regularization by noise: Theoretical foundations, numerical methods and applications driven by Levy noise, February 13 - 20, 2022, Mathematisches Forschungsinstitut Oberwolfach, February 16, 2022.

  • O. Butkovsky, Strong rate of convergence of the Euler scheme for SDEs with irregular drift driven by Levy noise, 15th Berlin-Oxford Young Researcher's Meeting on Applied Stochastic Analysis, May 12 - 14, 2022, WIAS & TU Berlin, May 12, 2022.

  • O. Butkovsky, Weak and mild solutions of SPDEs with distributional drift (online talk), 42nd Conference on Stochastic Processes and their Applications (Online Event), June 27 - July 1, 2022, Wuhan University, School of Mathematics and Statistics, Chinese Society of Probability and Statistics, China, June 28, 2022.

  • W. Salkeld, Lions calculus and regularity structures, Probability and Mathematical Physics 2022, Finland, June 28 - July 4, 2022.

  • W. Salkeld, Lions calculus and rough mean-field equation, Journées TRAG 2022, May 30 - June 1, 2022, Université Paris Nanterre, GdR TRAG (TRAjectoires ruGueuses).

  • W. Salkeld, Random controlled rough paths, 15th Berlin-Oxford Young Researcher's Meeting on Applied Stochastic Analysis, May 12 - 14, 2022, WIAS & TU Berlin, May 12, 2022.

  • W. Salkeld, Random controlled rough paths (online talk), Thursdays Seminar (Online Event), TU Berlin, Institut für Mathematik, March 10, 2022.

  • N. Tapia, The moving frame method for iterated-integrals signatures: Orthogonal invariants (online talk), Arbeitsgruppenseminar Analysis (Online Event), Universität Potsdam, Institut für Mathematik, January 28, 2022.

  • N. Tapia, Stability of deep neural networks via discrete rough paths, 15th Berlin-Oxford Young Researcher's Meeting on Applied Stochastic Analysis, May 12 - 14, 2022, WIAS & TU Berlin, May 13, 2022.

  • N. Tapia, Stability of deep neural networks via discrete rough paths (online talk), Rough Analysis and Data Science workshop 2022, July 26 - 27, 2022, Imperial College London, Department of Mathematics, UK, July 27, 2022.

  • N. Tapia, Transport and continuity equations with (very) rough noise, Regularization by noise: Theoretical foundations, numerical methods and applications driven by Levy noise, February 13 - 19, 2022, Mathematisches Forschungsinstitut Oberwolfach, February 18, 2022.

  • Y. Vargas, Algebraic combinatorics of moment-to-cumulant relations, Summer School in Algebraic Combinatorics, Krakau, Poland, July 11 - 15, 2022.

  • Y. Vargas, Combinational moment-to-cumulant formulas in free probability, TU Graz, Institut für Diskrete Mathematik, Austria, June 2, 2022.

  • Y. Vargas, Cumulant-to-moment relations from Hopf algebras, 15th Berlin-Oxford Young Researcher's Meeting on Applied Stochastic Analysis, May 12 - 14, 2022, WIAS & TU Berlin, May 12, 2022.

  • Y. Vargas, Primitiv basis for the Hopf-Algebra of permutations, Séminaire de combinatoire, Université Gustave Eiffel, Laboratoire d'Informatique Gaspard-Monge, Marne-la-Vallée, France, July 1, 2022.

  • P. Dvurechensky, Multimarginal optimal transport by accelerated alternating minimization (online talk), SIAM Conference on Imaging Science: IS22 (Online Event), March 21 - 25, 2022, Society for Industrial and Applied Mathematics, March 24, 2022.

  • CH. Bayer, Efficient Markovian approximation to rough volatility models, Rough Volatility Meeting, Imperial College London, UK, March 16, 2022.

  • CH. Bayer, Efficient markovian approximation of rough stochastic volatility models (online talk), Aarhus/SMU Volatility Workshop (Online Event), Aarhus University, Dept. of Economics and Business, Denmark, May 31, 2022.

  • CH. Bayer, Machine learning techniques in computational finance, Stochastic Numerics and Statistical Learning: Theory and Applications Workshop, May 15 - 28, 2022, King Abdullah University, Computer, Electrical and Mathematical Sciences and Engineering Division, Thuwal, Saudi Arabia, May 22, 2022.

  • CH. Bayer, On the existence and longtime behavior of solutions to a degenerate parabolic system (online talk), Regularization by noise: Theoretical foundations, numerical methods and applications driven by Levy noise, February 13 - 20, 2022, Mathematisches Forschungsinstitut Oberwolfach, February 14, 2022.

  • CH. Bayer, Optimal stopping with signatures, ISOR Colloquium, June 13 - 14, 2022, Universität Wien, Department of Statistics and Operations Research, Austria, June 13, 2022.

  • CH. Bayer, Optimal stopping with signatures, Advances in Mathematical Finance and Optimal Transport, June 27 - 30, 2022, Scuola Normale Superiore di Pisa, Centro di Ricerca Matematica Ennio De Giorgi, Italy, June 28, 2022.

  • CH. Bayer, Optimal stopping with signatures, Rough Analysis and Data Science workshop 2022, July 26 - 27, 2022, Imperial College London, Department of Mathematics, UK, July 27, 2022.

  • CH. Bayer, Optimal stopping with signatures, Oberseminar, Martin-Luther-Universität Halle Wittenberg, Institut für Mathematik, June 14, 2022.

  • CH. Bayer, Optimal stopping, machine learning, and signatures, Seminar der Stochastic Numerics Research Group, King Abdullah University of Science and Technology, Stochastic Numerics Research Group, Thuval, Saudi Arabia, January 31, 2022.

  • CH. Bayer, Simulating rough volatility models (online talk), MathFinance 2022 Conference (Online Event), March 21 - 22, 2022, March 22, 2022.

  • P. Dvurechensky, Accelerated alternating minimization methods, 20th French German Portuguese conference in optimization, May 3 - 6, 2022, University of Porto, School of Economics and Management, Portugal, May 5, 2022.

  • P. Dvurechensky, Generalized self-concordant analysis of Frank-Wolfe algorithms, 19th Workshop on Advances in Continuous Optimization (EUROPT 2022), July 29 - 30, 2022, NOVA University Lisbon, School of Science and Technology, Portugal, July 30, 2022.

  • P. Friz, A theory of rough differential equations (online talk), Webinar on Stochastic Analysis (Online Event), Beijing Institute of Technology, School of Mathematics and Statistics, China, March 31, 2022.

  • P. Friz, Local vol under rough vol, Rough Volatility Workshop, March 15 - 16, 2022, Imperial College London, UK, March 16, 2022.

  • P. Friz, Rough SDEs, rough semimartingales, Advances in Mathematical Finance and Optimal Transport, June 27 - July 1, 2022, Scuola Normale Superiore di Pisa, Centro di Ricerca Matematica Ennio De Giorgi, Italy, June 28, 2022.

  • TH. Koprucki, K. Tabelow, HackMD (online talk), E-Coffee-Lecture (Online Event), WIAS Berlin, March 25, 2022.

  • V. Spokoiny, Bayesian optimization by Laplace iterations, Workshop on statistical inference and convex optimization, June 13 - 15, 2022, Université Grenoble Alpes, Laboratoire Jean Kuntzmann, France, June 13, 2022.

  • V. Spokoiny, Laplace approximation in high dimension, Re-thinking High-dimensional Mathematical Statistics, May 16 - 20, 2022, Mathematisches Forschungsinstitut Oberwolfach (MFO), May 17, 2022.

  • K. Tabelow, Neural MRI, Tandem tutorial ``Mathematics of Imaging' ', Berlin Mathematics Research Center Math+, February 18, 2022.

  • S. Breneis, Markovian approximations of stochastic Volterra equations with the fractional kernel, 2021 Summer School of Berlin-Oxford IRTG Stochastic Analysis in Interaction (Hybrid Event), September 20 - 24, 2021, Technische Universität Berlin, Institut für Mathematik, September 24, 2021.

  • S. Breneis, Markovian approximations of stochastic volatility models (online talk), 16. DoktorandInnentreffen der Stochastik (Online Event), Ludwig-Maximilians-Universität München, Fakultät für Mathematik, Informatik und Statistik, July 30, 2021.

  • S. Breneis, On variation functions and their moduli of continuity (online talk), Methods of Nonlinear Analysis in Differential and Integral Equations (Online Event), May 15 - 16, 2021, Rzeszów University of Technology, Department of Nonlinear Analysis, Poland, May 16, 2021.

  • O. Butkovsky, New coupling techniques for exponential ergodicity of SPDEs in the hypoelliptic and effectively elliptic settings (online talk), Applied and Computational Mathematics Research Seminar, Tulane University, School of Science and Engineering, New Orleans, USA, April 30, 2021.

  • O. Butkovsky, Regularization by noise for PDEs: A stochastic sewing approach (online talk), Theory of Probability and Its Applications: P. L. Chebyshev -- 200 (The 6th International Conference on Stochastic Methods) (Online Event), May 17 - 22, 2021, Steklov Mathematical Institute of Russian Academy of Sciences, Moscow, Russian Federation, May 22, 2021.

  • O. Butkovsky, Regularization by noise for SPDEs and SDEs: A stochastic sewing approach (online talk), Bernoulli-IMS 10th World Congress in Probability and Statistics (Online Event), July 19 - 23, 2021, Seoul National University, Korea (Republic of), July 22, 2021.

  • O. Butkovsky, Regularization by noise via stochastic sewing with random controls, German Probability and Statistics Days (GPSD) 2021, September 27 - October 1, 2021, DMV-Fachgruppe Stochastik e.V., Mannheim, September 27, 2021.

  • O. Butkovsky, Skew fractional Brownian motion (online talk), LSA Autumn Meeting 2021 (Online Event), September 20 - 24, 2021, National Research University -- Higher School of Economics (HSE), Laboratory of Stochastic Analysis and its Applications, Moscow, Russian Federation, September 22, 2021.

  • O. Butkovsky, Skew fractional Brownian motion: Going beyond the Catellier--Gubinelli setting (online talk), 14th Oxford-Berlin Young Researchers Meeting on Applied Stochastic Analysis (Online Event), February 10 - 12, 2021, University of Oxford, Mathematical Institute, UK, February 11, 2021.

  • N. Tapia, Approximation of controlled rough paths (online talk), 14th Oxford-Berlin Young Researchers Meeting on Applied Stochastic Analysis (Online Event), February 10 - 12, 2021, University of Oxford, Mathematical Institute, UK, February 10, 2021.

  • N. Tapia, Iterated sums for time series classification, Leibniz MMS Summer School ``Mathematical Methods for Machine Learning'', August 22 - 27, 2021, Schloss Dagstuhl, Leibniz-Zentrum für Informatik, Wadern, August 23, 2021.

  • N. Tapia, Numerical schemes for rough partial differential equations (online talk), DNA Seminar (Online Event), Norwegian University of Science and Technology, Department of Mathematical Sciences, Trondheim, Norway, March 12, 2021.

  • N. Tapia, Robustness of ResNets, Forschungsseminar, Universität des Saarlandes, Fakultät für Mathematik und Informatik, Saarbrücken, October 29, 2021.

  • N. Tapia, Series iteradas para clasificación de series de tiempo (online talk), Seminario Chileno de Probabilidades (Online Event), Universidad de Chile, Centro de Modelamiento Matemático, Santiago de Chile, Chile, March 31, 2021.

  • N. Tapia, Transport and continuity equations with (very) rough noise (online talk), Bernoulli-IMS 10th World Congress in Probability and Statistics (Online Event), July 19 - 23, 2021, Seoul National University, Korea (Republic of), July 20, 2021.

  • N. Tapia, Transport and continuity equations with (very) rough noise (online talk), Seminario de Probabilidad Hispanohablante (Online Event), Universidad de Buenos Aires, Facultad de Ciencias Exactas y Naturales, Argentina, March 1, 2021.

  • N. Tapia, Unified signature cumulants and generalized Magnus expansions, Pathwise Stochastic Analysis and Applications (Online Event), March 8 - 12, 2021, Centre International de Rencontres Mathématiques, France, March 11, 2021.

  • D. Dvinskikh, D. Tiapkin, Improved complexity bounds in Wasserstein barycenter problem (online presentation), The 24th International Conference on Artificial Intelligence and Statistics (Online Event), April 13 - 15, 2021.

  • D. Dvinskikh, A clever mean: Wasserstein barycenters, Education Program: Modern Methods of Information Theory, Optimization and Management, July 19 - August 8, 2021, Sirius University of Science and Technology, Sochi, Russian Federation, July 19, 2021.

  • D. Dvinskikh, Decentralized algorithms for Wasserstein barycenters (online talk), Moscow Conference on Combinatorics and Applications (Online Event), May 31 - June 4, 2021, Moscow Institute of Physics and Technology, School of Applied Mathematics and Computer Science, Moscow, Russian Federation, June 2, 2021.

  • A. Suvorikova, Optimal transport in machine learning (onlne talk), Seminar of the International Laboratory of Stochastic Algorithms and High-dimensional Inference (HDI Lab), National Research University -- Higher School of Economics (HSE), Faculty of Computer Science (Online Event), Moscow, Russian Federation, April 29, 2021.

  • A. Suvorikova, Statistics for non-statisticians (online talk), ITaS Interdisciplinary Conference 2021 (Online Event), November 15 - 17, 2021, Russian Academy of Sciences, Institute for Information Transmission Problems, Moscow, Russian Federation, November 16, 2021.

  • A. Suvorikova, Survey of methods of k-means clustering with optimal transport (online talk), Third HSE-Yandex Autumn School on Generative Models (Hybrid Event), November 23 - 26, 2021, National Research University -- Higher School of Economics (HSE), Yandex SDA Campus, Moscow, Russian Federation, November 26, 2021.

  • CH. Bayer, A pricing BSPDE for rough volatility (online talk), MATH4UQ Seminar (Online Event), Rheinisch-Westfälische Technische Hochschule Aachen, Mathematics for Uncertainty Quantification, April 6, 2021.

  • CH. Bayer, Introduction to statistical learning theory, Leibniz MMS Summer School, August 22 - 27, 2021, Schloss Dagstuhl, Leibniz-Zentrum für Informatik, August 23, 2021.

  • CH. Bayer, Log-modulated rough stochastic volatility models (online talk), SIAM Conference on Financial Mathematics and Engineering (FM21, Online Event), Minisymposium MS46: ``UQ for Rough Volatility and Predictive Models in Finance'', June 1 - 4, 2021, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, USA, June 4, 2021.

  • CH. Bayer, Pricing of American options using high-dimensional non-linear networks, Next Generation Models of Financial Data, September 20 - 22, 2021, Technischen Universität München, Fakultät für Mathematik, Burghausen, September 21, 2021.

  • A. Caiazzo, F. Galarce Marín, J. Polzehl, I. Sack, K. Tabelow, Physics based assimilation of displacements data from magnetic resonance elastography, Kick-off Workshop of the MATH+ Thematic Einstein Semester on Mathematics of Imaging in Real-World Challenges (Hybrid Event), Berlin, October 6 - 8, 2021.

  • P. Dvurechensky, A short introduction to optimization (online talk), ITaS Interdisciplinary Conference 2021 (Online Event), November 15 - 17, 2021, Russian Academy of Sciences, Institute for Information Transmission Problems, Moscow, Russian Federation, November 15, 2021.

  • P. Dvurechensky, Accelerated gradient methods and their applications to Wasserstein barycenter problem (online talk), The XIII International Scientific Conference and Young Scientist School ``Theory and Numerics of Inverse and Ill-posed Problems'' (Online Event), April 12 - 22, 2021, Mathematical Center in Akademgorodok, Novosibirsk, Russian Federation, April 14, 2021.

  • P. Dvurechensky, Decentralize and randomize: Faster algorithm for Wasserstein barycenters (online talk), EUROPT 2021, 18th International Workshop on Continuous Optimization (Online Event), July 7 - 9, 2021, Toulouse, France, July 7, 2021.

  • P. Dvurechensky, Distributed optimization algorithms for Wasserstein barycenters (online talk), 2021 INFORMS Annual Meeting, October 24 - 27, 2021, Institute for Operations Research and the Management Sciences (Hybrid Event), Anaheim, California, USA, October 24, 2021.

  • P. Dvurechensky, Newton method over networks is fast up to the statistical precision (online talk), Thirty-eighth International Conference on Machine Learning (Online Event), July 18 - 24, 2021, Carnegie Mellon University, Pittsburgh, USA, July 20, 2021.

  • P. Dvurechensky, On a combination of alternating minimization and Nesterov's momentum (online talk), Thirty-eighth International Conference on Machine Learning (Online Event), July 18 - 24, 2021, Carnegie Mellon University, Pittsburgh, USA, July 20, 2021.

  • P. Dvurechensky, Primal-dual accelerated gradient methods with alternating minimization (online talk), Conference Optimization without Borders, July 12 - 18, 2021, Sirius University of Science and Technology, Sochi, Russian Federation, July 15, 2021.

  • P. Dvurechensky, Wasserstein barycenters from the computational perspective (online talk), Moscow Conference on Combinatorics and Applications (Online Event), May 31 - June 4, 2021, Moscow Institute of Physics and Technology, School of Applied Mathematics and Computer Science, Moscow, Russian Federation, June 2, 2021.

  • P. Friz, Liouville Brownian rough paths (online talk), Probability Seminar, Universität Wien, Fakultät für Mathematik, Austria, November 14, 2021.

  • P. Friz, Local volatility under rough volatility, QuantMinds International 2021, December 6 - 9, 2021, Barcelona, Spain, December 7, 2021.

  • P. Friz, New perspectives on rough paths, signatures and signature cumulants (online talk), DataSig Seminar Series (Online Event), University of Oxford, Mathematical Institute, UK, May 6, 2021.

  • P. Friz, On rough SDEs (online talk), International Seminar on SDEs and Related Topics (Online Event), hosted by University of Jyväskylä, Department of Mathematics and Statistics, October 29, 2021.

  • P. Friz, Rough stochastic differential equations, Probability Seminar, Maxwell Institute for Mathematical Science, Edinburgh, UK, October 7, 2021.

  • P. Friz, Rough stochastic differential equations (online talk), Pathwise Stochastic Analysis and Applications (Online Event), March 8 - 12, 2021, Centre International de Rencontres Mathématiques, France, March 8, 2021.

  • P. Friz, Unified cumulants and Magnus expansions, Noncommutative Algebra, Probability and Analysis in Action (Hybrid Event), September 20 - 25, 2021, Universität Greifswald, Alfried Krupp Wissenschaftskolleg, September 21, 2021.

  • P. Friz, What can mathematics do for artificial intelligence? (online talk), Berlin Research 50 Workshop on Artificial Intelligence in Research (Online Event), December 13, 2021, Berlin Research 50, December 13, 2021.

  • V. Spokoiny, Adaptive graph clustering, Statistics, Artificial Intelligence, Machine Learning, Probability, Learning Theory Event -- SAMPLE, October 26 - 30, 2021, National Research University Higher School of Economics, International Laboratory of Stochastic Algorithms and High-Dimensional Inference, Gelendzhik, Russian Federation, October 27, 2021.

  • V. Spokoiny, Adaptive manifold recovery, Conference Optimization without Borders, July 12 - 18, 2021, Sirius University of Science and Technology, Sochi, Russian Federation, July 15, 2021.

  • V. Spokoiny, Bayesian inference in Bernoulli model with application to ranking from pairwise comparison (online talk), Data Seminar, Université Grenoble Alpes, Laboratoire Jean Kuntzmann, France, January 21, 2021.

  • V. Spokoiny, Inference for nonlinear inverse problems (online talk), Bernoulli-IMS 10th World Congress in Probability and Statistics (Online Event), July 19 - 23, 2021, Seoul National University, Korea (Republic of), July 20, 2021.

  • V. Spokoiny, Random gradient free optimization: Bayesian view, Conference Optimization without Borders, July 12 - 18, 2021, Sirius University of Science and Technology, Sochi, Russian Federation, July 12, 2021.

  • K. Tabelow, MaRDI -- The mathematical research data Initiative within the NFDI (online talk), SFB 1294 Colloquium (Online Event), Universität Potsdam, Institut für Mathematik, April 16, 2021.

External Preprints

  • A. Kroshnin, E. Stepanov, D. Trevisan, Infinite multidimensional scaling for metric measure spaces, Preprint no. arXiv:2201.05885, Cornell University, 2022, DOI 10.48550/arXiv.2201.05885 .

  • V. Laschos, A. Mielke, Evolutionary variational inequalities on the Hellinger--Kantorovich and spherical Hellinger--Kantorovich spaces, Preprint no. arXiv:2207.09815, Cornell University Library, arXiv.org, 2022, DOI 10.48550/arXiv.2207.09815 .
    Abstract
    We study the minimizing movement scheme for some families of geodesically semiconvex functionals defined on either the Hellinger--Kantorovich or the spherical Hellinger--Kantorovich space. By exploiting some of the finer geometric properties of those spaces, we prove that the sequence of curves produced by geodesically interpolating the points generated by the scheme, parameterized by the step size, converges to curves that satisfy the Evolutionary Variational Inequality (EVI), when the step size goes to zero.

  • Y.-W. Sun, K. Papagiannouli, V. Spokoiny, High dimensional change-point detection: A complete graph approach, Preprint no. arXiv:2203.08709, Cornell University, 2022, DOI 10.48550/arXiv.2203.08709 .
    Abstract
    The aim of online change-point detection is for a accurate, timely discovery of structural breaks. As data dimension outgrows the number of data in observation, online detection becomes challenging. Existing methods typically test only the change of mean, which omit the practical aspect of change of variance. We propose a complete graph-based, change-point detection algorithm to detect change of mean and variance from low to high-dimensional online data with a variable scanning window. Inspired by complete graph structure, we introduce graph-spanning ratios to map high-dimensional data into metrics, and then test statistically if a change of mean or change of variance occurs. Theoretical study shows that our approach has the desirable pivotal property and is powerful with prescribed error probabilities. We demonstrate that this framework outperforms other methods in terms of detection power. Our approach has high detection power with small and multiple scanning window, which allows timely detection of change-point in the online setting. Finally, we applied the method to financial data to detect change-points in S&P 500 stocks.

  • M. Alkousa, A. Gasnikov, P. Dvurechensky, A. Sadiev, L. Razouk, An approach for non-convex uniformly concave structured saddle point problem, Preprint no. arXiv:2202.06376, Cornell University, 2022, DOI 10.48550/arXiv.2202.06376 .
    Abstract
    Recently, saddle point problems have received much attention due to their powerful modeling capability for a lot of problems from diverse domains. Applications of these problems occur in many applied areas, such as robust optimization, distributed optimization, game theory, and many applications in machine learning such as empirical risk minimization and generative adversarial networks training. Therefore, many researchers have actively worked on developing numerical methods for solving saddle point problems in many different settings. This paper is devoted to developing a numerical method for solving saddle point problems in the non-convex uniformly-concave setting. We study a general class of saddle point problems with composite structure and Hölder-continuous higher-order derivatives. To solve the problem under consideration, we propose an approach in which we reduce the problem to a combination of two auxiliary optimization problems separately for each group of variables, outer minimization problem w.r.t. primal variables, and inner maximization problem w.r.t the dual variables. For solving the outer minimization problem, we use the Adaptive Gradient Method, which is applicable for non-convex problems and also works with an inexact oracle that is generated by approximately solving the inner problem. For solving the inner maximization problem, we use the Restarted Unified Acceleration Framework, which is a framework that unifies the high-order acceleration methods for minimizing a convex function that has Hölder-continuous higher-order derivatives. Separate complexity bounds are provided for the number of calls to the first-order oracles for the outer minimization problem and higher-order oracles for the inner maximization problem. Moreover, the complexity of the whole proposed approach is then estimated.

  • C. Bellingeri, P.K. Friz, S. Paycha, R. Preiss, Smooth rough paths, their geometry and algebraic renormalization, Preprint no. arXiv:2111.15539, Cornell University, 2022, DOI 10.48550/arXiv.2111.15539 .

  • F. Delarue, W. Salkeld, Probabilistic rough paths II Lions-Taylor expansions and random controlled rough paths, Preprint no. arXiv:2203.01185, Cornell University, 2022, DOI 10.48550/arXiv.2203.01185 .

  • A. Gasnikov, A. Novitskii, V. Novitskii, F. Abdukhakimov, D. Kamzolov, A. Beznosikov, M. Takáč, P. Dvurechensky, B. Gu, The power of first-order smooth optimization for black-box non-smooth problems, Preprint no. arXiv:2201.12289, Cornell University, 2022, DOI 10.48550/arXiv.2201.12289 .
    Abstract
    Gradient-free/zeroth-order methods for black-box convex optimization have been extensively studied in the last decade with the main focus on oracle calls complexity. In this paper, besides the oracle complexity, we focus also on iteration complexity, and propose a generic approach that, based on optimal first-order methods, allows to obtain in a black-box fashion new zeroth-order algorithms for non-smooth convex optimization problems. Our approach not only leads to optimal oracle complexity, but also allows to obtain iteration complexity similar to first-order methods, which, in turn, allows to exploit parallel computations to accelerate the convergence of our algorithms. We also elaborate on extensions for stochastic optimization problems, saddle-point problems, and distributed optimization.

  • S. Mohammadi, T. Streubel, L. Klock, A. Lutti, K. Pine, S. Weber, L. Edwards, P. Scheibe, G. Ziegler, J. Gallinat, S. Kuhn, M. Callaghan, N. Weiskopf, K. Tabelow, Error quantification in multi-parameter mapping facilitates robust estimation and enhanced group level sensitivity, Preprint no. bioRxiv: 2022.01.11.475846, Cold Spring Harbor Laboratory, 2022, DOI 10.1101/2022.01.11.475846 .
    Abstract
    Multi-Parameter Mapping (MPM) is a comprehensive quantitative neuroimaging protocol that enables estimation of four physical parameters (longitudinal and effective transverse relaxation rates R1 and R2*, proton density PD, and magnetization transfer saturation MTsat) that are sensitive to microstructural tissue properties such as iron and myelin content. Their capability to reveal microstructural brain differences, however, is tightly bound to controlling random noise and artefacts (e.g. caused by head motion) in the signal. Here, we introduced a method to estimate the local error of PD, R1, and MTsat maps that captures both noise and artefacts on a routine basis without requiring additional data. To investigate the method's sensitivity to random noise, we calculated the model-based signal-to-noise ratio (mSNR) and showed in measurements and simulations that it correlated linearly with an experimental raw-image-based SNR map. We found that the mSNR varied with MPM protocols, magnetic field strength (3T vs. 7T) and MPM parameters: it halved from PD to R1 and decreased from PD to MT_sat by a factor of 3-4. Exploring the artefact-sensitivity of the error maps, we generated robust MPM parameters using two successive acquisitions of each contrast and the acquisition-specific errors to down-weight erroneous regions. The resulting robust MPM parameters showed reduced variability at the group level as compared to their single-repeat or averaged counterparts. The error and mSNR maps may better inform power-calculations by accounting for local data quality variations across measurements. Code to compute the mSNR maps and robustly combined MPM maps is available in the open-source hMRI toolbox.

  • J.M. Oeschger, K. Tabelow, S. Mohammadi, Axisymmetric diffusion kurtosis imaging with Rician bias correction: A simulation study, Preprint no. bioRxiv2022.03.15.484442, Cold Spring Harbor Laboratory, bioRxiv, 2022, DOI 10.1101/2022.03.15.484442 .

  • CH. Bayer, M. Fukasawa, N. Shonosuke , On the weak convergence rate in the discretization of rough volatility models, Preprint no. arXiv:2203.02943, Cornell University, 2022, DOI 10.48550/arXiv.2203.02943 .

  • CH. Bayer, P.K. Friz, N. Tapia, Stability of deep neural networks via discrete rough paths, Preprint no. arXiv:2201.07566, Cornell University, 2022.

  • P. Dvurechensky, S. Shtern, M. Staudigl, A conditional gradient homotopy method with applications to semidefinite programming, Preprint no. arXiv:2207.03101, Cornell University, 2022, DOI 10.48550/arXiv.2207.03101 .

  • P. Friz, Rough stochastic differential equations, Preprint no. arXiv:2106.10340, Cornell University, 2022, DOI 10.48550/arXiv.2106.10340 .

  • V. Spokoiny, Dimension free non-asymptotic bounds on the accuracy of high dimensional Laplace approximation, Preprint no. arXiv:2204.11038, Cornell University, 2022, DOI arXiv:2204.11038 .
    Abstract
    This note attempts to revisit the classical results on Laplace approximation in a modern non-asymptotic and dimension free form. Such an extension is motivated by applications to high dimensional statistical and optimization problems. The established results provide explicit non-asymptotic bounds on the quality of a Gaussian approximation of the posterior distribution in total variation distance in terms of the so called empheffective dimension ( dimL ). This value is defined as interplay between information contained in the data and in the prior distribution. In the contrary to prominent Bernstein - von Mises results, the impact of the prior is not negligible and it allows to keep the effective dimension small or moderate even if the true parameter dimension is huge or infinite. We also address the issue of using a Gaussian approximation with inexact parameters with the focus on replacing the Maximum a Posteriori (MAP) value by the posterior mean and design the algorithm of Bayesian optimization based on Laplace iterations. The results are specified to the case of nonlinear regression.

  • V. Spokoiny, Finite samples inference and critical dimension for stochastically linear models, Preprint no. arXiv:2201.06327, Cornell University, 2022.
    Abstract
    The aim of this note is to state a couple of general results about the properties of the penalized maximum likelihood estimators (pMLE) and of the posterior distribution for parametric models in a non-asymptotic setup and for possibly large or even infinite parameter dimension. We consider a special class of stochastically linear smooth (SLS) models satisfying two major conditions: the stochastic component of the log-likelihood is linear in the model parameter, while the expected log-likelihood is a smooth function. The main results simplify a lot if the expected log-likelihood is concave. For the pMLE, we establish a number of finite sample bounds about its concentration and large deviations as well as the Fisher and Wilks expansion. The later results extend the classical asymptotic Fisher and Wilks Theorems about the MLE to the non-asymptotic setup with large parameter dimension which can depend on the sample size. For the posterior distribution, our main result states a Gaussian approximation of the posterior which can be viewed as a finite sample analog of the prominent Bernstein--von Mises Theorem. In all bounds, the remainder is given explicitly and can be evaluated in terms of the effective sample size and effective parameter dimension. The results are dimension and coordinate free. In spite of generality, all the presented bounds are nearly sharp and the classical asymptotic results can be obtained as simple corollaries. An interesting case of logit regression with smooth or truncation priors is used to specify the results and to explain the main notions.

  • O. Butkovsky, K. Dareiotis, M. Gerencsér, Optimal rate of convergence for approximations of SPDEs with non-regular drift, Preprint no. arXiv:2110.06148, Cornell University Library, arXiv.org, 2021.

  • O. Butkovsky, V. Margarint, Y. Yuan, Law of the SLE tip, Preprint no. arXiv:2110.11247, Cornell University Library, arXiv.org, 2021.
    Abstract
    We analyse the law of the SLE tip at a fixed time in capacity parametrization. We describe it as the stationary law of a suitable diffusion process, and show that it has a density which is a unique solution of a certain PDE. Moreover, we identify the phases in which the even negative moments of the imaginary value are finite. For the negative second and negative fourth moments we provide closed-form expressions.

  • M. Coghi, W. Dreyer, P. Gajewski, C. Guhlke, P. Friz, M. Maurelli, A McKean--Vlasov SDE and particle system with interaction from reflecting boundaries, Preprint no. 2102.12315v1, Cornell University Library, arXiv.org, 2021.

  • A. Agafonov, P. Dvurechensky, G. Scutari, A. Gasnikov, D. Kamzolov, A. Lukashevich, A. Daneshmand, An accelerated second-order method for distributed stochastic optimization, Preprint no. arXiv:2103.14392, Cornell University Library, arXiv.org, 2021.
    Abstract
    We consider distributed stochastic optimization problems that are solved with master/workers computation architecture. Statistical arguments allow to exploit statistical similarity and approximate this problem by a finite-sum problem, for which we propose an inexact accelerated cubic-regularized Newton's method that achieves lower communication complexity bound for this setting and improves upon existing upper bound. We further exploit this algorithm to obtain convergence rate bounds for the original stochastic optimization problem and compare our bounds with the existing bounds in several regimes when the goal is to minimize the number of communication rounds and increase the parallelization by increasing the number of workers.

  • A. Beznosikov, P. Dvurechensky, A. Koloskova, V. Samokhin, S.U. Stich, A. Gasnikov, Decentralized local stochastic extra-gradient for variational inequalities, Preprint no. arXiv:2106.08315, Cornell University Library, arXiv.org, 2021.
    Abstract
    We consider decentralized stochastic variational inequalities where the problem data is distributed across many participating devices (heterogeneous, or non-IID data setting). We propose a novel method - based on stochastic extra-gradient - where participating devices can communicate over arbitrary, possibly time-varying network topologies. This covers both the fully decentralized optimization setting and the centralized topologies commonly used in Federated Learning. Our method further supports multiple local updates on the workers for reducing the communication frequency between workers. We theoretically analyze the proposed scheme in the strongly monotone, monotone and non-monotone setting. As a special case, our method and analysis apply in particular to decentralized stochastic min-max problems which are being studied with increased interest in Deep Learning. For example, the training objective of Generative Adversarial Networks (GANs) are typically saddle point problems and the decentralized training of GANs has been reported to be extremely challenging. While SOTA techniques rely on either repeated gossip rounds or proximal updates, we alleviate both of these requirements. Experimental results for decentralized GAN demonstrate the effectiveness of our proposed algorithm.

  • A. Daneshmand, G. Scutari, P. Dvurechensky, A. Gasnikov, Newton method over networks is fast up to the statistical precision, Preprint no. arXiv:2102.06780, Cornell University Library, arXiv.org, 2021.

  • E. Gladin, A. Sadiev, A. Gasnikov, P. Dvurechensky, A. Beznosikov, M. Alkousa, Solving smooth min-min and min-max problems by mixed oracle algorithms, Preprint no. arXiv:2103.00434, Cornell University Library, arXiv.org, 2021.
    Abstract
    In this paper, we consider two types of problems that have some similarity in their structure, namely, min-min problems and min-max saddle-point problems. Our approach is based on considering the outer minimization problem as a minimization problem with inexact oracle. This inexact oracle is calculated via inexact solution of the inner problem, which is either minimization or a maximization problem. Our main assumptions are that the problem is smooth and the available oracle is mixed: it is only possible to evaluate the gradient w.r.t. the outer block of variables which corresponds to the outer minimization problem, whereas for the inner problem only zeroth-order oracle is available. To solve the inner problem we use accelerated gradient-free method with zeroth-order oracle. To solve the outer problem we use either inexact variant of Vaydya's cutting-plane method or a variant of accelerated gradient method. As a result, we propose a framework that leads to non-asymptotic complexity bounds for both min-min and min-max problems. Moreover, we estimate separately the number of first- and zeroth-order oracle calls which are sufficient to reach any desired accuracy.

  • E. Gorbunov, M. Danilova, I. Shibaev, P. Dvurechensky, A. Gasnikov, Near-optimal high probability complexity bounds for non-smooth stochastic optimization with heavy-tailed noise, Preprint no. arXiv:2106.05958, Cornell University Library, arXiv.org, 2021.
    Abstract
    Thanks to their practical efficiency and random nature of the data, stochastic first-order methods are standard for training large-scale machine learning models. Random behavior may cause a particular run of an algorithm to result in a highly suboptimal objective value, whereas theoretical guarantees are usually proved for the expectation of the objective value. Thus, it is essential to theoretically guarantee that algorithms provide small objective residual with high probability. Existing methods for non-smooth stochastic convex optimization have complexity bounds with the dependence on the confidence level that is either negative-power or logarithmic but under an additional assumption of sub-Gaussian (light-tailed) noise distribution that may not hold in practice, e.g., in several NLP tasks. In our paper, we resolve this issue and derive the first high-probability convergence results with logarithmic dependence on the confidence level for non-smooth convex stochastic optimization problems with non-sub-Gaussian (heavy-tailed) noise. To derive our results, we propose novel stepsize rules for two stochastic methods with gradient clipping. Moreover, our analysis works for generalized smooth objectives with Hölder-continuous gradients, and for both methods, we provide an extension for strongly convex problems. Finally, our results imply that the first (accelerated) method we consider also has optimal iteration and oracle complexity in all the regimes, and the second one is optimal in the non-smooth setting.

  • A. Kroshnin, V. Spokoiny, A. Suvorikova, Multiplier bootstrap for Bures--Wasserstein barycenters, Preprint no. arXiv:2111.12612, Cornell University Library, arXiv.org, 2021.
    Abstract
    Bures-Wasserstein barycenter is a popular and promising tool in analysis of complex data like graphs, images etc. In many applications the input data are random with an unknown distribution, and uncertainty quantification becomes a crucial issue. This paper offers an approach based on multiplier bootstrap to quantify the error of approximating the true Bures--Wasserstein barycenter Q? by its empirical counterpart Qn. The main results state the bootstrap validity under general assumptions on the data generating distribution P and specifies the approximation rates for the case of sub-exponential P. The performance of the method is illustrated on synthetic data generated from the weighted stochastic block model.

  • V. Novitskii, A. Gasnikov, Improved exploiting higher order smoothness in derivative-free optimization and continuous bandit, Preprint no. arXiv:2101.03821, Cornell University Library, arXiv.org, 2021.

  • D. Pasechnyuk, P. Dvurechensky, S. Omelchenko, A. Gasnikov, Stochastic optimization for dynamic pricing, Preprint no. arXiv:2106.14090, Cornell University Library, arXiv.org, 2021.
    Abstract
    We consider the problem of supply and demand balancing that is stated as a minimization problem for the total expected revenue function describing the behavior of both consumers and suppliers. In the considered market model we assume that consumers follow the discrete choice demand model, while suppliers are equipped with some quantity adjustment costs. The resulting optimization problem is smooth and convex making it amenable for application of efficient optimization algorithms with the aim of automatically setting prices for online marketplaces. We propose to use stochastic gradient methods to solve the above problem. We interpret the stochastic oracle as a response to the behavior of a random market participant, consumer or supplier. This allows us to interpret the considered algorithms and describe a suitable behavior of consumers and suppliers that leads to fast convergence to the equilibrium in a close to the real marketplace environment.

  • A. Rogozin, A. Beznosikov, D. Dvinskikh, D. Kovalev, P. Dvurechensky, A. Gasnikov, Decentralized distributed optimization for saddle point problems, Preprint no. arXiv:2102.07758, Cornell University Library, arXiv.org, 2021.

  • A. Rogozin, M. Bochko, P. Dvurechensky, A. Gasnikov, V. Lukoshkin, An accelerated method for decentralized distributed stochastic optimization over time-varying graphs, Preprint no. arXiv:2103.15598, Cornell University Library, arXiv.org, 2021.
    Abstract
    We consider a distributed stochastic optimization problem that is solved by a decentralized network of agents with only local communication between neighboring agents. The goal of the whole system is to minimize a global objective function given as a sum of local objectives held by each agent. Each local objective is defined as an expectation of a convex smooth random function and the agent is allowed to sample stochastic gradients for this function. For this setting we propose the first accelerated (in the sense of Nesterov's acceleration) method that simultaneously attains optimal up to a logarithmic factor communication and oracle complexity bounds for smooth strongly convex distributed stochastic optimization. We also consider the case when the communication graph is allowed to vary with time and obtain complexity bounds for our algorithm, which are the first upper complexity bounds for this setting in the literature.

  • V. Tominin , Y. Tominin , E. Borodich , D. Kovalev, A. Gasnikov, P. Dvurechensky, On accelerated methods for saddle-point problems with composite structure, Preprint no. arXiv:2103.09344, Cornell University Library, arXiv.org, 2021.
    Abstract
    We consider strongly-convex-strongly-concave saddle-point problems with general non-bilinear objective and different condition numbers with respect to the primal and the dual variables. First, we consider such problems with smooth composite terms, one of which having finite-sum structure. For this setting we propose a variance reduction algorithm with complexity estimates superior to the existing bounds in the literature. Second, we consider finite-sum saddle-point problems with composite terms and propose several algorithms depending on the properties of the composite terms. When the composite terms are smooth we obtain better complexity bounds than the ones in the literature, including the bounds of a recently proposed nearly-optimal algorithms which do not consider the composite structure of the problem. If the composite terms are prox-friendly, we propose a variance reduction algorithm that, on the one hand, is accelerated compared to existing variance reduction algorithms and, on the other hand, provides in the composite setting similar complexity bounds to the nearly-optimal algorithm which is designed for non-composite setting. Besides that, our algorithms allow to separate the complexity bounds, i.e. estimate, for each part of the objective separately, the number of oracle calls that is sufficient to achieve a given accuracy. This is important since different parts can have different arithmetic complexity of the oracle, and it is desired to call expensive oracles less often than cheap oracles. The key thing to all these results is our general framework for saddle-point problems, which may be of independent interest. This framework, in turn is based on our proposed Accelerated Meta-Algorithm for composite optimization with probabilistic inexact oracles and probabilistic inexactness in the proximal mapping, which may be of independent interest as well.

  • P. Dvurechensky, D. Kamzolov, A. Lukashevich, S. Lee, E. Ordentlich, C.A. Uribe, A. Gasnikov, Hyperfast second-order local solvers for efficient statistically preconditioned distributed optimization, Preprint no. arXiv:2102.08246, Cornell University Library, arXiv.org, 2021.

  • P. Dvurechensky, M. Staudigl, S. Shtern, First-order methods for convex optimization, Preprint no. arXiv:2101.00935, Cornell University Library, arXiv.org, 2021.
    Abstract
    First-order methods for solving convex optimization problems have been at the forefront of mathematical optimization in the last 20 years. The rapid development of this important class of algorithms is motivated by the success stories reported in various applications, including most importantly machine learning, signal processing, imaging and control theory. First-order methods have the potential to provide low accuracy solutions at low computational complexity which makes them an attractive set of tools in large-scale optimization problems. In this survey we cover a number of key developments in gradient-based optimization methods. This includes non-Euclidean extensions of the classical proximal gradient method, and its accelerated versions. Additionally we survey recent developments within the class of projection-free methods, and proximal versions of primal-dual schemes. We give complete proofs for various key results, and highlight the unifying aspects of several optimization algorithms.

  • P. Dvurechensky, M. Staudigl, Hessian barrier algorithms for non-convex conic optimization, Preprint no. arXiv:2111.00100, Cornell University Library, arXiv.org, 2021.
    Abstract
    We consider the minimization of a continuous function over the intersection of a regular cone with an affine set via a new class of adaptive first- and second-order optimization methods, building on the Hessian-barrier techniques introduced in [Bomze, Mertikopoulos, Schachinger, and Staudigl, Hessian barrier algorithms for linearly constrained optimization problems, SIAM Journal on Optimization, 2019]. Our approach is based on a potential-reduction mechanism and attains a suitably defined class of approximate first- or second-order KKT points with the optimal worst-case iteration complexity O(??2) (first-order) and O(??3/2) (second-order), respectively. A key feature of our methodology is the use of self-concordant barrier functions to construct strictly feasible iterates via a disciplined decomposition approach and without sacrificing on the iteration complexity of the method. To the best of our knowledge, this work is the first which achieves these worst-case complexity bounds under such weak conditions for general conic constrained optimization problems.