Diese Seite auf Deutsch

Publications

Articles in Refereed Journals

  • H.T. Chu, L. Liang, K.-Ch. Toh, L. Yang, An efficient implementable inexact entropic proximal point algorithm for a class of linear programming problems, Computational Materials Science, 85 (2023), pp. 107--146, DOI 10.1007/s10589-023-00459-2 .

Contributions to Collected Editions

  • D. Agudelo-España, Y. Nemmour, B. Schölkopf, J.-J. Zhu, Learning random feature dynamics for uncertainty quantification, in: 2022 IEEE 61th Conference on Decision and Control (CDC), Cancun, Mexico, IEEE, 2022, pp. 4937--4944, DOI 10.1109/CDC51059.2022.9993152 .

  • H. Kremer, J.-J. Zhu, K. Muandet, B. Schölkopf, Functional generalized empirical likelihood estimation for conditional moment restrictions, in: Proceedings of the 39th International Conference on Machine Learning, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, S. Sabato, eds., 162 of Proceedings of Machine Learning Research, 2022, pp. 11665--11682.
    Abstract
    Important problems in causal inference, economics, and, more generally, robust machine learning can be expressed as conditional moment restrictions, but estimation becomes challenging as it requires solving a continuum of unconditional moment restrictions. Previous works addressed this problem by extending the generalized method of moments (GMM) to continuum moment restrictions. In contrast, generalized empirical likelihood (GEL) provides a more general framework and has been shown to enjoy favorable small-sample properties compared to GMM-based estimators. To benefit from recent developments in machine learning, we provide a functional reformulation of GEL in which arbitrary models can be leveraged. Motivated by a dual formulation of the resulting infinite dimensional optimization problem, we devise a practical method and explore its asymptotic properties. Finally, we provide kernel- and neural network-based implementations of the estimator, which achieve state-of-the-art empirical performance on two conditional moment restriction problems.

  • Y. Nemmour, H. Kremer, B. Schölkopf, J.-J. Zhu, Maximum mean discrepancy distributionally robust nonlinear chance-constrained optimization with finite-sample guarantee, in: 2022 IEEE 61st Conference on Decision and Control (CDC), Cancun, Mexico, 2022, pp. 5660--5667, DOI 10.1109/CDC51059.2022.9993212 .

  • J.-J. Zhu, Ch. Kouridi, Y. Nemmour, B. Schölkopf, Adversarially robust kernel smoothing, in: Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, G. Camps-Valls, F.J.R. Ruiz, I. Valera, eds., 151 of Proceedings of Machine Learning Research, 2022, pp. 4972--4994.

Preprints, Reports, Technical Reports

  • P. Dvurechensky, J.-J. Zhu, Kernel mirror prox and RKHS gradient flow for mixed functional Nash equilibrium, Preprint no. 3032, WIAS, Berlin, 2023, DOI 10.20347/WIAS.PREPRINT.3032 .
    Abstract, PDF (436 kByte)
    Kernel mirror prox and RKHS gradient flow for mixed functional Nash equilibrium Pavel Dvurechensky , Jia-Jie Zhu Abstract The theoretical analysis of machine learning algorithms, such as deep generative modeling, motivates multiple recent works on the Mixed Nash Equilibrium (MNE) problem. Different from MNE, this paper formulates the Mixed Functional Nash Equilibrium (MFNE), which replaces one of the measure optimization problems with optimization over a class of dual functions, e.g., the reproducing kernel Hilbert space (RKHS) in the case of Mixed Kernel Nash Equilibrium (MKNE). We show that our MFNE and MKNE framework form the backbones that govern several existing machine learning algorithms, such as implicit generative models, distributionally robust optimization (DRO), and Wasserstein barycenters. To model the infinite-dimensional continuous- limit optimization dynamics, we propose the Interacting Wasserstein-Kernel Gradient Flow, which includes the RKHS flow that is much less common than the Wasserstein gradient flow but enjoys a much simpler convexity structure. Time-discretizing this gradient flow, we propose a primal-dual kernel mirror prox algorithm, which alternates between a dual step in the RKHS, and a primal step in the space of probability measures. We then provide the first unified convergence analysis of our algorithm for this class of MKNE problems, which establishes a convergence rate of O(1/N ) in the deterministic case and O(1/√N) in the stochastic case. As a case study, we apply our analysis to DRO, providing the first primal-dual convergence analysis for DRO with probability-metric constraints.

Talks, Poster

  • L. Liang, A squared smoothing Newton method for semidefinite programming, SIAM Conference on Optimization (OP23), May 30 - June 3, 2023, Society for Industrial and Applied Mathematics, Seattle, USA, June 3, 2023.

  • J.J. Zhu, From gradient flow to distributionally robust optimization, Seminar of the Computer Science Department, University of British Columbia, Computer Science Department, Vancouver, Canada, June 5, 2023.

  • J.-J. Zhu, Optimization and dynamics: From Euclidean gradient descent to Wasserstein gradient flow, International Workshop of Intelligent Autonomous Learning Systems 2023, August 14 - 17, 2023, TU Darmstadt, Intelligent Autonomous Systems, Computer Science Department, August 15, 2023.

  • J.-J. Zhu, Principled robust machine learning in new geometries, Leibniz MMS Days 2023, April 17 - 19, 2023, Leibniz-Institut für Agrartechnik und Bioökonomie (ATB), Potsdam, April 17, 2023.

  • A. Pavlov, Bilevel Interior-point Differential Dynamic Programming, EUROPT2022 19th Workshop on Advances in Continuous Optimization, NOVA School of Science and Technology, Universidade Nova de Lisboa, Portugal, July 29, 2022.

  • H. Kremer, J.-J. Zhu, K. Muandet, B. Schölkopf, Functional generalized empirical likelihood estimation for conditional moment restrictions (spotlight, online talk), ICML 2022: 39th International Conference on Machine Learning (Online Event), July 18 - 23, 2022, Baltimore, USA, July 19, 2022.

  • J.-J. Zhu, F. Nüske, Data-Driven Modeling and Optimization of Dynamical Systems under Uncertainty (Ph.D. 16-hour minicourse), IRTG 2544 Stochastic Analysis in Interaction, July 11 - 14, 2022, Technische Universität Berlin.

  • J.-J. Zhu, Distributionally robust learning and optimization in the MMD geometry and beyond, Eurandom YES Workshop - Optimal Transport, Statistics, Machine Learning and moving in between, September 5 - 9, 2022, Eindhoven University of Technology, Netherlands, September 8, 2022.

  • J.-J. Zhu, Maximum Mean Discrepancy Distributionally Robust Nonlinear Chance-Constrained Program with Statistical Guarantee, ESPOO EURO 2022, Aalto University, Finland, July 3, 2022.

  • J.-J. Zhu, Distributionally robust learning and optimization in MMD geometry, KU Leuven, STADIUS Center for Dynamical Systems, Signal Processing, and Data, Belgium, September 9, 2022.

  • J.-J. Zhu, Kernel methods for distributionally robust machine learning and optimization, Vrije Universiteit Amsterdam, Department of Operations Analytics, Netherlands, July 28, 2022.

External Preprints

  • L. Liang, D. Sun, K.-Ch. Toh, A squared smoothing Newton method for semidefinite programming, Preprint no. arXiv:2303.05825, Cornell University, 2023, DOI 10.48550/arXiv.2303.05825 .

  • Z. Zhong, J.-J. Zhu, Nonlinear Wasserstein distributionally robust optimal control, Preprint no. arXiv:2304.07415, Cornell University, 2023, DOI 10.48550/arXiv.2304.07415 .

  • J.-J. Zhu, Propagating kernel ambiguity sets in nonlinear data-driven dynamics models, Preprint no. arXiv:2304.14057, Cornell University, 2023, DOI 10.48550/arXiv.2304.14057 .

  • Y. Nemmour, H. Kremer, B. Schölkopf, J.-J. Zhu, Maximum mean discrepancy distributionally robust nonlinear chance-constrained optimization with finite-sample guarantee, Preprint no. arXiv:2204.11564, Cornell University, 2022, DOI 10.48550/arXiv.2204.11564 .

Flexible Research Platform