WIAS Preprint No. 2981, (2022)

Tensor-train kernel learning for Gaussian processes



Authors

  • Kirstein, Max
  • Eigel, Martin
    ORCID: 0000-0003-2687-4497
  • Sommer, David
    ORCID: 0000-0002-6797-8009

2020 Mathematics Subject Classification

  • 15A69 60G15 65D15

Keywords

  • Tensor networks, tensor-train decomposition, Gaussian processes

DOI

10.20347/WIAS.PREPRINT.2981

Abstract

We propose a new kernel learning approach based on efficient low-rank tensor compression for Gaussian process (GP) regression. The central idea is to compose a low-rank function represented in a hierarchical tensor format with a GP covariance function. Compared to similar deep neural network architectures, this approach facilitates to learn significantly more expressive features at lower computational costs as illustrated in the examples. Additionally, over-fitting is avoided with this compositional model by taking advantage of its inherent regularisation properties. Estimates of the generalisation error are compared to five baseline models on three synthetic and six real-world data sets. The experimental results show that the incorporated tensor network enables a highly accurate GP regression with a comparatively low number of trainable parameters. The observed performance is clearly superior (usually by an order of magnitude in mean squared error) to all examined standard models, in particular to deep neural networks with more than 1000 times as many parameters.

Appeared in

  • Proceedings of the Eleventh Symposium on Conformal and Probabilistic Prediction with Applications, U. Johansson, H. Boström, K.A. Nguyen, Z. Luo, L. Carlsson, eds., vol. 179 of Proceedings of Machine Learning Research, 2022, pp. 253--272.

Download Documents