Description
Surrogate models for the turbulent transport model TGLF have been developed based on Gaussian process (GP) regression using stochastic variational Gaussian processes (SVGP) with the intrinsic coregionalization model (ICM), implemented in our in-house GP regression library dgpr [1]. The SVGP framework enables scalable GP regression for large datasets by introducing inducing points, while the ICM captures correlations among multiple output channels. In Ref. [1], it was demonstrated that GP-based surrogate models integrated into the steady-state transport code GOTRESS successfully reproduce the temperature profiles predicted by TGLF. Fast and accurate surrogate models are essential for integrated modeling, and GP regression can achieve competitive accuracy with smaller training datasets compared to neural network (NN) surrogates while additionally providing uncertainty estimates.
In this work, the regression performance is further improved by incorporating Deep Kernel Learning (DKL) [2,3] into the SVGP framework. DKL combines the structural properties of deep NNs with the non-parametric flexibility of kernel methods. A deep NN serves as a feature extractor that maps the original high-dimensional input into a learned low-dimensional feature space, on which a base GP kernel operates. All NN and GP hyperparameters are jointly optimized through the GP marginal likelihood, allowing the model to discover expressive representations while retaining principled uncertainty quantification.
DKL has been implemented in dgpr and combined with the Matern-5/2 kernel and SVGP with ICM. The results on the TGLF surrogate modeling task show that DKL substantially improves regression accuracy: the coefficient of determination ($R^2$) for the electron heat flux $Q_e$ improved from 0.964 with the standard Matern-5/2 kernel to 0.988 with DKL, and for the ion heat flux $Q_i$ from 0.949 to 0.984. Details of the DKL implementation will be presented, along with a discussion of the dependence on the NN architecture and robustness across different test datasets.
[1] M. Honda, S. Maeyama, and E. Narita, Phys. Plasmas 32 (2025) 103906.
[2] A.G. Wilson et al., Proc. 19th Int. Conf. Artif. Intell. Stat. (2016) 370.
[3] A.G. Wilson et al., Adv. Neural Inf. Process. Syst. 29 (2016) 2586.