Modelling uncertainty

We have developed a stream of pioneering work in the area of inference in Gaussian process (GP) models, ranging from regression, classification and multi-task problems to generic inference with black-box likelihoods. We have published this work at the very top ML venues. Our techniques are mainly underpinned by variational inference principles (optimizing rather than integrating). These efforts have been materialized recently in the development of one of the most flexible and scalable methods for inference in GP models (Rossi et al., 2021), which uses stochastic gradient Hamiltonian Monte Carlo (SGHMC) along with variational inference techniques. New directions pursue the generalization of such techniques to time-evolving GPs and our most recent work on Gaussian process state-space models that has been published at ICML (Fan et al., 2023).

More generally, we aim to address other problems in inference in graphical models (not necessarily using GP priors), and, e.g., understanding the theoretical properties of neural networks in frameworks such as state-space models (Bishop & Bonilla, 2023). Perhaps more importantly, when considering uncertainty in the graphical model itself (i.e., how to model distributions over directed acyclic graphs and how to estimate posteriors over them). This will have crucial implications across many areas, especially, in causality research, see, e.g., VDESP (Bonilla et al., 2024) and ProjDAG (Thompson et al., 2024).

Recent Developments

Inference in Gaussian Process Models

• Bayesian scalable GPs (Rossi et al., 2021)
• Inference in GP state-space models (Fan et al., 2023)

Bayesian Approaches to Directed Acyclic Graph (DAG) Estimation and Graph Neural Networks

• Bayesian DAG estimation via permutation-based distributions (Bonilla et al., 2024)
• Bayesian DAG estimation via Projection-induced distributions (Thompson et al., 2024)
• Bayesian Granger Causality (Zhao & Bonilla, 2024)
• Variational graph convolutional networks (Elinas et al., 2020)