3.3.2 Bayesian Linear Regression: Predictive Distribution - Pattern Recognition and Machine Learning

In this section, we show how to combine the uncertainty in our parameters with our model of how these parameters produce outputs to derive the predictive distribution over the output variable for new input. When our uncertainty over the parameters is Gaussian and our observations are linear in the parameters plus Gaussian noise, we can work out the predictive distribution analytically. This reveals the mean of the predictions to be the projection of the input features onto the mean value of the parameters, and the variance to be the covariance of the parameters projected onto the input features, plus the noise variance. As the number of observations increases, the uncertainty in the parameters decreases leaving only the noise term. Finally, we discuss the problem of how the predicted variance can be artificially low when inputs lie outside the support of the basis functions.