# Process Based Theory

Topics: Bayes' theorem, Statistical inference, Bayesian inference Pages: 36 (8774 words) Published: March 8, 2013
Bayesian Inference: An Introduction to Principles and Practice in Machine Learning Michael E. Tipping
Microsoft Research, Cambridge, U.K.

.................................................................... Published as: “Bayesian inference: An introduction to Principles and practice in machine learning.” In O. Bousquet, U. von Luxburg, and G. R¨tsch (Eds.), Advanced Lectures on a Machine Learning, pp. 41–62. Springer. 2004 June 26, 2006 http://www.miketipping.com/papers.htm mail@miketipping.com

Year of publication: This version typeset: Available from: Correspondence:

Abstract

This article gives a basic introduction to the principles of Bayesian inference in a machine learning context, with an emphasis on the importance of marginalisation for dealing with uncertainty. We begin by illustrating concepts via a simple regression task before relating ideas to practical, contemporary, techniques with a description of ‘sparse Bayesian’ models and the ‘relevance vector machine’.

1

Introduction

Bayesian Inference: Principles and Practice in Machine Learning

2

It is in the modelling procedure where Bayesian inference comes to the fore. We typically (though not exclusively) deploy some form of parameterised model for our conditional probability: P (B|A) = f (A; w), (1)

where w denotes a vector of all the ‘adjustable’ parameters in the model. Then, given a set D of N examples of our variables, D = {An , Bn }N , a conventional approach would involve the n=1 maximisation of some measure of ‘accuracy’ (or minimisation of some measure of ‘loss’) of our model for D with respect to the adjustable parameters. We then can make predictions, given A, for unknown B by evaluating f (A; w) with parameters w set to their optimal values. Of...