From a philosophical point of view, I think the most important point of confusion about Bayesian inference is the idea that it's about computing the probability that a model is true. In all the areas I've ever worked on, the model is never true. But what you can do is find out that certain important aspects of the data are highly unlikely to be captured by the fitted model, which can facilitate a "model shift" moment. This sort of falsification is why I believe Popper's philosophy of science to be a good fit to Bayesian data analysis.
Saturday, February 14, 2009
Gelman on statistics
I'm going to start linking to interesting blog posts by Andrew Gelman because they often have important points to remember. Today he talks about a couple of reviews of "The Black Swan" and includes the following note on Bayesian inference :