Nial Friel and Jason Wyse have just posted on Arxiv a very useful and clear review of techniques for estimating the evidence in Bayesian modelling. It’s hard to draw a very general conclusion from the comparison they make, except:
- using the Harmonic Mean Estimator seems like a really bad idea,
- the general problem is not easy and you probably are going to end up spending quite a bit of time until you find something that works well for your particular model
Missing from the review are Expectation Propagation, which often produces very good estimates of the evidence (although no one knows why) and the various variational techniques that give lower bounds on the evidence.