Sampling variation in effective sample size estimates (MCMC)

By: Julia on Tamás K. Papp's website

Re-posted from: http://tpapp.github.io/post/ess-sampling/

Introduction MCMC samples, used in Bayesian statistics, are not independent — in fact, unless one uses specialized methods or modern HMC, posterior draws are usually at highly autocorrelated. For independent draws, [ \text{variance of simulation mean} \propto \frac1N ] where $N$ is the sample size, but for correlated draws, one has to scale the sample size with a factor [ \tau = \frac{1}{1+2\sum_{k=1}^\infty \rho_k} ] where $\rho_k$ is the lag-$k$ autocorrelation.