This paper provides a theoretical and experimental comparison between conjugate-gradients and multigrid, two iterative schemes for solving linear systems, in the context of applying diffusion-based correlation models in data assimilation. In this context, a large number of such systems has to be (approximately) solved if the implicit mode is chosen for integrating the involved diffusion equation over pseudo-time, thereby making their efficient handling crucial for practical performance. It is shown that the multigrid approach has a significant advantage, especially for larger correlation lengths and/or large problem sizes.