We combine the replica approach from statistical physics with a varia- tional approach to analyze learning curves analytically. We apply the method to Gaussian process regression. As a main result we derive ap- proximative relations between empirical error measures, the generaliza- tion error and the posterior variance.