This paper studies the problem of cross lingual language model pretraining. Pros • An important problem is studied. • Paper is well written. • Very strong empirical results. • The methodology is solid. • Comprehensive review of related work. Cons • The proposed methods are not particularly novel. All the reviewers liked the paper.