NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:2060
Title:Verified Uncertainty Calibration

Congratulations, your paper has been accepted to NeurIPS2019. The reviewers found it to be original, important, and well written. When preparing the camera ready version, please bear in mind the reviewers comments. In particular, the following were raised during the discussion as outstanding points that should be considered when revising the paper: - discussion of the idea of a calibration budget - this is important because, whereas the new approach might improve calibration relative to traditional scaling, it tends to worsen sharpness (because overall performance is slightly degraded - appendix D.2). Without a discussion of the relative importance of overall performance, calibration and sharpness, and without highlighting the impact of the new approach on *all* of these attributes (compared to traditional scaling), readers might get the impression that there is little downside to the new approach. - Include the clarification from your response of the way in which theorem 4.1 should be understood. - The effect of delta on the sample complexity is still a bit unclear - can this be made clearer? - Include the information clarifying the asymptotics notation from your response. The follow on line 178 I now understand the subtlety and would recommend either clarifying or not attempting to push this as an important claim. - Include the new experimental results requested by R1 and R2 (that were in your response), as they support the conclusions. - There was a feeling that the name given to your method is misleading - is it worth finding a more appropriate name for your approach?