NeurIPS 2020

On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law

Meta Review

All reviewers recommend the submission for acceptance and I agree. This work provided a detailed analysis of work on the VQA under Changing Priors problem -- pointing out a number of systematic deficiencies and providing a baseline that acutely demonstrates these shortcomings. The audience for this work is likely narrow but it will be important to that group.