NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:1069
Title:The Cells Out of Sample (COOS) dataset and benchmarks for measuring out-of-sample generalization of image classifiers

The main strength of this paper is that it proposes an interesting new dataset of biological images, which has the potential to become an interesting benchmark particularly for problems with realistic "covariate shift" problems. The problem of generalization to the same task with differences in data acquisition protocols is a big issue in empirical sciences - especially in genomics and radiology. Changes in acquisition protocol, software updates or differences in hardware can throw off a well-performing classifier. This dataset will help tackle this problem. There was a consensus among reviewers that creating such a benchmark (and testing the performance of baseline methods) is an important effort that would be of interest to the NeurIPS community. The relevance of the benchmark to solve an important real-world problem (prediction of protein localization) was less evident.