This work was debated controversially among the reviewers. They all agreed that the work was presented well, and both the idea of the paper and how it is realised as a software interface are novel (or at least a clear improvement over existing frameworks). Software packages generally struggle to get accepted at major conferences. The discussion between the reviewers hinged primarily on this point, too. I would thus like to throw my own vote in for this paper. It is true that the community has not yet developed a good and consistent way to evaluate software contributions, in particular vis-a-vis theoretical and empirical papers. But it is high time that our community becomes more professional in software development. There is an abundance of papers at NeurIPS that provide insufficient or even no implementation details, and this leads to a shocking amount of published work never getting reproduced, used or even widely discussed. As the field matures, we need much more software development in ML. But software design is a thankless, unappreciated task in the academic community. It is time-consuming, and often met with *more* scrutiny by reviewers precisely because it is more accessible, and it's always easy to find something to criticise about a software toolkit. But good software toolkits are among the most impactful works of the past years in this community! (We wouldn't have had the deep learning boom without software toolkits). And not all software can and will be developed by commercial players. There are software tasks that just don't have an obvious business case. Examples include software for benchmarking and comparison of existing methods, and novel general functionality that does not address a specific application domain. I think the present paper fits into this domain, even if it is perhaps not the best example. The community indeed urgently needs to develop fair and transparent standards for how software contributions should be evaluated and (most importantly) appreciated. But it is not the present paper's fault that this has not yet happened, so this problem should not be held against it. It thus recommend that this work should be accepted. Having said all this, I also strongly encourage the authors to clean up the presentation of this paper. Right now, the layout is not clean and at times hard to read. Please take all the criticism of the reviewers into account when preparing the camera-ready version to ensure this paper actually reaches the NeurIPS audience.