After much discussion and follow up questions to the authors, the reviewers converged towards recommending to accept this submission. The reviewers were satisfied with the authors' response, and have updated their reviews accordingly. There are some remaining points about claims made in the paper which need to be toned down (single layer vs deep models), and the conclusions from empirical validation only supporting claims with small low dim data (while the effect reverses with the full 11000 datapoints in AL). I recommend acceptance and trust that the authors will address these remaining points for the camera ready.