This submission deals with a fundamental problems in ML and presents an interesting approach to permutation invariant learning. The ideas of this work are interesting despite the weak baseline used in the experiments. The authors are requested to include stronger baselines in the final version.