The paper studies representation learning for RL through a notion of adversarial value functions. This provides a new conceptual/theoretical understanding of representation learning for RL that all reviewers felt was interesting. The experimental results are somewhat preliminary, but sufficient.