This paper proposes an approach for discrete latent variable models that uses sparsity in order to explicitly marginalize. The idea in this paper is simple, elegant, and a refreshing departure from the typical relaxation / reparameterization approaches to this problem. Please include the clarifications requested by the reviewers.