The authors propose the high-order Polynomial Tensor Pooling (PTP) block, which can fuse locally mixed temporal features. A concrete neural architecture utilizing the PTP block as the main component shows good performance. To avoid the exponential parameter explosion of tensor product representations, the authors present a novel technique that does not restrict the order of feature interaction. Although there are many details that can be improved, including a more rigorous evaluation, the significance and originality of the work makes this a promising paper for NeurIPS.