This paper generated some disagreement among reviewers. The main strength is the novel technical contribution which can inspire others along similar directions. The main weakness is the lack of experimental evaluations that were suggested by the reviewers (additional datasets, baselines, and ablations). We hope that these can be included for the final revision.