NeurIPS 2020

Matrix Inference and Estimation in Multi-Layer Models


Meta Review

Out of the four reviews, three are above the acceptance threshold with higher confidence, and the remaining one is below the threshold, although with a weaker confidence. Arguably, the major weaknesses of this paper are: It is somehow incremental, extending ML-VAMP to ML-Mat-VAMP. Also, the numerical example dealt with in Section 5, where F_1 is generated as having iid entries, does not really demonstrate the advantage of ML-Mat-VAMP over ML-VAMP. As for the former point, Reviewers #1 and #4 state that it is still non-trivial. As for the latter, the authors in their response promised to provide results of cases with correlated features. Assuming this to be addressed in the final version, I would like to recommend acceptance of this paper. Upon my own reading of this paper, I noticed that there are several errors in the manuscript. The following is a list of those which I found and no reviewer commented. I would appreciate it if the authors take them into account in preparing the final version. - First of all, the assumption that L is an even number should be stated explicitly. - Equation (3): The fact that one takes argmin of H_L+H_0 with respect to Z_0 should be made explicit. The current expression may read as if one should take argmin of H_L with respect to Z_{L-1}. - Lin 142: The symbol \mathbb{Z}_{L-1} is undefined. (The definition may be \mathbb{Z}_{L-1}={\mathbf{Z}_l}_{l=0}^{L-1}, but there is a distinct symbol \mathbf{Z} introduced in line 165 for the same purpose.) - Line 150: Commas needed: (\mathbf{Z}_{L-1}, \cdots, \mathbf{Z}_0) - Line 156: \mathbf{Z} is undefined at this point. The definition is given in line 165. - Equation (11): \prod_{l=1}^{L-1} might read \prod_{l=1}^{L-2}. - Line 185: the belief density density - Line 195: ML-MAT-VAMP -> ML-Mat-VAMP - Line 206: The period at the end of the sentence is missing. - Line 207: The distribution of the remaining variables (are -> is) - Line 219: satisfy -> satisfies; convergence pointwise -> pointwise convergence - Line 229: requires only require - Line 237: approach to -> approach; G^+(\cdot) and G^+(\cdot) -> G^-(\cdot) and G^+(\cdot) - Line 238: exact an analysis -> an exact analysis - Displayed equation after line 241: \mathbf{G}_l^+ -> G_l^+; \Theta_{kl} undefined. - Line 249: The abbreviation LSL (=large system limit?) undefined. - Line 278: The section number 6 is missing. Other points I would like to mention are: - I guess that the ML-Mat-VAMP and its SE equations reduce to the ML-VAMP and its SE equations, respectively, if specializing the former with d=1. If it is the case then it should be stated explicitly. - In the experimental results it seems that the test errors are not monotonic in the number of training samples. Although I guess that it is due to statistical fluctuation originated from the Monte-Carlo evaluation, as mentioned in line 260, I would appreciate it if the authors improve the evaluation to show more stable results in the final manuscript.