The proposed method to model preplay and rapid path planning using successor representation is a very interesting and potentially important development. It is also in my opinion a more realistic implementation of model-based RL than tree search-based alternatives.

However, a few important questions remain: - is computation of advanced quantities as described in sections 2 and 3 biologically realistic - not just from the perspective of the resulting output and dynamics, but the mechanistic implementation (e.g. how weights that are derived here using least-squares learned)? - how does such successor-based representation work in more realistic scenarios when familiarity with the environment is partial and how does it change with learning - can it reproduce typically observed behavioral learning curves? - can it also be applied to other domains than space, e.g. time cells? I'm aware that addressing these questions in detail may be out of the scope of this paper, but it would be helpful if these aspects could at least be discussed briefly.

Clarity of the paper, particularly of the mathematical expressions and derivations, could be improved, e.g. line 2 of eq.3 or why a weak stimulus corresponds to alpha = epsilon = 0.05 explained in more detail.

It is an important study linking different computational ideas such as successor representation and slow feature analysis to recent experimental data of sequence preplay and rapid path planning.

Technical comments:

- Equation 2, should be -d^2, not -d? ...Otherwise it's not a gaussian bump

- Your comment on line 119 that the normalized metric can be interpreted as a steady state probaility distribution is not strictly correct.

The probability of the steady state is proportional to the number of random walks that end in that state, which is related to, but not quite the same as the shortest distance from the initial state.

If you meant this in a loose way, then I'm okay with but you should make it clear that it is an approximation.

This paper uses a simple model of CA3 hippocampus to perform planning through a maze task.

They do this by using a successor representation of the maze and initializing the netwrk in a starting location.

By providing a small constant input of the goal state the network representation moves towards the goal state, even if the goal is many steps away, and it takes the shortest path respecting any obstacles that might be in the way.

I like the task - and feel strongly that an easily updateable value map is something that is needed in the field of path planning.

I recommend that this paper be included in the conference.

Although I would have liked to see the limits of this method.

e.g. how complex a maze can be learnt and decoded with this method, before the method breaks.

The paper studied an important question, that is, how does the brain achieve rapid path integration in reality? The authors first presented a mathematical framework, the successor representation, for path-finding, and then demonstrated how this can be implemented in practice by using an artificial bump-like attractor network.

Mathematically, this is a quite interesting work. My main concern is about its biological relevance, although the authors argued that the involved complicated mathematical operations, such as extracting the dominant eigenvectors, can be implemented in practice.

The paper proposed a mathematical model for rapid path intergration based on attractor network dynamics.

The paper introduces a novel potential candidate for a model of place-cell firing in hippocampus that implements (or contributes to) path finding in maze-like environments based on a spatial representation called successor representation.

The only short coming of the work that it doesn't make any predictions that can be tested experimentally. For instance it requires a very specific input pre-processing, i.e. the mapping to successor representation. Can one come up with predictions for the remapping of place cell firing maps based one changes of the maze (e.g. a door closes) or the size of the environment?

Clarity is fine (typo on page 2 line 059).

Strong paper that contributed significantly to a hot topic by providing a link between experimental data on place cell behaviour and path planning implemented by attractor models.

We thank the reviewers for their valuable comments and addressing some important points that we wish to clarify.

REVIEWER 1

Thank you for raising several important questions. We will include 1-2 sentences in the discussion on existing rules that address learning the least-squares optimal weights in the recurrent network, and a note on how our model could combine with a slower (e.g. model-free) system to reproduce the learning curves at different levels of environmental familiarity in [4]. We will remove line 2 of Eq. 3 (defining pi in the text) and add a comment that we set alpha = epsilon to maintain a similar mean level of activity in the network throughout the stimulation (balancing input and decay).

REVIEWER 2

In regards to biological relevance, we believe that our model gives a testable hypothesis for the role that large place fields play in generating goal-directed trajectories (please see the prediction response to reviewer 4 below).

As a biologically plausible means of extracting the dominant eigenvectors, slow feature analysis has been proposed as a natural outcome of an STDP-like learning rule [Sprekeler et al, PloS Comp. Bio., 2007], albeit on the timescale of a standard EPSP rather than the behavioural timescale we consider here. However, STDP can be extended to behavioural timescales when combined with sustained firing and slowly decaying potentials [Drew & Abbott, PNAS, 2006] of the type observed on the single-neuron level in the input pathway to CA3 [Larimer & Strowbridge, Nature Neuroscience, 2010], or as a result of network effects. We will address these potential mechanisms for learning slow features / eigenvectors in the revised manuscript.

REVIEWER 4

We agree that it would be beneficial to add a short description of the predictions of our model to the discussion. In particular, the model proposes that large-scale attractor dynamics in ventral/intermediate CA3 enable long-distance goal-directed preplay activity. This suggests, for instance, that long-distance sequences in dorsal hippocampus (where place fields are much smaller) must be inherited from dynamics in ventral or intermediate hippocampus. In [6] it was shown that selectively sparing the dorsal hippocampus resulted in impaired rapid path planning; in this case, we would also predict a significant reduction in goal-directed preplay activity in the remaining dorsal region. In an intact hippocampus, we would predict that long-distance goal-directed preplay in the dorsal hippocampus is preceded by preplay tracing a similar path in intermediate hippocampus.

Concerning changes to the environment: [14] focused on arguing that changes in place field representations due to training protocols and alterations to the environment are well-described by coding based on the successor representation, and we therefore did not focus on this argument in the current work.

The typo will be corrected, thank you.

REVIEWER 5

For the affinity, d will be corrected to d^2, thank you.

Line 119 states that "The normalized metric can be interpreted as a transition probability...", not a steady-state probability. We will remove line 2 of Eq. 3 and adjust the text below that to clarify the definition.