{"title": "Sequential effects reflect parallel learning of multiple environmental regularities", "book": "Advances in Neural Information Processing Systems", "page_first": 2053, "page_last": 2061, "abstract": "Across a wide range of cognitive tasks, recent experience in\ufb02uences behavior. For example, when individuals repeatedly perform a simple two-alternative forced-choice task (2AFC), response latencies vary dramatically based on the immediately preceding trial sequence. These sequential effects have been interpreted as adaptation to the statistical structure of an uncertain, changing environment (e.g. Jones & Sieck, 2003; Mozer, Kinoshita, & Shettel, 2007; Yu & Cohen, 2008). The Dynamic Belief Model (DBM) (Yu & Cohen, 2008) explains sequential effects in 2AFC tasks as a rational consequence of a dynamic internal representation that tracks second-order statistics of the trial sequence (repetition rates) and predicts whether the upcoming trial will be a repetition or an alternation of the previous trial. Experimental results suggest that \ufb01rst-order statistics (base rates) also in\ufb02uence sequential effects. We propose a model that learns both \ufb01rst- and second-order sequence properties, each according to the basic principles of the DBM but under a uni\ufb01ed inferential framework. This model, the Dynamic Belief Mixture Model (DBM2), obtains precise, parsimonious \ufb01ts to data. Furthermore, the model predicts dissociations in behavioral (Maloney, Dal Martello, Sahm, & Spillmann, 2005) and electrophysiological studies (Jentzsch & Sommer, 2002), supporting the psychological and neurobiological reality of its two components.", "full_text": "Sequential effects re\ufb02ect parallel learning of multiple\n\nenvironmental regularities\n\nMatthew H. Wilder(cid:63), Matt Jones\u2020, & Michael C. Mozer(cid:63)\n\n(cid:63)Dept. of Computer Science\n\n\u2020Dept. of Psychology\nUniversity of Colorado\n\n<wildermh@colorado.edu, mcj@colorado.edu, mozer@colorado.edu>\n\nBoulder, CO 80309\n\nAbstract\n\nAcross a wide range of cognitive tasks, recent experience in\ufb02uences behavior. For\nexample, when individuals repeatedly perform a simple two-alternative forced-\nchoice task (2AFC), response latencies vary dramatically based on the immedi-\nately preceding trial sequence. These sequential effects have been interpreted\nas adaptation to the statistical structure of an uncertain, changing environment\n(e.g., Jones and Sieck, 2003; Mozer, Kinoshita, and Shettel, 2007; Yu and Co-\nhen, 2008). The Dynamic Belief Model (DBM) (Yu and Cohen, 2008) explains\nsequential effects in 2AFC tasks as a rational consequence of a dynamic internal\nrepresentation that tracks second-order statistics of the trial sequence (repetition\nrates) and predicts whether the upcoming trial will be a repetition or an alterna-\ntion of the previous trial. Experimental results suggest that \ufb01rst-order statistics\n(base rates) also in\ufb02uence sequential effects. We propose a model that learns both\n\ufb01rst- and second-order sequence properties, each according to the basic princi-\nples of the DBM but under a uni\ufb01ed inferential framework. This model, the Dy-\nnamic Belief Mixture Model (DBM2), obtains precise, parsimonious \ufb01ts to data.\nFurthermore, the model predicts dissociations in behavioral (Maloney, Martello,\nSahm, and Spillmann, 2005) and electrophysiological studies (Jentzsch and Som-\nmer, 2002), supporting the psychological and neurobiological reality of its two\ncomponents.\n\n1\n\nIntroduction\n\nPicture an intense match point at the Wimbledon tennis championship, Nadal on the defense from\nFederer\u2019s powerful shots. Nadal returns three straight hits to his forehand side. In the split second\nbefore the ball is back in his court, he forms an expectation about where Federer will hit the ball\nnext\u2014will the streak of forehands continue or will there be a switch to his backhand. As the point\ncontinues, Nadal gains the upper ground and begins making Federer alternate from forehand to\nbackhand to forehand. Now Federer \ufb01nds himself trying to predict whether or not this alternating\npattern will be continued with the next shot. These two are caught up in a high-stakes game of\nsequential effects\u2014their actions and expectations for the current shot have a strong dependence on\nthe past few shots. Sequential effects play a ubiquitous role in our lives\u2014our actions are constantly\naffected by our recent experiences.\nIn controlled environments, sequential effects have been observed across a wide range of tasks and\nexperimental paradigms, and aspects of cognition ranging from perception to memory to language\nto decision making. Sequential effects often occur without awareness and cannot be overriden by\ninstructions, suggesting a robust cognitive inclination to adapt behavior in an ongoing manner. Sur-\nprisingly, people exhibit sequential effects even when they are aware that there is no dependence\n\n1\n\n\f(a)\n\n(b)\n\nFigure 1: (a) DBM \ufb01t to the behavioral data from Cho et al. (2002). Predictions within each of the four\ngroups are monotonically increasing or decreasing. Thus the model is unable to account for the two circled\nrelationships. This \ufb01t accounts for 95.8% of the variance in the data. (p0 = Beta(2.6155, 2.4547), \u03b1 =\n0.4899) (b) The \ufb01t to the same data obtained from DBM2 in which probability estimates are derived from both\n\ufb01rst-order and second-order trial statistics. 99.2% of the data variance is explained by this \ufb01t. (\u03b1 = 0.3427,\nw = 0.4763)\n\nstructure to the environment. Progress toward understanding the intricate complexities of sequential\neffects will no doubt provide important insights into the ways in which individuals adapt to their\nenvironment and make predictions about future outcomes.\nOne classic domain where reliable sequential effects have been observed is in two-alternative forced-\nchoice (2AFC) tasks (e.g, Jentzsch and Sommer, 2002; Hale, 1967; Soetens et al., 1985; Cho et al.,\n2002). In this type of task, participants are shown one of two different stimuli, which we denote\nas X and Y, and are instructed to respond as quickly as possible by mapping the stimulus to a\ncorresponding response, say pressing the left button for X and the right button for Y. Response time\n(RT) is recorded, and the task is repeated several hundred or thousand times. To measure sequential\neffects, the RT is conditioned on the recent trial history. (In 2AFC tasks, stimuli and responses are\nconfounded; as a result, it is common to refer to the \u2019trial\u2019 instead of the \u2019stimulus\u2019 or \u2019response\u2019. In\nthis paper, \u2019trial\u2019 will be synonymous with the stimulus-response pair.) Consider a sequence such\nas XY Y XX, where the rightmost symbol is the current trial (X), and the symbols to the left are\nsuccessively earlier trials. Such a four-back trial history can be represented in a manner that focuses\nnot on the trial identities, but on whether trials are repeated or alternated. With R and A denoting\nrepetitions and alternations, respectively, the trial sequence XY Y XX can be encoded as ARAR.\nNote that this R/A encoding collapses across isomorphic sequences XY Y XX and Y XXY Y .\nThe small blue circles in Figure 1a show the RTs from Cho et al. (2002) conditioned on the recent\ntrial history. Along the abscissa in Figure 1a are all four-back sequence histories ordered according\nto the R/A encoding. The left half of the graph represents cases where the current trial is a repetition\nof the previous, and the right half represents cases where the current trial is an alternation. The\ngeneral pattern we see in the data is a triangular shape that can be understood by comparing the\ntwo extreme points on each half, RRRR vs. AAAR and RRRA vs. AAAA. It seems logical that\nthe response to the current trial in RRRR will be signi\ufb01cantly faster than in AAAR (RTRRRR <\nRTAAAR) because in the RRRR case, the current trial matches the expectation built up over the past\nfew trials whereas in the AAAR case, the current trial violates the expectation of an alternation. The\nsame argument applies to RRRA vs. AAAA, leading to the intuition that RTRRRA > RTAAAA.\nThe trial histories are ordered along the abscissa so that the left half is monotonically increasing\nand the right half is monotonically decreasing following the same line of intuition, i.e., many recent\nrepetitions to many recent alternations.\n\n2 Toward A Rational Model Of Sequential Effects\n\nMany models have been proposed to capture sequential effects, including Estes (1950), Anderson\n(1960), Laming (1969), and Cho et al. (2002). Other models have interpreted sequential effects as\nadaptation to the statistical structure of a dynamic environment (e.g., Jones and Sieck, 2003; Mozer,\nKinoshita, and Shettel, 2007). In this same vein, Yu and Cohen (2008) recently suggested a rational\n\n2\n\n300320340360380400Response TimeRRRRARRRRARRAARRRRARARARRAARAAARRRRAARRARARAAARARRAAARAARAAAAAAAChoDBM300320340360380400Response Time  RRRRARRRRARRAARRRRARARARRAARAAARRRRAARRARARAAARARRAAARAARAAAAAAAChoDBM2\f(a)\n\n(b)\n\n(c)\n\nFigure 2: Three graphical models that capture sequential dependencies. (a) Dynamic Belief Model (DBM) of\nYu and Cohen (2008). (b) A reformulation of DBM in which the output variable, St, is the actual stimulus\nidentity instead of the repetition/alternation representation used in DBM. (c) Our proposed Dynamic Belief\nMixture Model (DBM2). Models are explained in more detail in the text.\n\nexplanation for sequential effects such as those observed in Cho et al. (2002). According to their\nDynamic Belief Model (DBM), individuals estimate the statistics of a nonstationary environment.\nThe key contribution of this work is that it provides a rational justi\ufb01cation for sequential effects that\nhave been previously viewed as resulting from low-level brain mechanisms such as residual neural\nactivation.\nDBM describes performance in 2AFC tasks as Bayesian inference over whether the next trial in the\nsequence will be a repetition or an alternation of the previous trial, conditioned on the trial history. If\nRt is the Bernoulli random variable that denotes whether trial t is a repetition (Rt = 1) or alternation\n(Rt = 0) of the previous trial, DBM determines P (Rt| (cid:126)Rt\u22121), where (cid:126)Rt\u22121 denotes the trial sequence\npreceding trial t, i.e., (cid:126)Rt\u22121 = (R1, R2, ..., Rt\u22121).\nDBM assumes a generative model, shown in Figure 2a, in which Rt = 1 with probability \u03b3t and\nRt = 0 with probability 1\u2212\u03b3t. The random variable \u03b3t describes a characteristic of the environment.\nAccording to the generative model, the environment is nonstationary and \u03b3t can either retain the\nsame value as on trial t \u2212 1 or it can change. Speci\ufb01cally, Ct denotes whether the environment\nhas changed between t \u2212 1 and t (Ct = 1) or not (Ct = 0). Ct is a Bernoulli random variable\nwith success probability \u03b1. If the environment does not change, \u03b3t = \u03b3t\u22121. If the environment\nchanges, \u03b3t is drawn from a prior distribution, which we refer to as the reset prior denoted by\np0(\u03b3) \u223c Beta(a, b).\nBefore each trial t of a 2AFC task, DBM computes the probability of the upcoming stimulus condi-\ntioned on the trial history. The model assumes that the perceptual and motor system is tuned based\non this expectation, so that RT will be a linearly decreasing function of the probability assigned to\nthe event that actually occurs, i.e. of P (Rt = R| (cid:126)Rt\u22121) on repetition trials and of P (Rt = A| (cid:126)Rt\u22121)\n= 1 - P (Rt = R| (cid:126)Rt\u22121) on alternation trials.\nThe red plusses in Figure 1 show DBM\u2019s \ufb01t to the data from Cho et al. (2002). DBM has \ufb01ve\nfree parameters that were optimized to \ufb01t the data. The parameters are: the change probability,\n\u03b1; the imaginary counts of the reset prior, a and b; and two additional parameters to map model\nprobabilities to RTs via an af\ufb01ne transform.\n\n2.1\n\nIntuiting DBM predictions\n\nAnother contribution of Yu and Cohen (2008) is the mathematical demonstration that DBM is ap-\nproximately equivalent to an exponential \ufb01lter over trial histories. That is, the probability that the\ncurrent stimulus is a repetition is a weighted sum of past observations, with repetitions being scored\nas 1 and alternations as 0, and with weights decaying exponentially as a function of lag. The ex-\nponential \ufb01lter gives insight into how DBM probabilities will vary as a function of trial history.\nConsider two 4-back trial histories: an alternation followed by two repetitions (ARR\u2212) and two\nalternations followed by a repetition (AAR\u2212), where the \u2212 indicates that the current trial type is\nunknown. An exponential \ufb01lter predicts that ARR\u2212 will always create a stronger expectation for\nan R on the current trial than AAR\u2212 will, because the former includes an additional past repetition.\nThus, if the current trial is in fact a repetition, the model predicts a faster RT for ARR\u2212 compared\nto AAR\u2212 (i.e., RTARRR < RTAARR). Conversely, if the current trial is an alternation, the model\n\n3\n\nC t-1C t\u03b3 t-1\u03b3 tR t-1R tC t-1C t\u03b3 t-1\u03b3 tS t-1S tC t-1C tS t-1S t\u03c6 t-1\u03c6 t\u03b3 t-1\u03b3 t\fpredicts RTARRA > RTAARA. Similarly, if two sequences with the same number of Rs and As\nare compared, for example RAR\u2212 and ARR\u2212, the model predicts RTRARR > RTARRR and\nRTRARA < RTARRA because more recent trials have a stronger in\ufb02uence.\nComparing the exponential \ufb01lter predictions for adjacent sequences in Figure 1 yields the expecta-\ntion that the RTs will be monotonically increasing in the left two groups of four and monotonically\ndecreasing in the two right groups. The data are divided into groups of 4 because the relationships\nbetween histories like AARR and RRAR depend on the speci\ufb01c parameters of the exponential \ufb01l-\nter, which determine whether one recent A will outweigh two earlier As. It is clear in Figure 1 that\nthe DBM predictions follow this pattern.\n\n2.2 what\u2019s missing in DBM\n\nDBM offers an impressive \ufb01t to the overall pattern of the behavioral data. Circled in Figure 1,\nhowever, we see two signi\ufb01cant pairs of sequence histories for which the monotonicity prediction\ndoes not hold. These are reliable aspects of the data and are not measurement error. Consider\nthe circle on the left, in which RTARAR > RTRAAR for the human data. Because DBM functions\napproximately as an exponential \ufb01lter, and the repetition in the trial history is more recent for ARAR\nthan for RAAR, DBM predicts RTARAR < RTRAAR. An exponential \ufb01lter, and thus DBM, is\nunable to account for this deviation in the data.\nTo understand this mismatch, we consider an alternative representation of the trial history: the \ufb01rst-\norder sequence, i.e., the sequence of actual stimulus values. The two R/A sequences ARAR and\nRAAR correspond to stimulus sequences XY Y XX and XXY XX. If we consider an exponen-\ntial \ufb01lter on the actual stimulus sequence, we obtain the opposite prediction from that of DBM:\nRTXY Y XX > RTXXY XX because there are more recent occurrences of X in the latter sequence.\nThe other circled data in Figure 1a correspond to an analogous situation. Again, DBM also makes\na prediction inconsistent with the data, that RTARAA > RTRAAA, whereas an exponential \ufb01lter on\nstimulus values predicts the opposite outcome\u2014RTXY Y XY < RTXXY XY . Of course this analysis\nleads to predictions for other pairs of points where DBM is consistent with the data and a stimulus\nbased exponential \ufb01lter is inconsistent. Nevertheless, the variations in the data suggest that more\nimportance should be given to the actual stimulus values.\nIn general, we can divide the sequential effects observed in the data into two classes: \ufb01rst- and\nsecond-order effects. First-order sequential effects result from the priming of speci\ufb01c stimulus or\nresponse values. We refer to this as a \ufb01rst-order effect because it depends only on the stimulus\nvalues rather than a higher-order representation such as the repetition/alternation nature of a trial.\nThese effects correspond to the estimation of the baserate of each stimulus or response value. They\nare observed in a wide range of experimental paradigms and are referred to as stimulus priming\nor response priming. The effects captured by DBM, i.e. the triangular pattern in RT data, can be\nthought of as a second-order effect because it re\ufb02ects learning of the correlation structure between\nthe current trial and the previous trial. In second-order effects, the actual stimulus value is irrelevant\nand all that matters is whether the stimulus was a repetition of the previous trial. As DBM proposes,\nthese effects essentially arise from an attempt to estimate the repetition rate of the sequence.\nDBM naturally produces second-order sequential effects because it abstracts over the stimulus level\nof description: observations in the model are R and A instead of the actual stimuli X and Y . Because\nof this abstraction, DBM is inherently unable to exhibit \ufb01rst-order effects. To gain an understanding\nof how \ufb01rst-order effects could be integrated into this type of Bayesian framework, we reformulate\nthe DBM architecture. Figure 2b shows an equivalent depiction of DBM in which the generative\nprocess on trial t produces the actual stimulus value, denoted St. St is conditioned on both the\nrepetition probability, \u03b3t, and the previous stimulus value, St\u22121. Under this formulation, St = St\u22121\nwith probability \u03b3t, and St equals the opposite of St\u22121 (i.e., XY or Y X) with probability 1 \u2212 \u03b3t.\nAn additional bene\ufb01t of this reformulated architecture is that it can represent \ufb01rst-order effects if we\nswitch the meaning of \u03b3. In particular, we can treat \u03b3 as the probability of the stimulus taking on a\nspeci\ufb01c value (X or Y ) instead of the probability of a repetition. St is then simply a draw from a\nBernoulli process with rate \u03b3. Note that for modeling a \ufb01rst-order effect with this architecture, the\nconditional dependence of St on St\u22121 becomes unnecessary. The nonstationarity of the environ-\nment, as represented by the change variable C, behaves in the same way regardless of whether we\nuse the model to represent \ufb01rst- or second-order structure.\n\n4\n\n\f3 Dynamic Belief Mixture Model\n\nThe complex contributions of \ufb01rst- and second-order effects to the full pattern of observed sequential\neffects suggest the need for a model with more explanatory power than DBM. It seems clear that\nindividuals are performing a more sophisticated inference about the statistics of the environment\nthan proposed by DBM. We have shown that the DBM architecture can be reformulated to generate\n\ufb01rst-order effects by having it infer the baserate instead of the repetition rate of the sequence, but the\nempirical data suggest both mechanisms are present simultaneously. Thus the challenge is to merge\nthese two effects into one model that performs joint inference over both environmental statistics.\nHere we propose a Bayesian model that captures both \ufb01rst- and second-order effects, building on the\nbasic principles of DBM. According to this new model, which we call the Dynamic Belief Mixture\nModel (DBM2), the learner assumes that the stimulus on a given trial is probabilistically affected\nby two factors: the random variable \u03c6, which represents the sequence baserate, and the random\nvariable \u03b3, which represents the repetition rate. The combination of these two factors is governed\nby a mixture weight w that represents the relative weight of the \u03c6 component. As in DBM, the\nenvironment is assumed to be nonstationary, meaning that on each trial, with probability \u03b1, \u03c6 and \u03b3\nare jointly resampled from the reset prior, p0(\u03c6, \u03b3), which is uniform over [0, 1]2. Figure 2c shows\nthe graphical architecture for this model. This architecture is an extension of our reformulation of\nthe DBM architecture in Figure 2b. Importantly, the observed variable, S, is the actual stimulus\nvalue instead of the repetition/alternation representation used in DBM. This architecture allows for\nexplicit representation of the baserate, through the direct in\ufb02uence of \u03c6t on the physical stimulus\nvalue St, as well as representation of the repetition rate through the joint in\ufb02uence of \u03b3t and the\nprevious stimulus St\u22121 on St. Formally, we express the probability of St given \u03c6, \u03b3, and St\u22121 as\nshown in Equation 1.\n\nP (St = X|\u03c6t, \u03b3t, St\u22121 = X) = w\u03c6t + (1 \u2212 w)\u03b3t\nP (St = X|\u03c6t, \u03b3t, St\u22121 = Y ) = w\u03c6t + (1 \u2212 w)(1 \u2212 \u03b3t)\n\n(1)\n\nDBM2 operates by maintaining the iterative prior over \u03c6 and \u03b3, p(\u03c6t, \u03b3t|(cid:126)St\u22121). After each observa-\ntion, the joint posterior, p(\u03c6t, \u03b3t|(cid:126)St), is computed using Bayes\u2019 Rule from the iterative prior and the\nlikelihood of the most recent observation, as shown in Equation 2.\n\np(\u03c6t, \u03b3t|(cid:126)St) \u221d P (St|\u03c6t, \u03b3t, St\u22121)p(\u03c6t, \u03b3t|(cid:126)St\u22121).\n\n(2)\n\nThe iterative prior for the next trial is then a mixture of the posterior from the current trial, weighted\nby 1 \u2212 \u03b1, and the reset prior, weighted by \u03b1 (the probability of change in \u03c6 and \u03b3).\n\np(\u03c6t+1, \u03b3t+1|(cid:126)St) = (1 \u2212 \u03b1)p(\u03c6t, \u03b3t|(cid:126)St) + \u03b1p0(\u03c6t+1, \u03b3t+1).\n\n(3)\nThe model generates predictions, P (St|(cid:126)St\u22121), by integrating Equation 1 over the iterative prior on\n\u03c6t and \u03b3t. In our simulations, we maintain a discrete approximation to the continuous joint iterative\nprior with the interval [0,1] divided into 100 equally spaced sections. Expectations are computed by\nsumming over the discrete probability mass function.\nFigure 1b shows that DBM2 provides an excellent \ufb01t to the Cho et al. data, explaining the combina-\ntion of both \ufb01rst- and second-order effects. To account for the overall advantage of repetition trials\nover alternation trials in the data, a repetition bias had to be built into the reset prior in DBM. In\nDBM2, the \ufb01rst-order component naturally introduces an advantage for repetition trials. This occurs\nbecause the estimate of \u03c6t is shifted toward the value of the previous stimulus, St\u22121, thus leading\nto a greater expectation that the same value will appear on the current trial. This fact eliminates the\nneed for a nonuniform reset prior in DBM2. We use a uniform reset prior in all DBM2 simulations,\nthus allowing the model to operate with only four free parameters: \u03b1, w, and the two parameters for\nthe af\ufb01ne transform from model probabilities to RTs.\nThe nonuniform reset prior in DBM allows it to be biased either for repetition or alternation. This\n\ufb02exibility is important in a model, because different experiments show different biases, and the\nbiases are dif\ufb01cult to predict. For example, the Jentzsch and Sommer experiment showed little\n\n5\n\n\f(a)\n\n(b)\n\nFigure 3: DBM2 \ufb01ts for the behavioral data from (a) Jentzsch and Sommer (2002) Experiment 1 which accounts\nfor 96.5% of the data variance (\u03b1 = 0.2828, w = 0.3950) and (b) Maloney et al. (2005) Experiment 1 which\naccounts for 97.7% of the data variance. (\u03b1 = 0.0283, w = 0.3591)\n\nbias, but a replication we performed\u2014with the same stimuli and same responses\u2014obtained a strong\nalternation bias. It is our hunch that the bias should not be cast as part of the computational theory\n(speci\ufb01cally, the prior); rather, the bias re\ufb02ects attentional and perceptual mechanisms at play, which\ncan introduce varying degrees of an alternation bias. Speci\ufb01cally, four classic effects have been\nreported in the literature that make it dif\ufb01cult for individuals to process the same stimulus two times\nin a row at a short lag: attentional blink Raymond et al. (1992), inhibition of return Posner and\nCohen (1984), repetition blindness Kanwisher (1987), and the Ranschburg effect Jahnke (1969).\nFor example, with repetition blindness, processing of an item is impaired if it occurs within 500 ms\nof another instance of the same item in a rapid serial stream; this condition is often satis\ufb01ed with\n2AFC. In support of our view that fast-acting secondary mechanisms are at play in 2AFC, Jentzsch\nand Sommer (Experiment 2) found that using a very short lag between each response and the next\nstimulus modulated sequential effects in a dif\ufb01cult-to-interpret manner. Explaining this \ufb01nding via\na rational theory would be challenging. To allow for various patterns of bias across experiments, we\nintroduced an additional parameter to our model, an offset speci\ufb01cally for repetition trials, which\ncan serve as a means of removing the in\ufb02uence of the effects listed above. This parameter plays\nmuch the same role as DBM\u2019s priors. Although it is not as elegant, we believe it is more correct,\nbecause the bias should be considered as part of the neural implementation, not the computational\ntheory.\n\n4 Other Tests of DBM2\n\nWith its ability to represent both \ufb01rst- and second-order effects, DBM2 offers a robust model for a\nrange of sequential effects. In Figure 3a, we see that DBM2 provides a close \ufb01t to the data from\nExperiment 1 of Jentzsch and Sommer (2002). The general design of this 2AFC task is similar to\nthe design in Cho et al. (2002) though some details vary. Notably we see a slight advantage on\nalternation trials, as opposed to the repetition bias seen in Cho et al.\nSurprisingly, DBM2 is able to account for the sequential effects in other binary decision tasks that\ndo not \ufb01t into the 2AFC paradigm. In Experiment 1 of Maloney et al. (2005), subjects observed\na rotation of two points on a circle and reported whether the direction of rotation was positive\n(clockwise) or negative (counterclockwise). The stimuli were constructed so that the direction of\nmotion was ambiguous, but a particular variable related to the angle of motion could be manipulated\nto make subjects more likely to perceive one direction or the other. Psychophysical techniques were\nused to estimate the Point of Subjective Indifference (PSI), the angle at which the observer was\nequally likely to make either response. PSI measures the subject\u2019s bias toward perceiving a positive\nas opposed to a negative rotation. Maloney et. al. found that this bias in perceiving rotation was\nin\ufb02uenced by the recent trial history. Figure 3b shows the data for this experiment rearranged to be\nconsistent with the R/A orderings used elsewhere (the sequences on the abscissa show the physical\nstimulus values, ending with Trial t \u2212 1). The bias, conditioned on the 4-back trial history, follows\na similar pattern to that seen with RTs in Cho et al. (2002) and Jentzsch and Sommer (2002).\n\n6\n\n260280300320340360Response TimeRRRRARRRRARRAARRRRARARARRAARAAARRRRAARRARARAAARARRAAARAARAAAAAAAJentzsch 1DBM2N biasneutralP biasPSINNNNPNNNPPNNNPNNPPPNNPPNNNPNPNPNPPPPNPPPNNPPPNPPNNNPPNNPPPNPNPNPMaloney 1DBM2\fTable 1: A comparison between the % of data variance explained by DBM and DBM2.\n\nDBM\nDBM2\n\nCho\n95.8\n99.2\n\nJentzsch 1 Maloney 1\n95.5\n96.5\n\n96.1\n97.7\n\nIn modeling Experiment 1, we assumed that PSI re\ufb02ects the subject\u2019s probabilistic expectation about\nthe upcoming stimulus. Before each trial, we computed the model\u2019s probability that the next stimu-\nlus would be P, and then converted this probability to the PSI bias measure using an af\ufb01ne transform\nsimilar to our RT transform. Figure 3b shows the close \ufb01t DBM2 obtains for the experimental data.\nTo assess the value of DBM2, we also \ufb01t DBM to these two experiments. Table 1 shows the com-\nparison between DBM and DBM2 for both datasets as well as Cho et al. The percentage of variance\nexplained by the models is used as a measure for comparison. Across all three experiments, DBM2\ncaptures a greater proportion of the variance in the data.\n\n5 EEG evidence for \ufb01rst-order and second-order predictions\n\nDBM2 proposes that subjects in binary choice tasks track both the baserate and the repetition rate\nin the sequence. Therefore an important source of support for the model would be evidence for the\npsychological separability of these two mechanisms. One such line of evidence comes from Jentzsch\nand Sommer (2002), who used electroencephalogram (EEG) recordings to provide additional insight\ninto the mechanisms involved in the 2AFC task. The EEG was used to record subjects\u2019 lateralized\nreadiness potential (LRP) during performance of the task. LRP essentially provides a way to identify\nthe moment of response selection\u2014a negative spike in the LRP signal in motor cortex re\ufb02ects initia-\ntion of a response command in the corresponding hand. Jentzsch and Sommer present two different\nways of analyzing the LRP data: stimulus-locked LRP (S-LRP) and response-locked LRP (LRP-R).\nThe S-LRP interval measures the time from stimulus onset to response activation on each trial. The\nLRP-R interval measures the time elapsed between response activation and the actual response. To-\ngether, these two measures provide a way to divide the total RT into a stimulus-processing stage and\na response-execution stage.\nInterestingly, the S-LRP and LRP-R data exhibit different patterns of sequential effects when condi-\ntioned on the 4-back trial histories, as shown in Figure 4. DBM2 offers a natural explanation for the\ndifferent patterns observed in the two stages of processing, because they align well with the division\nbetween \ufb01rst- and second-order sequential effects. In the S-LRP data, the pattern is predominantly\nsecond-order, i.e. RT on repetition trials increases as more alternations appear in the recent history,\nand RT on alternation trials shows the opposite dependence. In contrast, the LRP-R results exhibit\nan effect that is mostly \ufb01rst-order (which could be easily seen if the histories were reordered under\nan X/Y representation). Thus we can model the LRP data by extracting the separate contributions\nof \u03c6 and \u03b3 in Equation 1. We use the \u03b3 component (i.e., the second term on the RHS of Eq. 1) to\npredict the S-LRP results and the \u03c6 component (i.e., the \ufb01rst term on the RHS of Eq. 1) to predict\nthe LRP-R results. This decomposition is consistent with the model of overall RT, because the sum\nof these components provides the model\u2019s RT prediction, just as the sum of the S-LRP and LRP-R\nmeasures equals the subject\u2019s actual RT (up to an additive constant explained below).\nFigure 4 shows the model \ufb01ts to the LRP data. The parameters of the model were constrained to\nbe the same as those used for \ufb01tting the behavioral results shown in Figure 3a. To convert the\nprobabilities in DBM2 to durations, we used the same scaling factor used to \ufb01t the behavioral data\nbut allowed for new offsets for the R and A groups for both S-LRP and LRP-R. The offset terms\nneed to be free because the difference in procedures for estimating S-LRP and LRP-R (i.e., aligning\ntrials on the stimulus vs.\nthe response) allows the sum of S-LRP and LRP-R to differ from total\nRT by an additive constant related to the random variability in RT across trials. Other than these\noffset terms, the \ufb01ts to the LRP measures constitute parameter-free predictions of EEG data from\nbehavioral data.\n\n7\n\n\f(a)\n\n(b)\n\n(c)\n\nFigure 4: (a) and (b) show DBM2 \ufb01ts to the S-LRP results of Jentzsch and Sommer (2002) Experiment 1. Model\nparameters are the same as those used for the behavioral \ufb01t shown in Figure 3a, except for offset parameters.\nDBM2 explains 73.4% of the variance in the S-LRP data and 87.0% of the variance in the LRP-R data. (c)\nBehavioral results and DBM2 \ufb01ts for Experiment 2 of Maloney et al. (2005). The model \ufb01t explains 91.9% of\nthe variance in the data (\u03b1 = 0.0283, w = 0).\n\n6 More evidence for the two components of DBM2\n\nIn the second experiment reported in Maloney et al. (2005), participants only responded on every\nfourth trial. The goal of this manipulation was to test whether the sequential effect occurred in\nthe absence of prior responses. Each ambiguous test stimulus followed three stimuli for which the\ndirection of rotation was unambiguous and to which the subject made no response. The responses\nto the test stimuli were grouped according to the 3-back stimulus history, and a PSI value was\ncomputed for each of the eight histories to measure subjects\u2019 bias toward perceiving positive vs.\nnegative rotation. The results are shown in Figure 4c. As in Figure 3b, the histories on the abscissa\nshow the physical stimulus values, ending with Trial t \u2212 1, and the arrangement of these histories is\nconsistent with the R/A orderings used elsewhere in this paper.\nDBM2\u2019s explanation of Jentzsch and Sommer\u2019s EEG results indicates that \ufb01rst-order sequential\neffects arise in response processing and second-order effects arise in stimulus processing. Therefore,\nthe model predicts that, in the absence of prior responses, sequential effects will follow a pure\nsecond-order pattern. The results of Maloney et al.\u2019s Experiment 2 con\ufb01rm this prediction. Just as\nin the S-LRP data of Jentzsch and Sommer (2002), the \ufb01rst-order effects have mostly disappeared,\nand the data are well explained by a pure second-order effect (i.e., a stronger bias for alternation\nwhen there are more alternations in the history, and vice versa). We simulated this experiment with\nDBM2 using the same value of the change parameter (\u03b1) from the \ufb01t of Maloney et al.\u2019s Experiment\n1. Additionally, we set the mixture parameter, w, to 0, which removes the \ufb01rst-order component of\nthe model. For this experiment we used different af\ufb01ne transformation values than in Experiment\n1 because the modi\ufb01cations in the experimental design led to a generally weaker sequential effect,\nwhich we speculate to have been due to lesser engagement by subjects when fewer responses were\nneeded. Figure 4c shows the \ufb01t obtained by DBM2, which explains 91.9% data variance.\n\n7 Discussion\n\nOur approach highights the power of modeling simultaneously at the levels of rational analysis and\npsychological mechanism. The details of the behavioral data (i.e. the systematic discrepancies from\nDBM) pointed to an improved rational analysis and an elaborated generative model (DBM2) that is\ngrounded in both \ufb01rst- and second-order sequential statistics. In turn, the conceptual organization\nof the new rational model suggested a psychological architecture (i.e., separate representation of\nbaserates and repetition rates) that was borne out in further data. The details of these latter \ufb01ndings\nnow turn back to further inform the rational model. Speci\ufb01cally, the \ufb01ts to Jentzsch and Sommer\u2019s\nEEG data and to Maloney et al.\u2019s intermittent-response experiment suggest that the statistics individ-\nuals track are differentially tied to the stimuli and responses in the task. That is, rather than learning\nstatistics of the abstract trial sequence, individuals learn the baserates (i.e., marginal probabilities) of\nresponses and the repetition rates (i.e., transition probabilities) of stimulus sequences. This division\nsuggests further hypotheses about both the empirical nature and the psychological representation\nof stimulus sequences and of response sequences, which future experiments and statistical analyses\nwill hopefully shed light on.\n\n8\n\n180200220240260280300Response Time  RRRRARRRRARRAARRRRARARARRAARAAARRRRAARRARARAAARARRAAARAARAAAAAAAS\u2212LRPDBM220406080100120Response Time  RRRRARRRRARRAARRRRARARARRAARAAARRRRAARRARARAAARARRAAARAARAAAAAAALRP\u2212RDBM2N biasneutralP biasPSINNNPNNPPNNPNPPPNPPNNPPNPMaloney 2DBM2\fReferences\nM. Jones and W. Sieck. Learning myopia: An adaptive recency effect in category learning. Journal\n\nof Experimental Psychology: Learning, Memory, & Cognition, 29:626\u2013640, 2003.\n\nM. Mozer, S. Kinoshita, and M. Shettel. Sequential dependencies offer insight into cognitive control.\nIn W. Gray, editor, Integrated Models of Cognitive Systems, pages 180\u2013193. Oxford University\nPress, 2007.\n\nA. Yu and J. Cohen. Sequential effects: Superstition or rational behavior? NIPS, pages 1873\u20131880,\n\n2008.\n\nL. Maloney, M. Dal Martello, C. Sahm, and L. Spillmann. Past trials in\ufb02uence perception of am-\nbiguous motion quartets through pattern completion. Proceedings of the National Academy of\nSciences, 102:3164\u20133169, 2005.\n\nI. Jentzsch and W. Sommer. Functional localization and mechanisms of sequential effects in serial\n\nreaction time tasks. Perception and Psychophysics, 64(7):1169\u20131188, 2002.\n\nD. Hale. Sequential effects in a two-choice reaction task. Quarterly Journal of Experimental Psy-\n\nchology, 19:133\u2013141, 1967.\n\nE. Soetens, L. Boer, and J. Hueting. Expectancy or automatic facilitation? separating sequential\neffects in two-choice reaction time. Journal of experimental psychology. Human perception and\nperformance, 11:598\u2013616, 1985.\n\nR. Cho, L. Nystrom, E. Brown, A. Jones, T. Braver, P. Holmes, and J. Cohen. Mechanisms under-\nlying dependencies of performance on stimulus history in a two-alternative forced-choice task.\nCognitive, Affective, & Behavioral Neuroscience, 4:283\u2013299, 2002.\n\nW. Estes. Toward a statistical theory of learning. Psychological Review, 57:94\u2013107, 1950.\nN. Anderson. Effect of \ufb01rst-order conditional probability in a two-choice learning situation. Journal\n\nof Experimental Psychology, 59(2):73\u201383, 1960.\n\nD. Laming. Subjective probability in choice-reaction experiments. Journal of Mathematical Psy-\n\nchology, 6:81\u2013120, 1969.\n\nJ. Raymond, K. Shapiro, and K. Arnell. Temporary suppression of visual processing in an rsvp task:\nan attentional blink? Journal of experimental psychology. Human perception and performance,\n18:849\u2013860, 1992.\n\nM. Posner and Y. Cohen. Components of visual orienting.\n\nIn H. Bouma and D. G. Bouwhuis,\neditors, Attention and Performance X: Control of language processes, pages 531\u2013556. Erlbaum,\nHillsdale, NJ, 1984.\n\nN. Kanwisher. Repetition blindness: Type recognition without token individuation. Cognition, 27:\n\n117\u2013143, 1987.\n\nJ. Jahnke. The ranschburg effect. Psychological Review, 76:592\u2013605, 1969.\n\n9\n\n\f", "award": [], "sourceid": 1078, "authors": [{"given_name": "Matthew", "family_name": "Wilder", "institution": null}, {"given_name": "Matt", "family_name": "Jones", "institution": null}, {"given_name": "Michael", "family_name": "Mozer", "institution": null}]}