A common idea about individuals with autism spectrum disorder (ASD) is that they have an above-average preference for predictability and sameness. However, surprisingly little research has gone toward this core symptom, and some studies suggest the preference for predictability in ASD might be less general than commonly assumed. Here, we investigated this important symptom of ASD using three different paradigms, which allowed us to measure preference for predictability under well-controlled experimental conditions. Specifically, we used a dimensional approach by...

When people make risky choices, two kinds of information are crucial: outcome values and outcome probabilities. Here, we demonstrate that the juncture at which value and probability information is provided has a fundamental effect on choice. Across four experiments involving 489 participants, we compared two decision-making scenarios: one in which value information was revealed during sampling (standard) and one in which value information was revealed after sampling (value ignorance). On average, participants made riskier choices when value information was provided...

Human decisions can be habitual or goal-directed, also known as model-free (MF) or model-based (MB) control. Previous work suggests that the balance between the two decision systems is impaired in psychiatric disorders such as compulsion and addiction, via overreliance on MF control. However, little is known whether the balance can be altered through task training. Here, 20 healthy participants performed a well-established two-step task that differentiates MB from MF control, across five training sessions. We used computational modelling and functional near-infrared...

In many daily tasks, we make multiple decisions before reaching a goal. In order to learn such sequences of decisions, a mechanism to link earlier actions to later reward is necessary. Reinforcement learning (RL) theory suggests two classes of algorithms solving this credit assignment problem: In classic temporal-difference learning, earlier actions receive reward information only after multiple repetitions of the task, whereas models with eligibility traces reinforce entire sequences of actions from a single experience (one-shot). Here, we show one-shot learning of...

Adaptive social behavior requires learning probabilities of social reward and punishment, and updating these probabilities when they change. Given prior research on aberrant reinforcement learning in affective disorders, this study examines how social anxiety affects probabilistic social reinforcement learning and dynamic updating of learned probabilities in a volatile environment. N=222 online participants completed questionnaires and a computerized ball-catching game with changing probabilities of reward and punishment. Dynamic learning rates were estimated to...

How do children and adults differ in their search for rewards? We considered three different hypotheses that attribute developmental differences to (a) children’s increased random sampling, (b) more directed exploration toward uncertain options, or (c) narrower generalization. Using a search task in which noisy rewards were spatially correlated on a grid, we compared the ability of 55 younger children (ages 7 and 8 years), 55 older children (ages 9-11 years), and 50 adults (ages 19-55 years) to successfully generalize about unobserved outcomes and balance the...

Visual selective attention acts as a filter on perceptual information, facilitating learning and inference about important events in an agents environment. A role for visual attention in reward-based decisions has previously been demonstrated, but it remains unclear how visual attention is recruited during aversive learning, particularly when learning about multiple stimuli concurrently. This question is of particular importance in psychopathology, where enhanced attention to threat is a putative feature of pathological anxiety. Using an aversive reversal learning task...

In a world where ideas flow freely between people across multiple platforms, we often find ourselves relying on others’ information without an objective standard to judge whether those opinions are accurate. The present study tests an agreement-in-confidence hypothesis of advice perception, which holds that internal metacognitive evaluations of decision confidence play an important functional role - namely being a learning signal that allows to learn about the reliability of others in the absence of feedback - in the perception and use of social information, such as...

Value-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Facing conflicts between competing action alternatives is considered aversive, as recruiting cognitive control to overcome conflict is effortful. Moreover, engaging control to proactively suppress irrelevant information that could conflict with task-relevant information would...

The human ability to compare time between sensory modalities implies a supramodal representation of time. This notion is consistent with the pacemaker-counter model (PCM), the core architecture of prominent timing theories. Some theorists, however, have promoted modality-specific timing mechanisms, which might hamper crossmodal temporal comparison. This study tested whether PCM is sufficient to account for intra- as well as crossmodal timing. To account for modality-specific timing differences, we proceeded from the common assumption that the pacemaker runs faster for...

Humans employ different strategies when making decisions. Previous research has reported reduced reliance on model-based strategies with aging, but it remains unclear whether this is due to cognitive or motivational factors. Moreover, it is not clear how aging affects the metacontrol of decision making, that is the dynamic adaptation of decision-making strategies to varying situational demands. In this cross-sectional study, we tested younger and older adults in a sequential decision-making task that dissociates model-free and model-based strategies. In contrast to...

Uncertainty about the waiting time before obtaining an outcome is integral to intertemporal choice. Here, we showed that people express different time preferences depending on how they learn about this temporal uncertainty. In two studies, people chose between pairs of options: one with a single, sure delay and the other involving multiple, probabilistic delays (a lottery). The probability of each delay occurring either was explicitly described (timing risk) or could be learned through experiential sampling (timing uncertainty; the delay itself was not experienced)....

During a threatening encounter, people can learn to associate the aversive event with a discrete preceding cue or with the context in which the event took place, corresponding to cue-dependent and context-dependent fear conditioning, respectively. Which of these forms of fear learning prevails has critical implications for fear-related psychopathology. We tested here whether acute stress may modulate the balance of cue-dependent and contextual fear learning. Participants (N = 72) underwent a stress or control manipulation 30 min before they completed a...

Physically salient but task-irrelevant distractors can capture attention in visual search, but resource-dependent, executive-control processes can help reduce this distraction. However, it is not only physically salient stimuli that grab our attention: Recent research has shown that reward history also influences the likelihood that stimuli will capture attention. Here, we investigated whether resource-dependent control processes modulate the effect of reward on attentional capture, much as for the effect of physical salience. To this end, we used eye tracking with a...

Depression is characterized by a marked decrease in social interactions and blunted sensitivity to rewards. Surprisingly, despite the importance of social deficits in depression, non-social aspects have been disproportionally investigated. As a consequence, the cognitive mechanisms underlying atypical decision-making in social contexts in depression are poorly understood. In the present study, we investigate whether deficits in reward processing interact with the social context and how this interaction is affected by self-reported depression and anxiety symptoms in the...

In classical fear conditioning, neutral conditioned stimuli that have been paired with aversive physical unconditioned stimuli eventually trigger fear responses. Here, we tested whether aversive mental images systematically paired with a conditioned stimulus also cause de novo fear learning in the absence of any external aversive stimulation. In two experiments (N = 45 and N = 41), participants were first trained to produce aversive, neutral, or no imagery in response to three different visual-imagery cues. In a subsequent imagery-based...

Anxiety is characterized by altered responses under uncertain conditions, but the precise mechanism by which uncertainty changes the behaviour of anxious individuals is unclear. Here we probe the computational basis of learning under uncertainty in healthy individuals and individuals suffering from a mix of mood and anxiety disorders. Participants were asked to choose between four competing slot machines with fluctuating reward and punishment outcomes during safety and stress. We predicted that anxious individuals under stress would learn faster about punishments and...

Reinforcement learning (RL) models describe how humans and animals learn by trial-and-error to select actions that maximize rewards and minimize punishments. Traditional RL models focus exclusively on choices, thereby ignoring the interactions between choice preference and response time (RT), or how these interactions are influenced by contextual factors. However, in the field of perceptual decision-making, such interactions have proven to be important to dissociate between different underlying cognitive processes. Here, we investigated such interactions to shed new...

The Iowa Gambling Task (IGT) is one of the most common paradigms used to assess decision-making and executive functioning in neurological and psychiatric disorders. Several reinforcement-learning (RL) models were recently proposed to refine the qualitative and quantitative inferences that can be made about these processes based on IGT data. Yet, these models do not account for the complex exploratory patterns which characterize participants behavior in the task. Using a dataset of more than 500 subjects, we demonstrate the existence of sequential exploration in the IGT...

Doubt is subjective uncertainty about ones perceptions and recall. It can impair decision-making and is a prominent feature of obsessive-compulsive disorder (OCD). We propose that evaluation of doubt during decision-making provides a useful endophenotype with which to study the underlying pathophysiology of OCD and potentially other psychopathologies. For the current study, we developed a new instrument, the Doubt Questionnaire, to clinically assess doubt. The random dot motion task was used to measure reaction time and subjective certainty, at varying levels of...

Popular computational models of decision-making make specific assumptions about learning processes that may cause them to underfit observed behaviours. Here we suggest an alternative method using recurrent neural networks (RNNs) to generate a flexible family of models that have sufficient capacity to represent the complex learning and decision- making strategies used by humans. In this approach, an RNN is trained to predict the next action that a subject will take in a decision-making task and, in this way, learns to imitate the processes underlying subjects choices and...

Can implicit bias be changed? In a recent longitudinal study, Lai and colleagues (2016, Study 2) compared nine interventions intended to reduce racial bias across 18 university campuses. Although all interventions changed participants’ bias on an immediate test, none were effective after a delay. This study has been interpreted as strong evidence that implicit biases are difficult to change. We revisited Lai et al.’s study to test whether the stability observed reflected persistent individual attitudes or stable environments. Our reanalysis (N = 4,842) indicates...

Psychological sciences have identified a wealth of cognitive processes and behavioral phenomena, yet struggle to produce cumulative knowledge. Progress is hamstrung by siloed scientific traditions and a focus on explanation over prediction, two issues that are particularly damaging for the study of multifaceted constructs like self-regulation. Here, we derive a psychological ontology from a study of individual differences across a broad range of behavioral tasks, self-report surveys, and self-reported real-world outcomes associated with self-regulation. Though both...

Theoretical models distinguish between neural responses elicited by distal threats and those evoked by more immediate threats1-3. Specifically, slower cognitive fear responses towards distal threats involve a network of brain regions including the ventral hippocampus (vHPC) and medial prefrontal cortex (mPFC), while immediate reactive fear responses rely on regions such as the periaqueductal grey4,5. However, it is unclear how anxiety and its neural substrates relate to these distinct defensive survival circuits. We tested whether individual differences in trait anxiety...

Dopamine is thought to provide reward prediction error signals to temporal lobe memory systems, but the role of these signals in episodic memory has not been fully characterized. Here we developed an incidental memory paradigm to (i) estimate the influence of reward prediction errors on the formation of episodic memories, (ii) dissociate this influence from surprise and uncertainty, (iii) characterize the role of temporal correspondence between prediction error and memoranda presentation and (iv) determine the extent to which this influence is dependent on memory...

Decisions must be implemented through actions, and actions are prone to error. As such, when an expected outcome is not obtained, an individual should be sensitive to not only whether the choice itself was suboptimal but also whether the action required to indicate that choice was executed successfully. The intelligent assignment of credit to action execution versus action selection has clear ecological utility for the learner. To explore this, we used a modified version of a classic reinforcement learning task in which feedback indicated whether negative prediction...

Much is known about the effects of reward and punishment on behavior, yet little research has considered how these incentives influence the information-processing dynamics that underlie decision making. We fitted the linear ballistic accumulator to data from a perceptual-judgment task to examine the impacts of reward- and punishment-based incentives on three distinct components of information processing: the quality of the information processed, the quantity of that information, and the decision threshold. The threat of punishment lowered the average quality and...

The time available to inform decisions is often limited, for example because of a response deadline. In such circumstances, accurate knowledge of the amount of time available for a decision is crucial for optimal choice behavior. However, the relation between temporal cognition and decision-making under time pressure is poorly understood. Here, we test how the precision of the internal representation of time affects choice behavior when decision time is limited by a deadline. We show that participants with a precise internal representation of time respond more...

Deep brain stimulation (DBS) is a circuit-oriented treatment for mental disorders. Unfortunately, even well-conducted psychiatric DBS clinical trials have yielded inconsistent symptom relief, in part because DBS mechanism(s) of action are unclear. One clue to those mechanisms may lie in the efficacy of ventral internal capsule/ventral striatum (VCVS) DBS in both major depression (MDD) and obsessive-compulsive disorder (OCD). MDD and OCD both involve deficits in cognitive control. Cognitive control depends on prefrontal cortex (PFC) regions that project into the...

Previous studies of strategic social interaction in game theory have predominantly used games with clearly-defined turns and limited choices. Yet, most real-world social behaviors involve dynamic, coevolving decisions by interacting agents, which poses challenges for creating tractable models of behavior. Here, using a game in which humans competed against both real and artificial opponents, we show that it is possible to quantify the instantaneous dynamic coupling between agents. Adopting a reinforcement learning approach, we use Gaussian Processes to model the policy...

How do we make simple choices such as deciding between an apple and an orange? Recent empirical evidence suggests that choice behaviour and gaze allocation are closely linked at the group level, whereby items looked at longer during the decision-making process are more likely to be chosen. However, it is unclear how variable this gaze bias effect is between individuals. Here we investigate this question across four different simple choice experiments and using a computational model that can be easily applied to individuals. We show that an association between gaze and...

Individuals employ different moral principles to guide their social decision-making, thus expressing a specific ‘moral strategy’. Which computations characterize different moral strategies, and how might they be instantiated in the brain? Here, we tackle these questions in the context of decisions about reciprocity using a modified Trust Game. We show that different participants spontaneously and consistently employ different moral strategies. By mapping an integrative computational model of reciprocity decisions onto brain activity using inter-subject representational...

Recent experiences influence the processing of new information even when those experiences are irrelevant to the current task. Does this reflect the indirect effects of a passively maintained representation of the previous experience, or is this representation reactivated when a new event occurs? To answer this question, we attempted to decode the orientation of the stimulus on the previous trial from the electroencephalogram on the current trial in a working memory task. Behavioral data confirmed that the previous-trial stimulus orientation influenced the reported...

Complex cognition relies on both on-line representations in working memory (WM), said to reside in the focus of attention, and passive off-line representations of related information. Here, we dissected the focus of attention by showing that distinct neural signals index the on-line storage of objects and sustained spatial attention. We recorded electroencephalogram (EEG) activity during two tasks that employed identical stimulus displays but varied the relative demands for object storage and spatial attention. We found distinct delay-period signatures for an attention...

Although some previous studies have investigated the time distortion of anxious patients, it remains open about the interactive effects of trait and state anxieties on time perception. In the present study, participants in high and low trait anxieties perceived 24 negative and 24 neutral words for 2 s in induced anxious and calm mood states, and their time perceptions were recorded by the time reproduction task. The results showed that high trait anxious individuals underestimated the 2-second duration while low trait anxious individuals overestimated the 2-second...

Learning and decision-making are modulated by socio-emotional processing and such modulation is implicated in clinically relevant personality traits of social anxiety. The present study elucidates the computational and neural mechanisms by which emotionally aversive cues disrupt learning in socially anxious human individuals. Healthy volunteers with low or high trait social anxiety performed a reversal learning task requiring learning actions in response to angry or happy face cues. Choice data were best captured by a computational model in which learning rate was...

The perception and integration of sequential numerical information is a common cognitive task. It is a prerequisite for experience-based economic choices, but it is usually not part of economic decision theory. To better understand the process of symbolic number integration and its influence on economic behavior, we performed three experimental studies that examined mean estimates and economic valuations of continuous number distributions. The results indicate that participants valued random number distributions below their respective arithmetic means and valued...

Both basal ganglia (BG) and orbitofrontal cortex (OFC) have been widely implicated in social and non-social decision-making. However, unlike OFC damage, BG pathology is not typically associated with disturbances in social functioning. Here we studied the behavior of patients with focal lesions to either BG or OFC in a multi-strategy competitive game known to engage these regions. We find that whereas OFC patients are significantly impaired, BG patients show intact learning in the economic game. By contrast, when information about the strategic context is absent, both...

Suicidal behavior is associated with impaired decision making in contexts of uncertainty. Existing studies, however, do not definitively address whether suicide attempers have 1) impairment in learning from experience or 2) impairment in choice based on comparison of estimated option values. Our reinforcement learning model-based behavioral study tested these hypotheses directly in middle-aged and older suicide attempters representative of those who die by suicide. Two samples (sample 1, n = 135; sample 2, n = 125) of suicide attempters with depression...

An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF...

Animals rely on learned associations to make decisions. Associations can be based on relationships between object features (e.g., the three leaflets of poison ivy leaves) and outcomes (e.g., rash). More often, outcomes are linked to multidimensional states (e.g., poison ivy is green in summer but red in spring). Feature-based reinforcement learning fails when the values of individual features depend on the other features present. One solution is to assign value to multi-featural conjunctive representations. Here, we test if the hippocampus forms separable conjunctive...

People learn differently from good and bad outcomes. We argue that valence-dependent learning asymmetries are partly driven by beliefs about the causal structure of the environment. If hidden causes can intervene to generate bad (or good) outcomes, then a rational observer will assign blame (or credit) to these hidden causes, rather than to the stable outcome distribution. Thus, a rational observer should learn less from bad outcomes when they are likely to have been generated by a hidden cause, and this pattern should reverse when hidden causes are likely to generate...

A well-established notion in cognitive neuroscience proposes that multiple brain systems contribute to choice behaviour. These include: (1) a model-free system that uses values cached from the outcome history of alternative actions, and (2) a model-based system that considers action outcomes and the transition structure of the environment. The widespread use of this distinction, across a range of applications, renders it important to index their distinct influences with high reliability. Here we consider the two-stage task, widely considered as a gold standard measure...

By combining computational, morphological, and functional analyses, this study relates latent markers of associative threat learning to overt post-traumatic stress disorder (PTSD) symptoms in combat veterans. Using reversal learning, we found that symptomatic veterans showed greater physiological adjustment to cues that did not predict what they had expected, indicating greater sensitivity to prediction errors for negative outcomes. This exaggerated weighting of prediction errors shapes the dynamic learning rate (associability) and value of threat predictive cues. The...

In this report, we provide the first evidence that mood and anxiety dimensions are associated with unique aspects of EEG responses to reward and punishment, respectively. We reanalyzed data from our prior publication of a categorical depiction of depression to address more sophisticated dimensional hypotheses. Highly symptomatic depressed individuals (N = 46) completed a probabilistic learning task with concurrent EEG. Measures of anxiety and depression symptomatology were significantly correlated with each other; however, only anxiety predicted better avoidance...

Computational models of decision making typically assume as people deliberate between options they mentally simulate outcomes from each one and integrate valuations of these outcomes to form a preference. In two studies, we investigated this deliberation process using a task where participants make a series of decisions between a certain and an uncertain option, which were shown as dynamic visual samples that represented possible payoffs. We developed and validated a method of reverse correlational analysis for the task that measures how this time-varying signal was...

Recent sensory history affects subsequent experience. Behavioral results have demonstrated this effect in two forms: repeated exposure to the same sensory input produces negative aftereffects wherein sensory stimuli like those previously experienced are judged as less like the exposed stimulation, while singular exposures can produce positive aftereffects wherein judgments are more like previously experienced stimulation. For timing perception, there is controversy regarding the influence of recent exposure-both singular and repeated exposure produce apparently negative...

Pavlovian influences are important in guiding decision-making across health and psychopathology. There is an increasing interest in using concise computational tasks to parametrise such influences in large populations, and especially to track their evolution during development and changes in mental health. However, the developmental course of Pavlovian influences is uncertain, a problem compounded by the unclear psychometric properties of the relevant measurements. We assessed Pavlovian influences in a longitudinal sample using a well characterised and widely used...

People can learn about the probabilistic consequences of their actions in two ways: One is by consulting descriptions of an action’s consequences and probabilities (e.g., reading up on a medication’s side effects). The other is by personally experiencing the probabilistic consequences of an action (e.g., beta testing software). In principle, people taking each route can reach analogous states of knowledge and consequently make analogous decisions. In the last dozen years, however, research has demonstrated systematic discrepancies between description- and...

A striking range of individual differences has recently been reported in three different visual search tasks. These differences in performance can be attributed to strategy, that is, the efficiency with which participants control their search to complete the task quickly and accurately. Here we ask if an individual’s strategy and performance in one search task is correlated with how they perform in the other two. We tested 64 observers in the three tasks mentioned above over two sessions. Even though the test-retest reliability of the tasks is high, an observer’s...