CAGE Young Talent Round Table: Biological and Neuro Foundations of Utility and RiskIMC Auditorium 0.02 - 9:00-10:40
Gaze Data Reveal Distinct Choice Processes Underlying Model-Based and Model-Free Reinforcement Learning
Organisms appear to learn and make decisions using different strategies known as model-free and model-based learning, where the former is mere reinforcement of previously rewarded actions and the latter is a forward-looking strategy that involves evaluation of action-state transition probabilities. Prior work has used neural data to argue that both model-based and model-free learners implement a value comparison process at trial onset, but model-based learners assign more weight to forward-looking computations. Using eye-tracking, we find evidence for a different interpretation of prior results: model-based subjects make their choices prior to trial onset. In contrast, model-free subjects tend to ignore model-based aspects of the task and instead seem to treat the decision problem as a simple comparison process between two differentially valued items, consistent with previous work on sequential-sampling models of decision making. These findings illustrate a problem with assuming that experimental subjects make their decisions at the same prescribed time.
Neurocomputational Insights into Values, Morals, and Self-control
Selfish, unethical, and short-sighted decision-making lies at the heart of some of society’s most pressing problems, but it is unclear why people so often struggle to make virtuous choices. Here, I show how a simple neurally-informed computational model of choice can account for a wide range of complex social decisions. The model makes novel predictions, borne out by behavioral and neural data, about when and why some choices are more difficult than others. It suggests new methods for estimating value in the absence of choice variability and reveals novel insights into when and why people may resist or succumb to temptations. Finally, it suggests a need to refine popular competitive dual-system models of choice in light of computational model predictions, and points to new ways to help people make better choices for themselves and others.
Dynamically Re-Evaluating Future Prospects
Even when future rewards are relatively certain, the associated costs can be hard to predict. A decision maker might be sure of obtaining a particular reward—the bus will eventually come, the checkout line will eventually move, the paper will eventually be written—but nevertheless face substantial uncertainty as to how much time or effort it will take. In this kind of situation it makes sense to reassess anticipated costs continuously over time while pursuing the reward, not just at the moment of the initial choice. Consider, for example, the case of a reward with an uncertain delay. If the delay has gone on longer than expected, this might, depending on the context, be grounds either for an optimistic or pessimistic assessment of how long a delay still remains. I will discuss behavioral evidence that decision makers reappraise future prospects in a dynamic and context-sensitive manner and use this ongoing reappraisal as a basis for deciding whether to sustain or curtail persistence. This form of ongoing reappraisal can account for patterns of choice that superficially resemble dynamic inconsistency or delay-of-gratification failure. I will also discuss how human neuroimaging methods can advance our understanding of valuation processes. A substantial existing literature has made progress in characterizing and reading out neural valuation signals during discrete economic choices. Our current work seeks to extend these findings to dynamic, temporally extended forms of valuation.
People and Pigeons Gamble Alike
Elliot Ludvig; Marcia Spetch, Chris Madan, and Jeff Pisklak
People often display different patterns of risk preference when making decisions based on explicit descriptions as opposed to when learning from experience. These decisions from experience resemble the decisions made by other animals, who can only rely on their own experience. Here, in a series of experiments, we show that both people and pigeons exhibit a similar bias toward overweighting the extreme outcomes (biggest win and biggest loss) in an experience-based risky choice task. As a result, and contrary to the described case, both species are more risk seeking for relative gains than losses. Pigeons are, however, more sensitive to the total absence of reward (zero outcomes) than people. The emergence of a common mechanism for this risky choice across disparate phyla either indicates similar evolutionary pressures or stems from a shared ancestry.