Dynamic Computation of Value Signals via a Common Neural Network in Multi-Attribute Decision-Making

The human reward system encodes the subjective value of ideas during creative thinking

Creative thinking involves the evaluation of one's ideas in order to select the best one, but the cognitive and neural mechanisms underlying this evaluation remain unclear. Using a combination of creativity and rating tasks, this study demonstrates that individuals attribute subjective values to their ideas, as a relative balance of their originality and adequacy. This relative balance depends on individual preferences and predicts individuals' creative abilities. Using functional Magnetic Resonance Imaging, we find that the Default Mode and the Executive Control Networks respectively encode the originality and adequacy of ideas, and that the human reward system encodes their subjective value. Interestingly, the relative functional connectivity of the Default Mode and Executive Control Networks with the human reward system correlates with the relative balance of adequacy and originality in individuals' preferences. These results add valuation to the incomplete behavioral and neural accounts of creativity, offering perspectives on the influence of individual preferences on creative abilities.

Neural underpinnings of ethical decisions in life and death dilemmas in naïve and expert firefighters

When a single choice impacts on life outcomes, faculties to make ethical judgments come into play. Here we studied decisions in a real-life setting involving life-and-death outcomes that affect others and the decision-maker as well. We chose a genuine situation where prior training and expertise play a role: firefighting in life-threatening situations. By studying the neural correlates of dilemmas involving life-saving decisions, using realistic firefighting situations, allowed us to go beyond previously used hypothetical dilemmas, while addressing the role of expertise and the use of coping strategies (n = 47). We asked the question whether the neural underpinnings of deontologically based decisions are affected by expertise. These realistic life-saving dilemmas activate the same core reward and affective processing network, in particular the ventromedial prefrontal cortex, nucleus accumbens and amygdala, irrespective of prior expertise, thereby supporting general domain theories of ethical decision-making. We found that brain activity in the hippocampus and insula parametrically increased as the risk increased. Connectivity analysis showed a larger directed influence of the insula on circuits related to action selection in non-experts, which were slower than experts in non rescuing decisions. Relative neural activity related to the decision to rescue or not, in the caudate nucleus, insula and anterior cingulate cortex was negatively associated with coping strategies, in experts (firefighters) suggesting practice-based learning. This shows an association between activity and expert-related usage of coping strategies. Expertise enables salience network activation as a function of behavioural coping dimensions, with a distinct connectivity profile when facing life-rescuing dilemmas.

Motivated for near impossibility: How task type and reward modulate task enjoyment and the striatal activation for extremely difficult task

Economic and decision-making theories suppose that people would disengage from a task with near zero success probability, because this implicates little normative utility values. However, humans often are motivated for an extremely challenging task, even without any extrinsic incentives. The current study aimed to address the nature of this challenge-based motivation and its neural correlates. We found that, when participants played a skill-based task without extrinsic incentives, their task enjoyment increased as the chance of success decreased, even if the task was almost impossible to achieve. However, such challenge-based motivation was not observed when participants were rewarded for the task or the reward was determined in a probabilistic manner. The activation in the ventral striatum/pallidum tracked the pattern of task enjoyment. These results suggest that people are intrinsically motivated to challenge a nearly impossible task but only when the task requires certain skills and extrinsic rewards are unavailable.

Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T

The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.

A New Decision-Making GMDH Neural Network: Effective for Limited and Fuzzy Data

This paper presents a new approach to solve multi-objective decision-making (DM) problems based on neural networks (NN). The utility evaluation function is estimated using the proposed group method of data handling (GMDH) NN. A series of training data is obtained based on a limited number of initial solutions to train the NN. The NN parameters are adjusted based on the error propagation training method and unscented Kalman filter (UKF). The designed DM is used in solving the practical problem, showing that the proposed method is very effective and gives favorable results, under limited fuzzy data. Also, the results of the proposed method are compared with some similar methods.

Human's Intuitive Mental Models as a Source of Realistic Artificial Intelligence and Engineering

Despite the success of artificial intelligence (AI), we are still far away from AI that model the world as humans do. This study focuses for explaining human behavior from intuitive mental models' perspectives. We describe how behavior arises in biological systems and how the better understanding of this biological system can lead to advances in the development of human-like AI. Human can build intuitive models from physical, social, and cultural situations. In addition, we follow Bayesian inference to combine intuitive models and new information to make decisions. We should build similar intuitive models and Bayesian algorithms for the new AI. We suggest that the probability calculation in Bayesian sense is sensitive to semantic properties of the objects' combination formed by observation and prior experience. We call this brain process as computational meaningfulness and it is closer to the Bayesian ideal, when the occurrence of probabilities of these objects are believable. How does the human brain form models of the world and apply these models in its behavior? We outline the answers from three perspectives. First, intuitive models support an individual to use information meaningful ways in a current context. Second, neuroeconomics proposes that the valuation network in the brain has essential role in human decision making. It combines psychological, economical, and neuroscientific approaches to reveal the biological mechanisms by which decisions are made. Then, the brain is an over-parameterized modeling organ and produces optimal behavior in a complex word. Finally, a progress in data analysis techniques in AI has allowed us to decipher how the human brain valuates different options in complex situations. By combining big datasets with machine learning models, it is possible to gain insight from complex neural data beyond what was possible before. We describe these solutions by reviewing the current research from this perspective. In this study, we outline the basic aspects for human-like AI and we discuss on how science can benefit from AI. The better we understand human's brain mechanisms, the better we can apply this understanding for building new AI. Both development of AI and understanding of human behavior go hand in hand.