# data[].attributes.body
In this section, it talks about the brain and its structures.  It tells each part and the function of it.

# data[].attributes.body
This section discusses the relevance of brain research. It discusses how knowing research can help with learning. It also discusses brain myths that are relevant to learning.

# data[].attributes.body
This section talks about how the brain processes motivation and emotions. It discusses how rewards relate to motivation. It discusses emotional stages.

# data[].attributes.body
This section discusses the brain and its stages of development.  It also discusses factors that come into play regarding brain development.

# data[].attributes.body
This section talks about brain processing during learning.  It looks at information processing models regarding neurophysiology.

# data[].attributes.attachment_link.readable_text
What is the brain?

The brain is a complex organ that controls thought, memory, emotion, touch, motor skills, vision, breathing, temperature, hunger and every process that regulates our body. Together, the brain and spinal cord that extends from it make up the central nervous system, or CNS.

What is the brain made of?

Weighing about 3 pounds in the average adult, the brain is about 60% fat. The remaining 40% is a combination of water, protein, carbohydrates and salts. The brain itself is a not a muscle. It contains blood vessels and nerves, including neurons and glial cells.

What is the gray matter and white matter?

Gray and white matter are two different regions of the central nervous system. In the brain, gray matter refers to the darker, outer portion, while white matter describes the lighter, inner section underneath. In the spinal cord, this order is reversed: The white matter is on the outside, and the gray matter sits within.

Gray matter is primarily composed of neuron somas (the round central cell bodies), and white matter is mostly made of axons (the long stems that connects neurons together) wrapped in myelin (a protective coating). The different composition of neuron parts is why the two appear as separate shades on certain scans.

Each region serves a different role. Gray matter is primarily responsible for processing and interpreting information, while white matter transmits that information to other parts of the nervous system.

How does the brain work?

The brain sends and receives chemical and electrical signals throughout the body. Different signals control different processes, and your brain interprets each. Some make you feel tired, for example, while others make you feel pain.

Some messages are kept within the brain, while others are relayed through the spine and across the body's vast network of nerves to distant extremities. To do this, the central nervous system relies on billions of neurons (nerve cells).

Main Parts of the Brain and Their Functions

At a high level, the brain can be divided into the cerebrum, brainstem and cerebellum.

Cerebrum

The cerebrum (front of brain) comprises gray matter (the cerebral cortex) and white matter at its center. The largest part of the brain, the cerebrum initiates and coordinates movement and regulates temperature. Other areas of the cerebrum enable speech, judgment, thinking and reasoning, problem-solving, emotions and learning. Other functions relate to vision, hearing, touch and other senses.

Cerebral Cortex

Cortex is Latin for "bark," and describes the outer gray matter covering of the cerebrum. The cortex has a large surface area due to its folds, and comprises about half of the brain's weight.

The cerebral cortex is divided into two halves, or hemispheres. It is covered with ridges (gyri) and folds (sulci). The two halves join at a large, deep sulcus (the interhemispheric fissure, AKA the medial longitudinal fissure) that runs from the front of the head to the back. The right hemisphere controls the left side of the body, and the left half controls the right side of the body. The two halves communicate with one another through a large, C-shaped structure of white matter and nerve pathways called the corpus callosum. The corpus callosum is in the center of the cerebrum.

Brainstem

The brainstem (middle of brain) connects the cerebrum with the spinal cord. The brainstem includes the midbrain, the pons and the medulla.

Midbrain. The midbrain (or mesencephalon) is a very complex structure with a range of different neuron clusters (nuclei and colliculi), neural pathways and other structures. These features facilitate various functions, from hearing and movement to calculating responses and environmental changes. The midbrain also contains the substantia nigra, an area affected by Parkinson's disease that is rich in dopamine neurons and part of the basal ganglia, which enables movement and coordination.

Pons. The pons is the origin for four of the 12 cranial nerves, which enable a range of activities such as tear production, chewing, blinking, focusing vision, balance, hearing and facial expression. Named for the Latin word for "bridge," the pons is the connection between the midbrain and the medulla.

Medulla. At the bottom of the brainstem, the medulla is where the brain meets the spinal cord. The medulla is essential to survival. Functions of the medulla regulate many bodily activities, including heart rhythm, breathing, blood flow, and oxygen and carbon dioxide levels. The medulla produces reflexive activities such as sneezing, vomiting, coughing and swallowing.

The spinal cord extends from the bottom of the medulla and through a large opening in the bottom of the skull. Supported by the vertebrae, the spinal cord carries messages to and from the brain and the rest of the body.

Cerebellum

The cerebellum ("little brain") is a fist-sized portion of the brain located at the back of the head, below the temporal and occipital lobes and above the brainstem. Like the cerebral cortex, it has two hemispheres. The outer portion contains neurons, and the inner area communicates with the cerebral cortex. Its function is to coordinate voluntary muscle movements and to maintain posture, balance and equilibrium. New studies are exploring the cerebellum's roles in thought, emotions and social behavior, as well as its possible involvement in addiction, autism and schizophrenia.

Brain Coverings: Meninges

Three layers of protective covering called meninges surround the brain and the spinal cord.

The outermost layer, the dura mater, is thick and tough. It includes two layers: The periosteal layer of the dura mater lines the inner dome of the skull (cranium) and the meningeal layer is below that. Spaces between the layers allow for the passage of veins and arteries that supply blood flow to the brain.

The arachnoid mater is a thin, weblike layer of connective tissue that does not contain nerves or blood vessels. Below the arachnoid mater is the cerebrospinal fluid, or CSF. This fluid cushions the entire central nervous system (brain and spinal cord) and continually circulates around these structures to remove impurities.

The pia mater is a thin membrane that hugs the surface of the brain and follows its contours. The pia mater is rich with veins and arteries.

Lobes of the Brain and What They Control

Each brain hemisphere (parts of the cerebrum) has four sections, called lobes: frontal, parietal, temporal and occipital. Each lobe controls specific functions.

Frontal lobe. The largest lobe of the brain, located in the front of the head, the frontal lobe is involved in personality characteristics, decision-making and movement. Recognition of smell usually involves parts of the frontal lobe. The frontal lobe contains Broca's area, which is associated with speech ability.

Parietal lobe. The middle part of the brain, the parietal lobe helps a person identify objects and understand spatial relationships (where one's body is compared with objects around the person). The parietal lobe is also involved in interpreting pain and touch in the body. The parietal lobe houses Wernicke's area, which helps the brain understand spoken language.

Occipital lobe. The occipital lobe is the back part of the brain that is involved with vision.

Temporal lobe. The sides of the brain, temporal lobes are involved in short-term memory, speech, musical rhythm and some degree of smell recognition.

Deeper Structures Within the Brain

Pituitary Gland

Sometimes called the "master gland," the pituitary gland is a pea-sized structure found deep in the brain behind the bridge of the nose. The pituitary gland governs the function of other glands in the body, regulating the flow of hormones from the thyroid, adrenals, ovaries and testicles. It receives chemical signals from the hypothalamus through its stalk and blood supply.

Hypothalamus

The hypothalamus is located above the pituitary gland and sends it chemical messages that control its function. It regulates body temperature, synchronizes sleep patterns, controls hunger and thirst and also plays a role in some aspects of memory and emotion.

Amygdala

Small, almond-shaped structures, an amygdala is located under each half (hemisphere) of the brain. Included in the limbic system, the amygdalae regulate emotion and memory and are associated with the brain's reward system, stress, and the "fight or flight" response when someone perceives a threat.

Hippocampus

A curved seahorse-shaped organ on the underside of each temporal lobe, the hippocampus is part of a larger structure called the hippocampal formation. It supports memory, learning, navigation and perception of space. It receives information from the cerebral cortex and may play a role in Alzheimer's disease.

Pineal Gland

The pineal gland is located deep in the brain and attached by a stalk to the top of the third ventricle. The pineal gland responds to light and dark and secretes melatonin, which regulates circadian rhythms and the sleep-wake cycle.

Ventricles and Cerebrospinal Fluid

Deep in the brain are four open areas with passageways between them. They also open into the central spinal canal and the area beneath arachnoid layer of the meninges.

The ventricles manufacture cerebrospinal fluid, or CSF, a watery fluid that circulates in and around the ventricles and the spinal cord, and between the meninges. CSF surrounds and cushions the spinal cord and brain, washes out waste and impurities, and delivers nutrients.

Blood Supply to the Brain

Two sets of blood vessels supply blood and oxygen to the brain: the vertebral arteries and the carotid arteries.

The external carotid arteries extend up the sides of your neck, and are where you can feel your pulse when you touch the area with your fingertips. The internal carotid arteries branch into the skull and circulate blood to the front part of the brain.

The vertebral arteries follow the spinal column into the skull, where they join together at the brainstem and form the basilar artery, which supplies blood to the rear portions of the brain.

The circle of Willis, a loop of blood vessels near the bottom of the brain that connects major arteries, circulates blood from the front of the brain to the back and helps the arterial systems communicate with one another.

Cranial Nerves

Inside the cranium (the dome of the skull), there are 12 nerves, called cranial nerves:

Cranial nerve 1: The first is the olfactory nerve, which allows for your sense of smell.

Cranial nerve 2: The optic nerve governs eyesight.

Cranial nerve 3: The oculomotor nerve controls pupil response and other motions of the eye, and branches out from the area in the brainstem where the midbrain meets the pons.

Cranial nerve 4: The trochlear nerve controls muscles in the eye. It emerges from the back of the midbrain part of the brainstem.

Cranial nerve 5: The trigeminal nerve is the largest and most complex of the cranial nerves, with both sensory and motor function. It originates from the pons and conveys sensation from the scalp, teeth, jaw, sinuses, parts of the mouth and face to the brain, allows the function of chewing muscles, and much more.

Cranial nerve 6: The abducens nerve innervates some of the muscles in the eye.

Cranial nerve 7: The facial nerve supports face movement, taste, glandular and other functions.

Cranial nerve 8: The vestibulocochlear nerve facilitates balance and hearing.

Cranial nerve 9: The glossopharyngeal nerve allows taste, ear and throat movement, and has many more functions.

Cranial nerve 10: The vagus nerve allows sensation around the ear and the digestive system and controls motor activity in the heart, throat and digestive system.

Cranial nerve 11: The accessory nerve innervates specific muscles in the head, neck and shoulder.

Cranial nerve 12: The hypoglossal nerve supplies motor activity to the tongue.

The first two nerves originate in the cerebrum, and the remaining 10 cranial nerves emerge from the brainstem, which has three parts: the midbrain, the pons and the medulla.

# data[].attributes.attachment_link.description
The brain is an important organ that controls thought, memory, emotion, touch, motor skills, vision, respiration, and every process that regulates your body.

# data[].attributes.attachment_link.readable_text
Arnaud Messé,

David Rudrauf,

Habib Benali,

Guillaume Marrelec

Subject Areas

For more information about PLOS Subject Areas, click here.

We want your feedback. Do these Subject Areas make sense for this article? Click the target next to the incorrect Subject Area and let us know. Thanks for your help!

Abstract

Investigating the relationship between brain structure and function is a central endeavor for neuroscience research. Yet, the mechanisms shaping this relationship largely remain to be elucidated and are highly debated. In particular, the existence and relative contributions of anatomical constraints and dynamical physiological mechanisms of different types remain to be established. We addressed this issue by systematically comparing functional connectivity (FC) from resting-state functional magnetic resonance imaging data with simulations from increasingly complex computational models, and by manipulating anatomical connectivity obtained from fiber tractography based on diffusion-weighted imaging. We hypothesized that FC reflects the interplay of at least three types of components: (i) a backbone of anatomical connectivity, (ii) a stationary dynamical regime directly driven by the underlying anatomy, and (iii) other stationary and non-stationary dynamics not directly related to the anatomy. We showed that anatomical connectivity alone accounts for up to 15% of FC variance; that there is a stationary regime accounting for up to an additional 20% of variance and that this regime can be associated to a stationary FC; that a simple stationary model of FC better explains FC than more complex models; and that there is a large remaining variance (around 65%), which must contain the non-stationarities of FC evidenced in the literature. We also show that homotopic connections across cerebral hemispheres, which are typically improperly estimated, play a strong role in shaping all aspects of FC, notably indirect connections and the topographic organization of brain networks.

Author Summary

By analogy with the road network, the human brain is defined both by its anatomy (the 'roads'), that is, the way neurons are shaped, clustered together and connected to each others and its dynamics (the 'traffic'): electrical and chemical signals of various types, shapes and strength constantly propagate through the brain to support its sensorimotor and cognitive functions, its capacity to learn and adapt to disease, and to create consciousness. While anatomy and dynamics are organically intertwined (anatomy contributes to shape dynamics), the nature and strength of this relation remain largely mysterious. Various hypotheses have been proposed and tested using modern neuroimaging techniques combined with mathematical models of brain activity. In this study, we demonstrate the existence (and quantify the contribution) of a dynamical regime in the brain, coined 'stationary', that appears to be largely induced and shaped by the underlying anatomy. We also reveal the critical importance of specific anatomical connections in shaping the global anatomo-functional structure of this dynamical regime, notably connections between hemispheres.

Citation: Messé A, Rudrauf D, Benali H, Marrelec G (2014) Relating Structure and Function in the Human Brain: Relative Contributions of Anatomy, Stationary Dynamics, and Non-stationarities. PLoS Comput Biol 10(3): e1003530. https://doi.org/10.1371/journal.pcbi.1003530

Editor: Claus C. Hilgetag, Hamburg University, Germany

Received: August 14, 2013; Accepted: February 8, 2014; Published: March 20, 2014

Copyright: © 2014 Messé et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: This work is supported by the Inserm and the University Pierre et Marie Curie (Paris, France). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Coherent behavior and cognition involve synergies between neuronal populations in interaction [1]-[3]. Even at rest, in the absence of direct environmental stimulations, these interactions drive the synchronization of spontaneous activity across brain systems, shedding light on the large-scale anatomo-functional organization of the brain [4]. The study of such patterns of synchronization has known important developments due to recent methodological advances in brain imaging data acquisition and analysis. These advances have enabled investigators to estimate interactions in the brain by measuring functional connectivity (FC) from resting-state functional MRI (rs-fMRI). Analyses of FC at rest have supported the hypothesis that the brain is spatially organized into large-scale intrinsic networks [5]-[7], e.g. the so-called resting-state networks [8], [9], such as the default mode network, which have been linked to central integrative cognitive functions [10]-[13]. The study of large-scale intrinsic networks from rs-fMRI has become a central and active area for neuroscience research. However, the mechanisms and factors driving FC, as well as their relative contribution to empirical data, are still highly debated [14] and remain to be elucidated.

Theoretical rationale and empirical findings support the hypothesis that FC is driven and shaped by structural connectivity (SC) between brain systems, i.e., by the actual bundles of white matter fiber connecting neurons [15]. As a first approximation, SC can be inferred from fiber tractography based on diffusion-weighted imaging (DWI) [16]-[19]. A recent study [20], which focused on a small subset of robustly estimated structural connections, demonstrated the existence of a statistical, yet complex, correspondence between FC and specific features of SC (e.g., low vs. high fiber density, short vs. long fibers, intra vs. interhemispheric connections). However, a large part of FC cannot be explained by SC alone [21]. There appears that FC is the result of at least two main contributing factors: (i) the underlying anatomical structure of connectivity, and (ii) the dynamics of neuronal populations emerging from their physiology [3]. A key issue is to better understand the relative contributions of these two components to FC. Besides, recent studies using windowed analyses have suggested that FC estimated over an entire acquisition session (referred to as 'stationary FC' in the literature) breaks down into a variety of reliable correlation patterns (also referred to as 'dynamic FC' or 'non-stationarities') when estimated over short time windows (30 s) [14], [22]. The authors advocated that FC estimated over short time windows (or windowed FC, for short) mostly reflects recurrent transitory patterns that are aggregated when estimating FC over a whole session. They further suggested that whole-session FC may only be an epiphenomenon without clear physiological underpinning, and not the reflection of an actual process with stationary FC [14]. This perspective remains to be reconciled with the fact that whole-session FC has been found to be highly reproducible, functionally meaningful and a useful biomarker in many pathological contexts [23], [24]. Note that, in the recent literature of fMRI data analysis, stationarity implicitely refers to a stationary FC (i.e., the invariance of FC over time), to be contrasted with the more general notion of (strong) stationarity, where a model or process is stationary if its parameters remain constant over time [25], [26]. SC being temporally stable at the scale of a whole resting state fMRI session (typically 10 min), we could expect SC to drive a stationary process (in the strong sense). Since SC is furthermore expected to drive FC, we can hypothesize that this stationary process contributes to generate a stationary FC.

In order to bring together the structural and dynamical components underlying FC, some studies have used computational models that incorporate SC together with biophysical models of neuronal activity to generate coherent brain dynamics [27]-[32]. This approach has yielded promising results for the understanding of the relationship between structure and function [17], [33], [34]. Here, we used a testbed of well-established generative models simulating neuronal dynamics combined with empirical measures, to investigate the relative contributions of anatomical connections, stationary dynamics, and non-stationarities to the emergence of empirical functional connectivity. In particular, we considered the following hypotheses: (H1) part of FC directly reflects SC; (H2) models of physiological mechanisms added to SC increase predictive power all the more as they are complex; (H3) part of the variance of FC that is unexplained by models is due to issues in the estimation of SC, e.g., problems with measuring homotopic connections; (H4) there is an actual stationary process reflected in whole-session FC that is not merely an artifact but substantially reflects the driving of the dynamics by SC.

In order to test these hypotheses and estimate the relative contribution of anatomy, stationary dynamics and non-stationarities to FC, we relied on the following approach. After T ₁-weighted MRI based parcellation of the brain ( N = 160 regions), SC was estimated using the proportion of white matter fibers connecting pairs of regions, based on probabilistic tractography of DWI data [35]. FC was measured on rs-fMRI data using Pearson correlation between the time courses of brain regions. We quantified the correlation between SC alone and FC as a reference, and also fed SC to generative neurodynamical models of increasing complexity: a spatial autoregressive (SAR) model [36], analytic models with or without conduction delays [28]-[31], [37], and biologically constrained models [29], [32]. Importantly, all these models were used in their stationary regime in the strong sense, since their parameters were not changed during the simulations. Of these models, only the SAR is explicitely associated with a stationary FC; other, more complex models, generate dynamics that are compatible with a non-stationary FC. We computed FC from data simulated by these models and compared the results to empirical FC. For each model, performance was quantified using predictive power [29], for each subject as well as on the 'average subject' (obtained by averaging SC and empirical FC across subjects). Values for the model parameters were based on the literature, except for the structural coupling strength that was optimized in order to maximize each model's performance.

Results

Predictive power of models

In agreement with H1, SC explained a significant amount of the variance of whole-session FC for all subjects, as did all generative models (permutation test, p<0.05 corrected) ( Figure 1, panel A). Generative models predicted FC better than SC alone (paired permutation test, p<0.05 corrected). Predictive power obtained with the average subject ranged from 0.32 for SC alone to 0.43 for the SAR model ( Table 1). For a given model, predictive power was reproducible across subjects. Contrary to our hypothesis H2, generative models had similar performance, and complexity was not predictive of performance. The results remained unchanged when no global signal regression was applied ( Figure S1). Also, findings were found to be similar for SC alone and the SAR model at finer spatial scales ( N = 461 and N = 825 regions, Figure S2) and consistent with a replication dataset ( Figure S3). Most importantly, a large part of the variance ( R ²) in the empirical data (at least 82%) remained unexplained by this first round of simulations.

(A) Predictive power for all connections and when restricted to intra/interhemispheric, direct/indirect connections. For each type of connections and each model, we represented the individual predictive powers (bar chart representing means and associated standard deviations), as well as the predictive power for the average subject computed using the original SC (diamonds), or after adding homotopic connections (circles). Of note, SC alone has no predictive power (zero) for the subset of indirect connections, by definition. (B) Patterns of SC, empirical FC and FC simulated from the SAR model for the average subject and associated scatter plot of simulated versus empirical FC (solid line represents perfect match). SARh stands for the SAR model with added homotopic connections. Matrices were rearranged such that network structure is highlighted. Homologous regions were arranged symmetrically with respect to the center of the matrix; for instance, the first and last regions are homologous. (C) Similarity of functional brain networks across subjects (bar chart with means and associated standard deviations), for the average subject (diamonds), and when adding homotopic connections (circles) (left). Network maps for the average subject and empirical FC, as well as for FC simulated using either the SAR model with original SC or the SARh.

Role of homotopic connections

We reasoned (see hypothesis H3) that part of the unexplained variance could reflect issues with the estimation of SC from DWI, which can be expected because of limitations in current fiber tracking algorithms and the problem of crossing fibers [38]. We know for instance that many fibers passing through the corpus callosum are poorly estimated in diffusion imaging, in particular those connecting more lateral parts of the cerebral cortex [39]. Yet, the corpus callosum is the main interhemispheric commissure of the mammal brain, see [40]. It systematically connects homologous sectors of the cerebral cortex across the two hemispheres in a topographically organized manner, with an antero-posterior gradient, through a system of myelinated homotopic fibers or 'homotopic connections'. The hypothesis of an impact of SC estimation problems on FC unexplained variance was supported by the observation that, in our results, intrahemispheric connections yielded on average a much higher predictive power (e.g., 0.59 for the SAR model) than interhemispheric connections (0.16 for the SAR model).

In order to further test the role of white matter connections in driving FC, we artificially set all homotopic connections to a constant SC value (0.5) for the average subject and reran all simulations. As a result, the predictive power strongly increased for all models ( Figure 1, panels A and B), ranging from 0.39 for SC alone to 0.61 for the SAR model ( Table 1). Thus the variance unexplained (1- R ²) was reduced to 63%. Moreover, predictive power for intra and interhemispheric connections became equivalent (0.60 and 0.62, respectively). Interestingly, adding homotopic connections also led to a substantial increase in predictive power for indirect connections, that is, pairs of regions for which SC is zero (increasing from 0.07 to 0.45). The effect of adding interhemispheric anatomical connections on increasing predictive power was highly specific to homotopic connections. When applying the SAR model to the SC matrix with added homotopic connections and randomly permuting (10 000 permutations) the 80 corresponding interhemispheric connections (one region in one hemisphere was connected to one and only one region in the other hemisphere), the predictive power strongly decreased, even compared to results with the original SC ( Figure 2, panel A). Moreover, we further assessed the specificity of this result by systematically manipulating SC. In three different simulations, we randomly removed, added, and permuted structural connections (10 000 times). In all cases, the predictive power decreased as a function of the proportion of connections manipulated ( Figure 2, panel B). Moreover, changes induced by these manipulations remained small (<0.05), far below the changes that we were able to induce by adding homotopic connections. All in all, these results suggest that homotopic connections play a key role in shaping the network dynamics, in a complex and non-trivial manner.

(A) Predictive power of the SAR model with original SC (green), when adding homotopic connections ('SARh', red), or with shuffled homotopic connections (black). (B) Predictive power of the SAR model with original SC (red) and when SC values were randomly permuted, removed or added (from left to right). For each graph, predictive power was quantified as a function of the percentage of connections manipulated.

Predicting functional brain networks

Beyond predicting the overall pattern of FC, we also assessed whether models could predict the empirical organization of FC into a set of intrinsic networks. Connectivity matrices were clustered into groups of non-overlapping brain regions showing high within-group correlation and low between-group correlation, and the resulting partitions into functional brain networks were compared between empirical and simulated FC using the adjusted Rand index (see Methods). Again, the SAR model tended to perform best among all computational models ( Figure 1, panel C).

Without adding homotopic connections in the SC matrix, the simulated networks highly differed from the empirical networks. In particular, most networks were found to be lateralized. After adding homotopic connections, the resemblence between simulated and empirical networks greatly improved. Networks were more often bilateral and overall consistent with the topography of empirical functional networks, including somatosensory, motor, visual, and associative networks. High FC between the amygdala and ventral-lateral sectors of the prefrontal cortex was also correctly predicted by the simulations. There were nevertheless some notable differences. First, the clustering of empirical FC yielded a long-range fronto-parieto-temporal association network ( Figure 1, panel C, cyan) that was not observed in simulated FC as such. Second, a parieto-temporal cluster ( Figure 1, panel C, red), which was associated with thalamo-striatal networks, was predicted by simulations but was not present in the empirical data. Third, a cluster encompassing the entire cingulate gyrus and precuneus ( Figure 1, panel C, green) was predicted by simulations but was broken down into more clusters in the empirical data.

Stationary FC, non-stationary FC, and non-stationarities

The results above show that SC plays a causal role in FC, but one can still wonder what aspects of the underlying dynamics are the most directly related to this influence. A hypothesis is that SC, in combination with stable physiological processes (e.g., overall gain in synaptic transmission), drives a stationary regime of the dynamics. This hypothesis is supported by the finding that all models tested in this study, which were used in a stationary regime (in the strong sense), could explain significantly more variance than SC alone. Furthermore, the fact that the SAR could predict FC significantly better than all other models is evidence that this stationary regime is associated with stationary FC (paired permutation test, p<0.05 corrected).

But, clearly, many variations in the dynamical patterns of brain activity, be it in the process of spontaneous cognition, physiological regulation, or context-dependent changes, cannot be expected to be associated with a purely stationary FC. Modeling how the brain dynamics deal with endogenous and environmental contexts should require more complex models, either stationary or non-stationary, that are able to generate non-stationary (i.e., time-varying) patterns of FC. Given that at best 37% of the variance could be explained by the model of a purely stationary FC (the SAR), we can wonder why the models of higher complexity used in our simulation testbed did not perform better in predicting FC. One possible hypothesis is that the SAR model was favored in the simulations, because we estimated FC over about 10 minutes of actual brain dynamics. In such configuration, we can imagine that the non-stationarities of FC cancel out, the estimation effectively keeping the stationary part of FC. We thus wondered whether the more complex models would better perform when non-stationary FC had the potential of being more strongly reflected in the data. We approached this question by computing predictive power on windowed FC as a function of the length of the time-window used [22], for all possible time-windows over which FC could be estimated and for all models. We also investigated the effect of simulation duration (see Methods). We found that the relative performance of more complex models was still lower than that of the SAR model ( Figures 3 and S4). The average predictive power was lower for shorter time-windows and increased towards a limit for longer time-windows. The SAR model behaved like an 'upper-bound' for predictive power. The performance of all other models, irrespective of the size of the time-window, was between that of SC alone and that of the SAR model.

Predictive power as a function of time-window length across subjects (left) and of duration of simulated runs on the average subject (right). For color code see Figure 1.

A straightforward explanation is that the non-stationary patterns of FC, as generated by the simulation models, did not match the non-stationary patterns of the empirical FC as they unfolded during the acquisition in the brain of the participants. Context-dependent and transient dynamics are likely to be missed by models of the dynamics that cannot be contextually constrained in the absence of further information. It is thus difficult to infer how much of the 63% of unexplained variance remaining in whole-session FC actually reflect physiologically meaningful non-stationary FC, and more broadly, non-stationary dynamics.

Discussion

In the present study, we investigated the respective contributions of anatomical connections, stationary dynamics, and non-stationarities to the emergence of empirical functional connectivity. We compared the performance of computational models in modeling FC and manipulated SC in order to analyze the impact of SC on FC, with and without the filter of combined physiological models of the dynamics.

The importance of white matter fiber pathways in shaping functional brain networks is a known fact, for a review, see [15], [17], [21], [23]. Previous modeling studies have supported the importance of the underlying anatomical connections, i.e., SC, in shaping functional relationships among brain systems [16], [41], [42]. In agreement with our hypothesis H1, we showed that functional connectivity could at least in part be explained by structural connectivity alone. Adding homotopic connections in the matrix of SC, we found a slight increase in explained variance when considering the prediction of whole-session FC from SC alone (+4% of explained variance). In agreement with H2, adding models of physiological interactions above and beyond SC alone increased the explained variance in whole-session FC, by 8% for the best performing model, the SAR model, when no homotopic connections were added, and by 22% when homotopic connections were added. This latter fact, which strongly supports H3, suggests a complex interplay between anatomy as reflected by SC and physiological mechanisms in generating FC. This impact of SC manipulations on predicted FC pertained not only to direct but also to indirect connections. For indirect connections, whole-session FC was much better predicted after adding homotopic connections to SC than before adding them (0.45 versus 0.07 in predictive power). The problem of limited predictive power for FC based on SC when considering indirect connections has puzzled the field [43]. For this reasons many studies only assess the performance of models on direct connections. Here, we showed that a major factor in driving FC for indirect anatomical connections (+20% in explained variance) is the interplay between a subset of anatomical connections, i.e., homotopic connections (which are typically underestimated by DWI), and physiological parameters that generate the dynamics underlying FC, themselves conditioned by the possible interactions defined by SC.

Contrary to our expectation (see hypothesis H2), all models tended to perform similarly, irrespective of model complexity. The best performing model in most cases was the SAR model, a model of stationary FC driven by SC, with 63% of the variance remaining unexplained. It is likely that, above and beyond problems with the estimation of SC from DWI, and other incompressible sources of irrelevant noise, much of the unexplained variance in FC relates to non-stationary patterns in FC, and more generally to non-stationarities in the strong-sense. Such non-stationarities are difficult to model in the experimentally unconstrained resting-state and in the absence of further information regarding the specific parameters shaping FC. Irrespective of their complexity, computational models are only capable of generating prototypical brain activities, and not the subject-dependent activity that took place in the brain of the participants during scanning. The scientific necessity of modeling brain dynamics is hindered by such uncertainty and it will be a challenge to find solutions to approach this problem [26], [44]. Even though one objective for neuroscience is to propose generative models that are capable of generating detailed neuronal dynamics, generative models cannot be informed by this unknown context and, as a consequence, cannot generate context-dependent activity in a manner that would be predictive of empirical data, in the absence of additional measures and experimental controls. Nevertheless, and perhaps for that very reason, the study of non-stationarity in FC should become of central interest for the field, as such non-stationarities could explain much of FC (up to 63% according to our simulation results), and thus reflect critical mechanisms for neurocognitive processing.

In the absence of adequate modeling principles, determining the precise contribution of non-stationarities to the unexplained variance in FC is impossible, as other confounding sources of unexplained variance are expected. As we showed, even naive manipulations aimed at estimating the impact of the known errors in DWI-based reconstruction of homotopic connections showed that such errors could cause 20% of the unexplained variance in predicting empirical FC. How DWI and fiber tracking should be used for an optimal estimation of structural connectivity is still a topic of intense debates [45]-[48]. It is likely that part of the unexplained variance in predicting FC will be reduced as better estimates of SC become available.

The model showing the best results, the SAR model, explicitly modeled a stationary process with a stationary FC. In line with our hypothesis H4, empirical FC is likely to incorporate stationary components driven by SC. Further knowledge about this stationary process might be gained by analyzing FC computed over much longer periods of time than is commonly performed (e.g., hours versus minutes). This stationary process is itself likely to be only locally stationary, as it might be expected that slow physiological cycles, from nycthemeral cycles to hormonal cycles, development, learning and aging, will modify the parameters controlling it.

In the present study, we did not take into account the statistical fluctuations induced by the fact that the time series were of finite length. Such a finiteness entails that even a model that is stationary in the strong sense could generate sample moments that fluctuate over time. For instance, the sample sum of square of a multivariate normal model with covariance matrix Σ computed from time series of size N is not equal to NΣ but is Wishart distributed with N-1 degrees of freedom and scale matrix Σ. This phenomenon will artificially increase the part of variance that cannot be accounted for by stationary models and, hence, play against stationary models. Since it is conversely very unlikely for a non-stationary model to generate sample moments that are constant over time, statistical fluctuations cannot at the same time artificially increase the part of variance that can be accounted for by stationary models. As a consequence, not considering these statistical fluctuations made us underestimate the part of variance that can be accounted for by models that are stationary in the strong sense. In other words, our estimate of the part of variance accounted for by a stationary model is a lower bound for the true value. We can therefore be confident that taking statistical fluctuations into account will only strengthen H4.

Our goal here was to investigate how current generative models of brain acticity fare in predicting the relationship between structure and function. The complexity of some of these models was such that the simulations included here were only possible thanks to a computer cluster. The behavior of all these models depends on the values of some parameters and, in the present study, we set these parameters in agreement with the literature. In what measure this choice affects how well models predict FC is unclear. Yet a full investigation of this issue remains beyond the scope of this study, since parameter optimization through extensive exploration of the parameter space for all models is at this stage unrealistic. Nevertheless, in order to get a sense of the sensitivity of our results to parameter values in a way that is compatible with the computational power available, we explored the behavior of the Fitzhugh-Nagumo, Wilson and Kuramoto models over a subset of the parameter space (see Figures S5 and S6). We found that parameter values had little influence on predictive power, which, in all cases, remained below that of the SAR, the simplest model tested.

We formulated H2 to test for the existence of a relationship between complexity and realism in the models that we used. Indeed, there should exist a very tight connection between the two, since the more complex generative models in our study have been designed to take biophysical mechanisms into account, with parameters that are physiologically relevant and values often chosen based on prior experimental results. Now, realism usually comes at the cost of complexity. As a consequence, it is often (implicitely) assumed that, among the models we selected, the more complex a model is, the more realistic it will also be and the better it will fit the data. This is the reason why we stated H2, based on such rationale inspired from the literature, in order to put such hypothesis to the test. The results show that for the models we used, with their sets of parameters, an increase in complexity was not associated with an increase in performance. This suggests that, for these models, complexity and realism are not quite as tightly connected as expected.

Given that the SAR model is the only model that does not include a step of hemodynamic modeling (Balloon-Windkessel), it cannot be ruled out that the superiority of the SAR reflects issues with this step. In order to check that this is not the case, we computed predictive power for all models without the hemodynamic model. The predictive power was largely insensitive to the presence of the hemodynamic model (see Figure S7). In particular, the SAR model remained overall an upper bound in terms of predictive power.

Finally, we should note that we relied on a definition of SC restricted to the white matter compartment. Although this is standard in the field, in reality, local intrinsic SC exists in the gray matter. However, current models generally make prior assumptions about such SC. Moreover, intrinsic SC currently remains impossible to measure reliably for the entire brain.

In spite of the complexity of the problems and the limitations of current modeling approaches, computational modeling of large-scale brain dynamics remains an essential scientific endeavor. It is key to better understand generative mechanisms and make progress in brain physiology, physiopathology and, more generally, theoretical neuroscience. It is also central to the endeavor of searching for accurate and meaningful biomarkers in aging and disease [49]. Moreover, computational modeling of FC opens the possibility of making inference on specific biophysical parameters, including inference about the underlying anatomical connectivity itself. In spite of their limited predictive powers, simpler models can be useful in this context. The SAR model, introduced in [36], may appear well-suited to model essential stationary aspects of the generative mechanisms of FC. One interest of such a simple and analytically tractable model is that, beyond its very low computational burden, it could be the basis for straightforward estimation of the model parameters that can be used to compare clinical populations, and could constitute a potentially important biomarker of disease.

Methods

Ethics statement

All participants gave written informed consent and the protocol was approved by the local Ethics Committee of the Pitié-Salpêtrière Hospital (number: 44-08; Paris, France).

Data

Twenty-one right-handed healthy volunteers were recruited within local community (11 males, mean age 22±2.4 years). Data were acquired using a 3 T Siemens Trio TIM MRI scanner (CENIR, Paris, France). For acquisition and preprocessing details, see Text S1. For each subject, the preprocessing yielded three matrices: one of SC, one with the average fiber lengths, and one of empirical FC. These matrices were also averaged across subjects ('average subject').

Simulations

We used eight generative models with various levels of complexity: the SAR model, a purely spatial model with no dynamics that expresses BOLD fluctuations within one region as a linear combination of the fluctuations in other regions; the Wilson-Cowan system, a model expressing excitatory and inhibitory neuronal populations activity; the two rate models (with or without conduction delays), simplified versions of the Wilson-Cowan system obtained by considering exclusively the excitatory population; the Kuramoto model, which simulates neuronal activity using oscillators; the Fitzhugh-Nagumo model, which aims at reproducing complex behaviors such as those observed in conductance-based models; the neural-mass model, also based on conductance and with strong biophysiological constraints; and finally, the model of spiking neurons, the most constrained model in the current study which models neuron populations as attractors. For more details, see Text S2.

All models took an SC matrix as input, and all but the SAR were taken as models of neuronal (rather than BOLD) activity. Simulated fMRI BOLD signal was obtained from simulated neuronal activity by means of the Balloon-Windkessel hemodynamic model [50], [51]. Global mean signal was then regressed out from each region's time series. Finally, simulated FC was computed as Pearson correlation between simulated time series. For the SAR model, we directly computed simulated FC from the analytical expression of the covariance, see Equation (2) in Text S2. All models had a parameter that represents the coupling strength between regions. This parameter was optimized separately for each model on the average subject to limit computational burden ( Text S2). After optimization, we generated three runs of 8 min BOLD activity and averaged the corresponding FCs to obtain the simulated FC for each dynamical model and each subject. For the average subject, simulated FC was obtained by feeding the average SC matrix to the different models.

Performance

Modeling performance was assessed using predictive power and similarity of spatial patterns. Predictive power was quantified for each subject and for the average subject by means of Pearson correlation between simulated and empirical FC [29]. Regarding the similarity of functional brain networks, SC, empirical FC and simulated FC were decomposed into 10 networks using agglomerative hierarchical clustering and generalized Ward criterion [52]. The resulting networks from SC and simulated FC were compared to the ones resulting from empirical FC using the adjusted Rand index [53], [54]. The Rand index quantifies the similarity between two partitions of the brain into networks by computing the proportion of pairs of regions for which the two partitions are consistent (i.e., they are either in the same network for both partitions, or in a different network for both partitions). The adjustment accounts for the level of similarity that would be expected by chance only.

Analysis of dynamics

Empirical and simulated windowed FC were computed on individual subjects using sliding time-windows (increment of 20 s) of varying length (from 20 to 420 s by step of 20 s). Predictive power was computed as the correlation between any pair of time-windows of equal length corresponding to simulated and empirical windowed FC, respectively. This approach was only applied to the dynamical models; for SC alone and the SAR model, simulated FC remained, by definition, constant through time and, as a consequence, windowed FC was equaled to whole-session FC. The influence of simulated run duration on predictive power was also investigated. For each model, three runs of one hour were simulated on the average subject. Predictive power was then computed as a function of simulated run duration. For the same reason as above, SC alone and the SAR model did not depend on simulation duration.

Supporting Information

Performance of computational models on the replication dataset. The replication dataset was from the study of Hagmann and colleagues [55]. Brain network was defined at low anatomical granularity (N = 66 regions), and connectivity measures were averaged over five healthy volunteer subjects. (A) Predictive power for all connections and when restricted to intra/interhemispheric, direct/indirect connections. For each type of connections and each model, we represented the individual predictive powers (bar chart representing means and associated standard deviations), as well as the predictive power for the average subject computed using the original SC (diamonds), or after adding homotopic connections (circles). Of note, SC alone has no predictive power (zero) for the subset of indirect connections, by definition. (B) Patterns of SC, empirical FC and FC simulated from the SAR model for the average subject and associated scatter plot of simulated versus empirical FC (solid line represents perfect match). SARh stands for the SAR model with added homotopic connections. Matrices were rearranged such that network structure is highlighted. Homologous regions were arranged symmetrically with respect to the center of the matrix; for instance, the first and last regions are homologous. (C) Similarity of functional brain networks across subjects (bar chart with means and associated standard deviations), for the average subject (diamonds), and when adding homotopic connections (circles) (left). Network maps for the average subject and empirical FC, as well as for FC simulated using either the SAR model with original SC or the SARh.
https://doi.org/10.1371/journal.pcbi.1003530.s003
(EPS)

Exploration of the parameter space for the Fitzhugh-Nagumo model. (Left) Phase diagrams (i.e., x- y plane) for an uncoupled model ( k = 0) over various parameter values of α and β. The model operate mostly in an oscillatory regime for the range of parameter values investigated. (Right) Predictive power as a function of α and β. The black dot represents the parameter set used in our simulations, while the black square corresponds to the values from [28]. The values used in our simulations gave rise to higher predictive power than the parameters values from [28]. In any case, for the range of parameters considered, the predictive power always remained lower than that obtained with a SAR model.
https://doi.org/10.1371/journal.pcbi.1003530.s005
(EPS)

Effect of the hemodynamic model. Predictive power for all connections and when restricted to intra/interhemispheric, direct/indirect connections. For each type of connections and each model, we represented the predictive power for the average subject computed using the BOLD signal (diamonds, solid line) or using the neuronal activity (circles, dashed line). Of note, the prediction differs slightly from that of the Figure 1 due to the stochastic component of most models at each run.
https://doi.org/10.1371/journal.pcbi.1003530.s007
(EPS)

Acknowledgments

The authors are thankful to Olaf Sporns (Department of Psychology, Princeton University, Princeton, USA) and Christopher J. Honey (Department of Psychological and Brain Sciences, Indiana University, Bloomington, USA) for providing the neural-mass model; to Gustavo Deco, Étienne Hugues and Joanna Cabral (Computational Neuroscience Group, Department of Technology, Universitat pompeu Fabra, Barcelona, Spain) for providing the Kuramoto and rate models as well as the spike model; and to Olaf Sporns and Patric Hagmann (Department of Radiology, University Hospital Center and University of Lausanne, Lausanne, Switzerland) for sharing their data for replication. We would also like to thank them for fruitful discussions. The authors are grateful to Stéphane Lehéricy and his team (Center for Neuroimaging Research, Paris, France) for providing them with the data, and especially to Romain Valabrègue for his help in handling coarse-grained distributed parallelization of computational tasks.

Author Contributions

Conceived and designed the experiments: AM DR HB GM. Performed the experiments: AM DR GM. Analyzed the data: AM. Wrote the paper: AM DR HB GM.

References

1. 40. Schmahmann JD, Pandya DN (2006) Fiber pathways of the brain. Oxford University Press.

2. 52. Batageli V (1988) Generalized ward and related clustering problems. In: Bock HH, editor. Classification and Related Methods of Data Analysis. North-Holland. pp. 67-74.

# data[].attributes.attachment_link.title
Relating Structure and Function in the Human Brain: Relative Contributions of Anatomy, Stationary Dynamics, and Non-stationarities

# data[].attributes.attachment_link.description
Author Summary By analogy with the road network, the human brain is defined both by its anatomy (the 'roads'), that is, the way neurons are shaped, clustered together and connected to each others and its dynamics (the 'traffic'): electrical and chemical signals of various types, shapes and strength constantly propagate through the brain to support its sensorimotor and cognitive functions, its capacity to learn and adapt to disease, and to create consciousness.

# data[].attributes.body
Hebb's Theory- Doing repeated stimulations helped the cell assembly to be activated which helped with neuroprocessing.

# data[].attributes.body
Teaching for Consolidation-Teachers can organize information in a way that is easier for students to comprehend. They can also have the students rehearse things to help them remember information. They can elaborate on a topic to make it more meaningful to the students.

# data[].attributes.attachment_link.title
Introduction to the Neuroscience of Learning | lynda.com

# data[].attributes.attachment_link.description
Dr. Britt Andreatta introduces her presentation, The Neuroscience of Learning. Understand how the neuroscience of learning can help educators and learners alike tap into hidden potential. http://www.lynda.com/Education-Higher-Education-tutorials/Neuroscience-Learning/188434-2.html?utm_campaign=BEMTmumw8Rk&utm_medium=social&utm_source=youtube-earned Watch the entire 1 hour course in the lynda.com library. Get the latest research and inspiring examples that will help you tap into your hidden potential.

# data[].attributes.attachment_link.readable_text
Experiment 1

Participants completed two counterbalanced blocks of 180 trials. In a "control" block, participants only needed to apply a negligible amount of force (>5% MVC) to register their choices. In a separate "effort" block, choices required a greater amount of force. The precise amount of force required in this "effort" block was systematically varied across three experimental groups (>18%, 31%, or 44% MVC in separate groups of n = 32, 31, and 31, respectively). Block order ("control" vs "effort") was counterbalanced across participants, and each block was preceded by 15 practice trials to familiarize participants with the force requirements. On any given trial, one of the two stimuli had a 0.7 probability of being rewarded, and the other a 0.3 probability of reward. These reward contingencies reversed when participants reached a cumulative accuracy of 70% (after a minimum of 10 trials) or else after every 16 trials ( Park et al., 2010; Schlagenhauf et al., 2013).

Experiment 2

Experiment 2 was similar to experiment 1 except that effort requirements were manipulated within an experimental block, rather than across blocks. The same two stimuli were presented across all trials in experiment 2. One of these stimuli was designated the "low effort" stimulus, and required only a negligible amount of force to be selected (>5% MVC). The other was designated the "high effort" stimulus, which required a higher amount of force (>44% MVC). These stimulus-effort mappings remained constant for the duration of the experiment, and participants were explicitly informed about the identity of the low and high effort stimuli before the experimental task began.

To reinforce these stimulus-effort mappings, participants performed a preliminary block of 50 trials in which they were cued to generate the force corresponding to either the low or high effort stimulus (randomly determined). Participants then received binary feedback (correct vs incorrect) about whether they generated the correct amount of force (5-44% of MVC for the low effort stimulus, or >44% MVC for the high effort stimulus). To proceed to the experimental block, participants had to apply the correct force on at least 20 of the final 25 trials. All participants achieved this on their first attempt, with the exception of one participant, who achieved this on their second attempt.

The experimental task comprised two blocks of 150 trials, preceded by a practice block of 50 trials. As in experiment 1, participants were required to choose between the two stimuli on offer, but the key difference was that, here, they had to incorporate on every trial a consideration of the effort required to select each stimulus. Although the effort required to select a given stimulus was fixed, the value of each stimulus varied periodically. On any given trial, both stimuli could have a high probability of being rewarded (0.7); both could have a low probability of reward (0.3); or one could be superior to the other (0.7 vs 0.3). These contingencies changed after every 12 trials according to a pseudorandomized sequence that ensured the number of transitions and trials involving each combination of reward contingencies was the same for all participants ( Manohar et al., 2021).

Signal shift model (M2)

Previous studies suggest that the mere act of investing effort increases the overall value of acquired rewards ( Klein et al., 2005; Alessandri et al., 2008; Syed et al., 2016; L. Wang et al., 2017). This work suggests that effort should have a directional (positive) effect on RPEs, such that it should increase the amplitude of RPEs that are positive, and blunt those that are negative. We tested this prediction in a signal shift model (M1), which examined the effect of effort E on learning rate α. This model assumed a linear effect of effort on the signal gain parameter γ:

E is the peak amplitude of force applied on each trial (as a proportion of MVC), and k is a subject-specific parameter that captures individuals' sensitivity to effort. We let k take positive or negative values, such that denotes that effort shifts RPEs in a positive direction as predicted (i.e., boosts positive and blunts negative RPEs) and denotes the opposite (i.e., boosts negative and blunts positive RPEs).

Signal enhancement model (M3)

An alternative possibility is that effort amplifies RPEs in response to both positive and negative rewards (i.e., regardless of the valence of the outcome). Normative accounts of choice behavior describe effort as the cost of investing limited energy into one action at the expense of other candidate actions: an "opportunity cost" ( Niv et al., 2007; Shadmehr et al., 2019). Effort exertion would potentially offset such costs if it enhanced learning rates independent of the outcome. We tested this in a signal enhancement (M3) model by again estimating learning rate α as a function of both subject-specific signal gain γ and the amount of effort exerted E. However, in this model, effort modulates signal gain symmetrically, such that denotes that effort boosts RPEs (i.e., increases the amplitude of both positive and negative RPEs), and denotes that effort blunts RPEs (i.e., decreases the amplitude of both positive and negative RPEs):

Models incorporating separate learning rates for positive and negative RPEs (M4-M7)

M2 and M3 both assume that the magnitude of any effort-related effect on signal gain γ is equivalent for positive and negative RPEs. However, several studies have suggested that the neural processes underpinning positive and negative RPEs may be at least partially dissociable ( Bayer and Glimcher, 2005; Cools et al., 2008; Matsumoto and Hikosaka, 2009; Collins and Frank, 2014; Westbrook et al., 2020). Consequently, we fit a further family of models that aimed to decouple the effect of effort on different reward outcomes.

In addition, to examine whether effort had a weighted effect on learning as a function of RPE sign, we also fit a more complex model with separate effort parameters for positive and negative RPEs [dual learning rate (effort) model, M6]:

Finally, we included a model that assumes asymmetrical learning from positive and negative RPEs that is entirely independent of effort exertion [dual learning rate (no effort) model, M7]. This model includes separate parameters for positive and negative RPEs, in the absence of an effort ( k) parameter:

We constrained the signal gain parameter γ to the range [−5, 5], which ensured we allowed for the full range of learning rates (effectively 0.01 ≤ α ≤ 0.99) while also preventing extreme values close to the asymptotes. We used flat priors for all parameters. The best-fitting model parameters for each participant were found using maximum likelihood estimation, and we compared overall model fits based on the Akaike Information Criterion (AIC; Akaike, 1974). We also quantified the relative likelihood that the winning model best accounted for choice behavior compared with the others in the model space (i.e., Akaike weights): where is the Akaike weight of model i; is the difference in AIC between model i and the winning model; and M is the number of models in the space. Computational modeling was performed in MATLAB (R2018a, MathWorks).

To ensure that each of these models was uniquely identifiable, we performed a model recovery analysis based on synthetic data. We simulated behavior on a reinforcement learning task in which agents registered their choices by exerting effort. On each simulation, we randomly sampled subject-specific parameter values from a plausible range, as well as each agent's effort on each trial. We performed 50 simulations, each of which generated data from 100 learning agents making 200 choices each. This analysis revealed that our procedure was able to correctly identify the true generative model with an accuracy of ≥0.88 for each model ( Fig. 1 C).

Finally, we tested the reliability of the parameter estimates from the winning model in both experiments by performing a parameter recovery analysis. We generated data from the best-fitting parameter estimates for each participant, and sampled trial-by-trial effort exertion from a Gamma distribution fit using maximum likelihood estimation to the observed distribution of peak amplitudes for each subject. We then repeated our model-fitting procedure on these synthetic data and quantified the reliability of the parameter estimates as the rank-order correlation between the generative "true" value and the recovered value.

Effort increased the tendency to repeat rewarded choices

The aim of experiment 1 was to examine the effect of effort on learning, independent of any individual preferences to exert effort itself. First, we compared the effect of effort on choice accuracy across the three (low, medium and high) effort groups. For each participant, this effect was computed as the proportion of trials in which the more rewarded stimulus was selected in the effort block relative to the control block. A one-way ANOVA revealed a significant effect of effort ( F(1,91) = 6.22, p = 0.014; Fig. 2 C), which was primarily driven by greater relative accuracy in the high vs the low effort group ( p = 0.045, Bonferroni-corrected).

We investigated whether this effect on accuracy could be explained by differences in choice strategy. A GLM examining win-stay behavior found that trial-by-trial Effort was associated with a higher probability of choosing the same stimulus following a positive reward outcome (β = 1.08, SE = 0.47, p = 0.021; Fig. 2 D). Neither the simple effect of Block nor the Block × Effort interaction was a significant predictor (both p > 0.22). An analogous GLM on lose-switch behavior showed that effort was associated with a lower probability of switching following a negative reward outcome (β = −0.84, SE = 0.34, p = 0.014; Fig. 2 D). Again, neither Block nor the Block × Effort interaction was a significant predictor (both p > 0.11).

Together, these analyses suggest that effort may have a directional effect on reward signals during learning, boosting positive outcomes (promoting win-stay behavior), and blunting negative outcomes (reducing lose-switch behavior).

Computational models demonstrated that effort boosted positive and blunted negative RPEs

To test our key prediction that effort modulates learning, we compared a baseline Rescorla-Wagner model against alternative models that hypothesized distinct effects of effort on RPEs ( Fig. 1). Our model comparisons revealed that the signal shift model provided the best fit to the observed data, with an improved AIC relative to the baseline model of 223.32 units ( Fig. 3 A), and an Akaike weight in excess of 0.99 across the model space. Critically, k values derived from the model were significantly greater than zero (0.76 ± 0.28; Wilcoxon sign-rank test, W = 2964, p = 0.006; Fig. 3 B), demonstrating that the overall effect of effort was to boost positive RPEs, and blunt negative RPEs. These results provide a computational explanation for our earlier finding that greater effort was associated with more prominent win-stay behavior and less prominent lose-switch behavior. Post hoc comparisons revealed that this effect was driven by the medium effort group (1.48 ± 0.44, W = 408, p = 0.005, Bonferroni-corrected), rather than the low or high effort groups (both p > 0.16). A parameter recovery analysis confirmed that all parameters were reliably recoverable (parameter estimation reliability, γ = 0.86, β = 0.94, k = 0.67; p < 0.001 for all parameters).

Notably, the winning signal shift model provided a superior fit compared with the dual learning rate (no effort) model (ΔAIC = 25.39), indicating that effort was critical in increasing learning rates for positive RPEs, and reducing learning rates for negative RPEs. To further confirm that this result was not simply because of a differential effect of positive and negative RPEs on value-updating independent of effort, we ran a post hoc permutation test in which we randomly shuffled the effort exerted across trials for each participant. On each permutation, we compared model fits for the signal shift model based on the empirical data against the permuted data. Across 100 permutations, the empirical data resulted in superior model fits relative to the permuted data in every case ( p = 1, Z = 57.14, p < 0.001), confirming that effort played a critical role in modulating learning rate in response to positive and negative RPEs.

In summary, experiment 1 revealed that effort reinforced learning by shifting the RPEs in a positive direction. Notably, on every trial of this experiment, the effort required to select either stimulus was identical. This allowed us to focus solely on how effort affects the capacity to learn from choice outcomes, independent of an individual's aversion toward effort. An important question that remains is how learning is modulated by the willingness of an individual to invest effort in the first place. We addressed this question in a second experiment that was similar to the first, but in which the two stimuli presented on every trial were associated with different effort requirements. This allowed us to test the capacity of the winning signal shift model to account for effort on both the prospective and retrospective valuation of reward.

Choices between low and high effort stimuli demonstrated effort aversion

Participants displayed an overall aversion to effort, shown by a higher proportion of choices for the low compared with the high effort stimulus (0.53 ± 0.01, d = 0.45, t₍₄₅₎ = 3.1, p = 0.003; Fig. 4 C). This did not affect overall choice accuracy (mean difference = 0.01 ± 0.01, d = 0.13, t₍₄₅₎ = 0.97, p = 0.34, Fig. 4 D). Greater effort was associated with reduced win-stay behavior (β = −0.88, SE = 0.17, p < 0.001; Fig. 4 E), and increased lose-switch behavior (β = 0.46, SE = 0.1, p < 0.001; Fig. 4 E). Note that this contrasts with experiment 1, in which effort increased win-stay behavior and reduced lose-switch behavior. This is unsurprising, given that the aversiveness of the high effort stimulus ( Fig. 4 C) was likely to have obscured any subtler effects of effort on the RPE in a GLM. Therefore, to test whether these two possible effects of effort could be dissociated, we used a similar computational approach to experiment 1.

Computational models indicated that effort discounted value and shifted RPEs in a positive direction

The key difference between experiments 1 and 2 was that participants now had to balance an aversion to the high effort stimulus against their desire to maximize reward. To capture individuals' aversion to effort, we paired models from experiment 1 (M2-M7) with an effort discounting function, such that the discounted value of each stimulus ( v′) was computed as its learned value ( v), discounted by the amount of effort required to select it ( E), whereby E = 0.05 for the low effort stimulus and E = 0.44 for the high effort stimulus. Effort was scaled by a separate subject-specific effort discounting parameter (), which indexed each individual's aversion to effort:

As in experiment 1, learning rate varied as a function of trial-by-trial effort exertion, scaled by a free parameter capturing the effect of effort on RPEs (). Our model space in experiment 2 also included a baseline model (M1), as well as an effort discounting model (M8) that included an effort discounting function in the absence of any effect of effort on learning.

Model comparisons replicated our earlier findings by showing that the best fitting model comprised the same signal shift model of experiment 1, now paired with an effort discounting function to account for choices between the low and high effort stimulus on each trial. This model had an improved AIC of 156.02 units relative to the baseline model, and an Akaike weight in excess of 0.99 across the model space ( Fig. 5 A). Across the group, values were significantly greater than zero, demonstrating that effort discounted value before choices were made (0.32 ± 0.05; W = 1000, p < 0.001; Fig. 5 B). values also correlated with choice bias toward the low effort stimulus (Spearman's ρ = 0.36, p = 0.015; Fig. 5 E), confirming that this parameter successfully captured observed effort aversion. In addition, values across the group were significantly greater than 0, showing that effort boosted positive and blunted negative RPEs after choices were made, consistent with experiment 1 (0.63 ± 0.32; W = 833, p = 0.001; Fig. 5 B).

A parameter recovery analysis confirmed that all parameter estimates from the winning signal shift model were reliably recoverable (parameter estimation reliability, γ = 0.85, β = 0.91, = 0.62, = 0.60, p < 0.001 for all parameters).

Again, we note that the winning signal shift model provided a superior fit to the dual learning rate (no effort) model (ΔAIC = 12.53), confirming that effort played a significant role in modulating RPEs. To verify that the signal shift model was not merely approximating dual learning rates independent of effort requirements, we ran the same post hoc permutation test as in experiment 1, involving 100 permutations with effort data shuffled across trials for each participant. On every one of these permutations, the empirical data provided superior fits to the permuted data ( p = 1, Z = 26.3, p < 0.001), again pointing to a critical role for effort in increasing learning rates for positive RPEs, and reducing learning rates for negative RPEs.

The effect of effort on RPEs was greater in individuals who were more averse to effort

These results demonstrate that an individual's sensitivity to effort results in both a greater aversion to effort (i.e., greater effort discounting, ), and a greater positive shift in the direction of RPEs (). An obvious question is whether these two effort parameters are related. Indeed, we found a strong correlation between and values derived from the winning model (Spearman's ρ = 0.7, p < 0.001; Fig. 5 F). This indicates that, across our group of participants, the effect of effort on RPEs was directly related to the degree to which individuals found effort aversive.

# data[].attributes.attachment_link.description
Humans routinely learn the value of actions by updating their expectations based on past outcomes - a process driven by reward prediction errors (RPEs). Importantly, however, implementing a course of action also requires the investment of effort.

# data[].attributes.body
Give simple and straightforward directions when giving directions to students. Do not be too wordy and be concise.

# data[].attributes.attachment_link.readable_text
Abstract

Although many scholars deem non-invasive measures of neurophysiology to have promise in assessing learning, these measures are currently not widely applied, neither in educational settings nor in training. How can non-invasive neurophysiology provide insight into learning and how should research on this topic move forward to ensure valid applications? The current article addresses these questions by discussing the mechanisms underlying neurophysiological changes during learning followed by a SWOT (strengths, weaknesses, opportunities, and threats) analysis of non-invasive neurophysiology in learning and training. This type of analysis can provide a structured examination of factors relevant to the current state and future of a field. The findings of the SWOT analysis indicate that the field of neurophysiology in learning and training is developing rapidly. By leveraging the opportunities of neurophysiology in learning and training (while bearing in mind weaknesses, threats, and strengths) the field can move forward in promising directions. Suggestions for opportunities for future work are provided to ensure valid and effective application of non-invasive neurophysiology in a wide range of learning and training settings.

Keywords: learning, training, neurophysiology, brain activity, heart rate, eye tracking, skin conductance, respiration

Introduction

While behavioral methods, such as pre- and post-test assessments, are most commonly used to assess learning, non-invasive neurophysiological methods provide promising complementary options ( Leff et al., 2011; Lai et al., 2013; Krigolson et al., 2015; Tinga et al., 2019a). Non-invasive neurophysiology includes measures that provide insight into the nervous system through relatively non-invasive sensors placed on the body or in the environment. Examples of these measures are heart rate, electrodermal activity (EDA) and electroencephalography (EEG).

Although many scholars deem these measures to be promising in assessing learning, they are currently not widely applied in educational settings and in training. This article aims at providing insights that help the field move toward valid and effective application of neurophysiology in a wide range of learning and training settings. In the current article learning is considered as processing of information from experiences in order to update system properties ( Barron et al., 2015). Through experience the ability to execute tasks or operations or processing of information improves and becomes faster, less effortful and more automatic ( Schneider and Chein, 2003; Borghini et al., 2016). A large body of cognitive and neurocognitive studies demonstrates that learning manifests itself in different ways, such as the acquisition of new information, the development of perception, reasoning, psychomotor skills, and problem solving skills ( Lai et al., 2013). When referring to learning in the current article we specifically refer to psychomotor and cognitive learning, but generalize to other types of learning. Affective learning, such as fear conditioning and fear extinction, however, is not included as this type of learning falls outside of the scope of the current article, as studies on this topic primarily reflect the affective response to stimuli. To gain insight into how we can move in the direction of valid and effective application of neurophysiology in a wide range of learning and training settings, it is first of all important to know the underlying mechanisms of how non-invasive measures of neurophysiology are able to inform on learning. To this aim we will first discuss the most frequently applied non-invasive measures and into what aspects of the nervous system these measures provide insight. Subsequently, we will discuss how activity in the nervous system, in turn, can provide insight into learning processes.

The nervous system consists of the autonomic and central nervous system. The autonomic nervous system has two branches that operate at the same time, but in an antagonistic way: the sympathetic nervous system and the parasympathetic nervous system, in which the sympathetic branch activates a neurophysiological response, while the parasympathetic branch inhibits one ( Berntson et al., 1991), with parasympathetic nerves exerting their effects faster than sympathetic ones ( Nunan et al., 2010). The nervous system regulates a wide variety of functions related to processes such as homeostasis, digestion and attention ( Berntson and Cacioppo, 2000; Mauss and Robinson, 2009).

Non-invasive methods providing insight into the autonomic nervous system include measures of peripheral physiology such as heart rate (variability), respiration, EDA and electromyography (EMG). Heart rate and heart rate variability (i.e., the variation in the intervals between consecutive heartbeats) can be measured either via electrocardiography (ECG) or photoplethysmography (PPG). While ECG is more conventional, PPG has the benefit of being relatively less invasive. With ECG multiple sensors are placed on the body to measure the heart's electrical activity, while with PPG a single optical sensor is placed on the finger or earlobe in order to measure peripheral changes in blood flow which are affected by activity of the heart. Although ECG is considered to be more precise than PPG, especially in measuring heart rate variability, the two measures are highly correlated under ideal conditions ( Lu et al., 2009). Respiration can be measured through a respiration belt around the upper body measuring the body's expansion and contraction. EDA refers to variations in the electrical properties of the skin in response to sweat secretion. A non-invasive measure of the change in conductance of the skin can be acquired by applying a low constant voltage across two electrodes placed on the skin ( Fowles et al., 1981). EMG provides insight into muscle activity as this method measures electrical activity that is generated by muscle fibers. While the heart and the lungs are connected to the sympathetic as well as to the parasympathetic nerves, the skin and its sweat glands and skeletal muscles are solely innervated by the sympathetic nerves ( Roatta and Passatore, 2008; Setz et al., 2010).

The central nervous system refers to the part of the nervous system consisting of the brain and the spinal cord. Brain activity can be measured non-invasively through measures such as EEG and functional near-infrared spectroscopy (fNIRS). fNIRS is a method in which cortical hemodynamic activity is measured using near-infrared light and is based on the assumption that neural activation and the vascular response are coupled ( León-Carrión and León-Domínguez, 2012). EEG records cortical electrical activity non-invasively through electrodes on the scalp ( Antonenko et al., 2010). EEG is applied more often than fNIRS, as fNIRS is still a relatively new measurement technique ( Kopton and Kenning, 2014). Compared to EEG, fNIRS has a better spatial resolution but a reduced temporal resolution ( Crosson et al., 2010; Zama and Shimada, 2015). Other well-established techniques of measuring brain activity include functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). However, these measures tend to be more invasive ( Tinga et al., 2019a) and therefore fall outside the scope of the current review.

Eye-related measures are another category of neurophysiological measures. The eye is affected both by the central and autonomic nervous system; the retina is an integral part of the central nervous system, while the muscles that control eye-movements, eyelid elevation and pupil dilation are innervated by both branches of the autonomic nervous system ( McDougal and Gamlin, 2015; Eckstein et al., 2017; Larsen and Waters, 2018). Eye-related measures can be collected non-invasively through an eye-tracker either placed on a table in front of or mounted on the head of the trainee. Alternatively, sensors can be placed on the skin close to the eyes to measure the electrical activity that is paired to the eye's position and movements, a technique called electrooculography (EOG). Eye-related measures include (1) pupil dilation; (2) eye gaze; (3) blink rate; and (4) blink duration ( McDougal and Gamlin, 2015; Eckstein et al., 2017). Why would the activity in the nervous system measured through neurophysiology be related to learning processes? Measures of neurophysiology respond in a predictable manner to cognitive demand/mental workload or mental effort ( Gevins and Smith, 2003; Antonenko et al., 2010; Setz et al., 2010; Ayaz et al., 2012; Brouwer et al., 2012; Borghini et al., 2014; Hogervorst et al., 2014; Reiner and Gelfeld, 2014; Gable et al., 2015; Charles and Nixon, 2019). As mental effort increases, sympathetic activation increases and parasympathetic inhibition decreases ( Berntson et al., 1991), paralleled by changes in the central nervous system such as in oscillations in the EEG signal with alpha oscillations generally increasing and theta oscillations generally decreasing with decreasing demands ( Klimesch, 1999; Klimesch et al., 2005; Antonenko et al., 2010; Brouwer et al., 2012) and in activation patterns in the fNIRS signal with activity in the (pre)frontal cortex generally increasing ( Ayaz et al., 2012; Hiyamizu et al., 2014).

According to several influential cognitive theoretical frameworks learning can be thought of as a change in how a to-be-learned task is processed and therefore in how cognitively demanding a task is. The dual processing theory constitutes one of these frameworks. From the dual processing perspective, learning can be thought of as a transition from controlled to automatic processing ( Schneider and Chein, 2003). Controlled processing is slow, effortful and attentionally demanding and is comparable to a high mental effort. Automatic processing, in contrast, is fast and occurs in the absence of control and attention and is comparable to low mental effort. Evidence for the transition from slow controlled processes to faster automatic processes comes from behavioral findings demonstrating that the beginning of the learning process is characterized by slow and often inaccurate responses ( Schneider and Shiffrin, 1977) and that when learning advances the time needed for task execution decreases ( Ritter and Schooler, 2001).

Similarly, from the perspective of cognitive load theory learning can be seen as the process of schema acquisition and schema automation ( Schnotz and Kurschner, 2007), whereby items that were first processed separately become integrated within one schema and regarded as a single item. In this way, fewer items need to be held in working memory, reducing mental effort. Processing of the acquired schema can in turn become automatic during further learning. This schema automation reduces working memory demands or demands on cognition even more.

Mental effort additionally increases with increased task difficulty ( Backs and Seljos, 1994; Hockey, 1997; Brouwer et al., 2014) and task difficulty is thought to modulate learning-related changes in neurophysiology ( Wang et al., 2010). In general, execution of a task becomes less difficult during learning, eliciting changes in mental effort and therefore neurophysiology ( Fairclough et al., 2005). Moreover, according to motivational intensity theory mental effort is proportional to the level of task demand provided that success is possible and successful performance is worthwhile ( Brehm and Self, 1989). This has been demonstrated on several neurophysiological measurements as well ( Richter et al., 2008, 2016; Fairclough and Ewing, 2017), implying that success needs to be possible in order for learning to occur and for mental effort and neurophysiology to change in a predictable way during learning.

The topic of learning consists of various aspects of development in cognition. It is important to take into account that different aspects of learning progress differently and mostly do not affect cognitive development in a linear fashion ( Schneider and Chein, 2003; Luft and Buitrago, 2005; Murre, 2014; Stoianov et al., 2016). Even though cognitive development does not necessarily progress in a linear fashion, it is clear that important cognitive changes take place during learning, transitioning from more controlled and demanding processing to less demanding automation, which can be detected by neurophysiological measures. Experimental evidence comes from studies demonstrating changes in neurophysiology suggesting a decrease in mental effort over time during learning (for an overview of 113 experiments from 2006 to 2016 and a first model based on the findings from these experiments see Tinga et al., 2019a). Generally, in such experimental studies, a learning task is presented in which improvements over time (e.g., trials or blocks) on behavioral outcomes occur and the effect over time on simultaneously recorded neurophysiological measures is assessed and related to the behavioral learning process. For example, in a recent study including two experiments ( Tinga et al., 2020) participants were presented with a visuomotor sequence learning task while behavioral measures and measures of EEG, skin conductance, heart rate (variability) and respiration were collected. Results indicated that, in both of the two experiments, skin conductance level and EEG oscillations (in the alpha and gamma band) changed during general task learning indicating less mental effort over time and were related to behavioral performance. These findings suggest neurophysiology is able to provide robust insights in learning. Moreover, several studies have compared learning and non-learning task conditions and report a clear difference on neurophysiology ( Krigolson et al., 2009; Alain et al., 2010; Takeuchi et al., 2011; Moisello et al., 2013; Tan et al., 2014), supporting that there is potential in these measurements as they are able to dissociate learning from non-learning.

Different measures of neurophysiology all provide insight into activity in a part of the nervous system, yet there are important differences in specific challenges associated with each measurement technique. For example, measurements of heart rate variability are sensitive to whether the trainee is sitting, standing or lying down ( Acharya et al., 2005). As another example, measures of pupil size are affected by the luminance of the surroundings ( Höfle et al., 2008). Even though important differences between the different neurophysiological measures exist, we will consider all measures together in this review. These different measures of neurophysiology are rarely considered together, while it has been pointed out that it is important to discover what specific aspects of the learning process are reflected in specific neurophysiological changes ( Tinga et al., 2019a). Moreover, different measures of neurophysiology can potentially be combined to provide better insight into learning than a single measure (cf. Wilson and Russell, 2003, 2007). However, where needed, important differences between different measures of neurophysiology will be discussed.

Even though neurophysiological measures have potential to provide insight into learning, the question remains how research on this topic should move forward to ensure valid applications. The nature of this current overview does not allow to go into the details of methodological issues inherent to the studies reviewed, but will instead assume the importance of these peer-reviewed articles and focus on how to best move the field forward.

A Swot Analysis for the Field of Non-Invasive Neurophysiology in Learning and Training

To gain a better understanding of how the field of non-invasive neurophysiology can best move forward, it is important to comprehensively assess the current state of the field. To this aim, we will identify strengths, weaknesses, opportunities, and threats (SWOT) for the field of non-invasive neurophysiology in learning and training. Although the SWOT framework is most commonly employed in business for analyzing factors that influence a company's position in the marketplace with a focus on the future, it can also be useful for other domains such as in a scientific field ( Rizzo and Kim, 2005). Before we discuss each of the strengths, weaknesses, opportunities and threats, we present a summary of the analysis as a guideline ( Table 1 ).

TABLE 1

Strengths

Weaknesses

* Formative assessment providing insight into the process of learning
* Supports predicting the outcome of learning
* Objective assessment
* Supporting learning through instruction or feedback
* Tailoring learning task to the individual in real-time
* Multi-channel data collected at a high sample rate
* Deeper insights into learning through rich data
* Flexible and precise application and measurement

* Neurophysiology reflects a wide variety of functions supported by the nervous system
* Neurophysiological changes during learning are sensitive to individual differences and task-related aspects
* Signal noise
* Limited freedom of movement of trainee (wired/uncomfortable equipment)
* Technical challenges (synchronization and analysis of large amounts of data from multiple data streams)
* Personnel training, device maintenance, preparations, and procedures needed for high quality data
* Hardware and software expenses

Opportunities Threats

* The potential of neurophysiology in learning has been pointed out by multiple scholars
* The research field is highly interdisciplinary, bringing knowledge together from experts from different fields
* General changes in neurophysiology do not require precisely time-locking signals to repeated exposure of specific events, allowing for application in a wider range of contexts
* Solutions for dealing (automatically) with noise
* Real-time data analysis and visualization
* Development of adequate wireless sensors (e.g., wearables) that take less set-up time and are more comfortable to wear
* Reduction of costs
* Neurophysiological sensors embedded in learning systems/widely used devices/promising new technologies

* Mostly applied in controlled lab environments using carefully structured tasks with simple stimuli
* Focus has been mainly on specific learning topics; unclear how generalizable current findings are across different learning tasks
* Focus has been mainly on specific neurophysiological measures; unclear how generalizable current findings are across a wide range of measures
* Doubts about the validity and applicability of neurophysiological measures in learning have been raised
* Ethical challenges such as data privacy
* Trainee can feel uncomfortable being measured through sensors
* Non-experts tend to trust conclusions from neurophysiological data even if they are unwarranted

Strengths in Applying Neurophysiological Measures in Learning and Training

Formative and Objective Assessment Supporting Predicting the Outcome of Learning

Learning can be assessed in a summative or formative way. Summative assessment focuses primarily on assessing the outcome of learning while formative assessment aims to gain insight into learning processes ( Stobart, 2008). While summative assessment informs on whether learning is sufficient, formative assessment additionally allows for supporting learning based on the examination of the learning process itself. Learning is commonly evaluated using summative measures of behavioral performance by for example using a post hoc written examination ( Weinstein, 2001) focused on the outcome of learning while neglecting the learning process itself. Having insight into the learning process could even support predicting the outcome of this process even before someone has finished learning ( Cowley et al., 2013; Vecchio et al., 2019).

Additionally, assessment needs to be objective, valid, and reliable ( Brenner et al., 2015). Post hoc behavioral measures might lack objectivity and validity as they might be affected by the process of measurement itself with participants changing their behavior on purpose or without being conscious about it when they are able to detect that specific behaviors are being measured ( Page et al., 1966). Non-invasive neurophysiological measures can be an interesting substitution or addition to behavioral outcomes of learning by being a formative assessment tool providing insight into the learning process online in an objective way.

Supporting Learning Through Instruction or Feedback and Tailoring Task to the Individual

Providing feedback during learning can greatly benefit learning ( Hattie and Timperley, 2007; Sigrist et al., 2013). As neurophysiology is a formative measure, it can be employed for real-time support of learning. When measures indicate that the trainee is having difficulty with learning, the trainer can provide additional instruction and feedback. Potentially, automated feedback as a response to incoming neurophysiological data and additional task data could even be implemented in the learning task itself. The continuous stream of objective data could also be used for personalization of the learning environment to the needs of the individual in order to ensure that learning is neither too easy nor too difficult. For example, the learning task could be made more challenging when the trainee is learning sufficiently, but when learning is insufficient more challenging material could be presented at a later point in time or the task could be made simpler. Walter et al. (2017) studied this principle using EEG data during arithmetic learning in students. They adapted the difficulty of the material to the learner's workload and concluded that such an adaptive learning environment is feasible. Additionally, systems are being developed in which eye tracking data is employed in order to track learning and support learning through feedback and adaptation in e-learning environments ( Barrios et al., 2004; Song et al., 2015).

Multi-Channel Data Collected at a High Sample Rate Providing Rich Data

Non-invasive neurophysiology allows for measuring through multiple channels within one modality as well as across a range of modalities. Channels can be combined in order to reach conclusions of enhanced reliability as it has been demonstrated by Wilson and Russell (2003, 2007) that mental effort could be classified using a combination of neurophysiological measures (however, see Hogervorst et al., 2014; Tinga et al., 2020 who did not find that combining different neurophysiological measures improved assessment of mental effort and learning specifically). Besides the potential of enhancing reliability, if one channel fails to record sufficient data there are other channels to fall back on. Additionally, data can be collected at a high sample rate during the learning experience. For example, the sample rate typically ranges from 250 to 2000 Hz for EEG ( Reis et al., 2014), from 25 to 2000 Hz for eye tracking ( Reis et al., 2014), and from 500 to 1000 Hz for heart rate ( Kwon et al., 2018). Although a higher sample rate does not always provide extra information ( Reis et al., 2014; Kwon et al., 2018), a relatively higher sample rate allows for a precise description of processes over time. Data provided by frequently sampled multi-channel neurophysiological methods is therefore much richer than a single post hoc behavioral measure such as an exam score.

Flexible and Precise Application and Measurement

Another benefit of neurophysiological methods is that there is flexibility in where each sensor will be placed with multiple ways in which one modality can be measured while still recording with high precision. When measuring brain activity through EEG or fNIRS for example, one can decide to place sensors only at specific locations on the scalp or on locations distributed over the scalp. EEG sensors have an outstanding temporal resolution providing insights in the temporal dynamics of neuronal activity below 100 ms. Additionally, EEG sensors allow for source localization with adequate precision, especially from 64 electrodes, with a higher number of electrodes not being necessary ( Baillet et al., 2001; Michel et al., 2004; Michel and Murray, 2012). For applications other than source localization, it is not required to include a large number of electrodes for measuring EEG, with studies reporting successful mental effort assessment through a single EEG sensor ( Mak et al., 2013; Hogervorst et al., 2014). Regarding measurements such as heart rate and EDA, different configurations of electrode placement are possible ( Francis, 2016), providing flexibility across a range of tasks. For example, heart rate, while traditionally measured via sensors on the chest, can alternatively be measured on the ankle/foot ( Díaz and Pallas-Areny, 2010; Jarchi and Casson, 2016). As EDA works by measuring the change in conductance of the skin ( Fowles et al., 1981), several different placement locations are possible. Cameras collecting eye tracking measures can be integrated into glasses worn by the trainee or can be positioned at a distance from the trainee ( Meißner et al., 2019), as long as the cameras can reliably detect the eyes. Alternatively, sensors can be placed on the skin close to the eyes (EOG). Thus, there is a lot of flexibility in the type of sensors and the location of placements of the sensors, with measurements being able to be highly precise when they are suited to the goal of the measurements.

Weaknesses in Applying Neurophysiological Measures in Learning and Training

Neurophysiology Is Reflective of a Wide Variety of Functions Supported by the Nervous System

Besides the important strengths associated with applying non-invasive measures of neurophysiology in learning and training, there are also important weaknesses. One weakness is inherent to the general-purpose nature of the nervous system. Its activity is not reflective of one process, but rather of a wide variety of functions related to homeostasis, digestion, attention, and so forth ( Berntson and Cacioppo, 2000; Mauss and Robinson, 2009), which is an important point to keep in mind when interpreting measures reflective of nervous system activity. Although learning affects neurophysiology and these effects have been detected across several studies ( Krigolson et al., 2009; Alain et al., 2010; Takeuchi et al., 2011; Moisello et al., 2013; Tan et al., 2014), neurophysiology is additionally sensitive to many learning-irrelevant processes and events. For example, pupil size is determined to a great extent by luminance of the surroundings making it important to control for luminance when interpreting pupil size data ( Höfle et al., 2008). Effects of ambient light have also been reported for other neurophysiological measures, such as heart rate, with effects depending on the time of day ( Ruger et al., 2006). Additionally, effects of emotion and stress ( Mauss and Robinson, 2009) and mental fatigue ( Gergelyfi et al., 2015) on measures of neurophysiology have been reported frequently. As learning is a process involving prolonged cognitive activity, mental fatigue can be induced in the trainee. Learning and mental fatigue are thought to both progress over time and therefore disentangling the two is important in order to determine which changes in neurophysiology are truly reflective of learning. Gergelyfi et al. (2015) experimentally demonstrated (as the first to do so according to the authors) that pupil size, skin conductance, heart rate, and EEG were unaffected by mental fatigue while blink rate and heart rate variability were affected by mental fatigue. These findings suggest that different neurophysiological measures are affected by mental fatigue in different ways, although more research is needed in order to be able to deal with effects of mental fatigue sufficiently.

Neurophysiological Changes During Learning Are Sensitive to Individual Differences and Task-Related Aspects

Changes in neurophysiological outcome measures during learning are also affected by individual differences and aspects of the learning task ( Tinga et al., 2019a). For example, several studies have demonstrated changes in brain activity as assessed through EEG during learning to be different between young and older adults ( Alain and Snyder, 2008; Eppinger et al., 2008; Pietschmann et al., 2008, 2011; Eppinger and Kray, 2011). Effects of age on learning-related changes in neurophysiology have not yet been studied for other outcome measures than measures of brain activity. It therefore is unclear whether age also affects measures of peripheral physiology. Neurophysiological changes during learning are also reported to be affected by the sensory system in which learning takes place, the number of learning days, and whether feedback on performance is provided ( Tinga et al., 2019a). However, when controlling for individual differences and for task-related aspects, effect sizes for measuring learning through neurophysiological outcome measures become as large as effect sizes for measuring learning through behavioral outcome measures. This means that even though the sensitivity of neurophysiology might make assessment somewhat more complicated, neurophysiology can provide just as powerful insights in learning as behavior. Potentially, the sensitivity of neurophysiological outcome measures to individual differences and task-related aspects could even provide for valuable insights into learning processes in addition to insights obtained through behavioral measurements.

Signal Noise and Limitation of Freedom of Movement of the Trainee

Non-invasive neurophysiological measures can be noisy, especially outside of the controlled environment of the lab. Most techniques provide the most valid outcomes under ideal conditions. Commonly encountered noise sources are electrode motion and muscle contraction artifacts, usually caused by movements of the trainee ( Vazquez et al., 2012; Poungponsri and Yu, 2013). The sensitivity to motion and muscle activity can be problematic for techniques using electrodes placed on the body, especially when measuring signals of lower amplitudes, such as EEG. When collecting eye-related measures using an eye tracker, muscle and motion artifacts are non-existent as no electrodes are placed on the body of the trainee, while other sources of noise can arise such as movements of the eye tracking camera with respect to the eye and changes in corneal reflection ( Li et al., 2008). To deal with artifacts, in research on neurophysiology in learning the trainee is generally presented with a seated task and instructed to remain seated as still as possible. This brings us to another weakness, namely that of the limitation of freedom of movement of the trainee, which can, besides by being induced by the design of the task, be induced by neurophysiological measuring equipment worn by the trainee. Most such equipment is wired, and, depending on the type of measurement collected, can be uncomfortable to wear ( Liao et al., 2012), affecting the trainee's mobility.

Technical Challenges and Personnel Training

Technical challenges also arise when applying neurophysiology in learning and training, as collection, synchronization, and analysis of large amounts of data from multi-channel data streams are required ( Dahlstrom-Hakki et al., 2019). Technological advancements are making these requirements increasingly easier, yet overcoming these technical challenges as much as possible is time-consuming as things such as time desynchronization between multiple devices and delays between the device and scripts occur frequently ( Dahlstrom-Hakki et al., 2019). Sufficient attention and time should be given to personnel training and device maintenance and extra time should also be allocated for experimental preparations and procedures during testing to ensure that data of high quality is collected ( Kivikangas et al., 2011).

Hardware and Software Expenses

Besides the costs associated with the time-investment in personnel training, maintenance of devices and experimental preparations and procedures, hardware, and software for non-invasive neurophysiological measurements can also be expensive. For example, costs of state-of-the-art eye tracking equipment range between US$ 20,000 and US$ 30,000 ( Ferhat and Vilariño, 2016) and EEG equipment can cost more than US$ 75,000 ( Krigolson et al., 2017). Costs of sensors measuring peripheral physiology such as heart rate, respiration and EDA can also be high ( Lucena et al., 2015), with a complete research system including hardware and software easily reaching US$ 10,000 per modality.

Opportunities for Applying Neurophysiological Measures in Learning and Training

Potential Is Recognized and the Field Is Highly Interdisciplinary

Applying non-invasive measures of neurophysiology opens up many opportunities. Multiple scholars have pointed out the potential of neurophysiology in learning and training. For example, Lai et al. (2013) reviewed 113 studies evaluating the effect of learning in general on eye-related outcome measures and indicated that they are promising in assessing learning within various themes with various topics. Regarding research on brain activity in learning, a review of about 90 studies on EEG changes during motor learning ( Krigolson et al., 2015) concluded specific event-related brain potentials (ERPs) are able to provide important insights into learning. Additionally, Leff et al. (2011) assessed over 80 studies on brain activity measured via fNIRS during motor tasks and pointed out that this modality is promising for studying motor learning. Tinga et al. (2019a) reviewed 113 experiments examining a wide range of non-invasive neurophysiological measures during learning (including measures of brain activity, eye-related outcome measures, measures of heart rate, skin conductance and respiration) and demonstrated that effect sizes were large for neurophysiological changes during learning, although they were smaller than for behavioral changes. Yet, the difference in effect sizes between neurophysiology and behavior disappeared when controlling for task-related aspects and individual differences. Again, the authors concluded that neurophysiology is promising for the assessment of learning. Although most work on neurophysiology in learning is conducted in the controlled environment of the lab, research has also demonstrated neurophysiological measures can be successfully applied for examining learning in real-life scenarios such as in a classroom ( Ko et al., 2017; Poulsen et al., 2017; Bevilacqua et al., 2019; Dahlstrom-Hakki et al., 2019).

The research field of neurophysiology in learning and training is also highly interdisciplinary, which allows for combining knowledge from experts in fields such as sensor technology, signal processing, artificial intelligence, neurophysiology, and engineering. Of course, it is a challenge to oversee and bring together the research in all these areas of expertise ( Brouwer et al., 2015), but when one brings together people from these different areas we believe that this will greatly benefit formulating solutions.

Employing General Changes in Neurophysiology

The feasibility of applying measures of neurophysiology outside the lab increases, when examining changes in neurophysiology reflective of general changes in mental effort, instead of examining specific responses to specific stimuli as is the case for example with ERPs from EEG. Specific responses can only be computed when a task itself presents many similar stimuli that can elicit an ERP that can be reliably time-locked to those stimuli, or when a task-irrelevant probe can be repeatedly presented during learning ( Gevins and Smith, 2003). Therefore, measures of more general changes in neurophysiology over time that do not need to be precisely time-locked after repeated exposure of a specific event, such as changes in frequency bands in EEG, pupil size or heart rate, are presently better suited for application in real-world learning and training contexts, opening up many possibilities.

Solutions for Dealing With Noise and Real-Time Data Analysis and Visualization

As discussed above, neurophysiological techniques generally provide the most valid outcomes under ideal conditions. Yet, solutions for dealing (automatically) with noise are available and are being enhanced ( Vorobyov and Cichocki, 2002; Vazquez et al., 2012; Poungponsri and Yu, 2013), increasing validity and applicability. The rapid developments in machine learning contribute to the processing (e.g., enhancing signal to noise ratio) and analysis (e.g., exploiting multiple modalities) of neurophysiological measures ( Müller et al., 2008; Lemm et al., 2011).

Moreover, there are several ways in which neurophysiological data can be analyzed, with some applied studies having proposed a simple form of analysis. For example, in analyzing eye tracking data one can look at the percentage of visual attention allocation across regions of interest in the visual field, which is easier to perform even with eye-trackers with a lower sample rate and facilitates interpretation ( Dahlstrom-Hakki et al., 2019). Yet, simplifying analyses can come at the cost of having less deep insight into collected data than more advanced analysis techniques. Advanced analyses are however becoming more accessible through the increased availability of powerful software tools. Examples of relatively frequently used open source software tools for aiding in analyzing brain activity signals are FieldTrip ( Oostenveld et al., 2011), Brainstorm ( Tadel et al., 2011), and MNE ( Gramfort et al., 2013).

Neurophysiological signals can also be analyzed and visualized in real-time. For example, it is possible to analyze EEG data online taking into account individual variations ( Guger et al., 2000) and to visualize the data using open-source software ( Mullen et al., 2013). Another example is real-time analysis and monitoring of neurophysiology via a cloud system which has shown to be accurate and effective for ECG ( Xia et al., 2013). These advancements aid the ease and speed at which data can be assessed and support providing the trainee with effective feedback and tailoring the learning experience to the individual.

Development of Wireless Sensors and Reduction of Costs

Advancements to enhance the trainee's comfort and ease of movement are also being made with wireless and more compact sensors already being made available. For example, wearable wristbands such as the Empatica E4 and the Everion (Biovotion) are able to measure multiple peripheral neurophysiological outcomes and systems such as B-ALERT (Advanced Brain Monitoring) and g.Nautilus (g.tec) provide wireless EEG measures. These newer devices require considerably less set-up time and are less sensitive to motion artifacts. Sensors are even becoming fully portable with the possibility for data collection to occur via a smartphone or tablet, even for measures of brain activity ( Stopczynski et al., 2014; Poulsen et al., 2017). All these devices promise to provide medical grade data and several studies have provided evidence of their successful application ( Amaral et al., 2017; Corino et al., 2017; Barrios et al., 2018; Beeler et al., 2018; Tinga et al., 2019b).

In general, wearables and more applied systems are substantially cheaper, easily being 5-20 times less costly to purchase. Together with the reduction of set-up time and advanced solutions for dealing with noise and enhancing data analysis, the total costs involved for being able to apply neurophysiology in learning and training are greatly reduced.

However, the performance of these systems is in many instances still not on par with more traditional lab-based systems due to limitations such as a lower number of channels, less precise application and lower sample rate ( Hairston et al., 2014; Melnik et al., 2017). Yet, we expect technological advancements in the upcoming years to substantially reduce the difference in data quality between lab-based systems and wearables, making these wearables increasingly interesting to apply.

Embedding Sensors in Learning Systems and Widely Used Devices

Another interesting direction for future developments is to embed equipment measuring neurophysiology in learning systems. Such systems could then potentially be applied to diagnose learning states and provide instantaneous help or adapt the material based on the recorded neurophysiological data ( Lai et al., 2013). One category of systems that has high potential in education and training is virtual reality (de Back et al., submitted) and existing virtual reality headsets and to-be developed headsets with integrated eye tracking and brain activity sensors can contribute to this potential ( Jantz et al., 2017). As an example, Sood and Singh (2018) proposed a framework on combining virtual reality and EEG in an educational game for robust and resilient learning with an enhanced quality of the learning experience.

Additionally, integrating sensors in smartphones, laptops, and tablets might also be an interesting endeavor, with for example advancements being made in using cameras of such devices for collecting eye-related measures ( Lin et al., 2013; Lim et al., 2015; Krafka et al., 2016; Meißner et al., 2019) and measures of heart rate and heart rate variability ( Heathers, 2013; Lucena et al., 2015).

Threats for Applying Neurophysiological Measures in Learning and Training

Measures Are Mostly Applied in Lab Settings Using Controlled Tasks

Although applying neurophysiology in learning and training brings along interesting possibilities, there are important threats involved that also need to be considered. To start with, studies examining neurophysiology in learning are mostly fundamental, being conducted in the controlled environment of the lab using carefully selected tasks with simple stimuli ( Dahlstrom-Hakki et al., 2019; Tinga et al., 2019a). This fundamental work is obviously important to understand in what way and under what circumstances neurophysiology provides insight into learning. Yet, if we want to be able to assess and enhance learning in the real-world, outside of the lab, we would benefit from combining this fundamental knowledge with more applied knowledge.

Focus on Specific Learning Topics and Specific Neurophysiological Measures

The majority of studies on non-invasive neurophysiology in learning have focused on brain activity and eye-related outcome measures, while relatively few studies examined other outcome measures such as skin conductance, heart rate, EMG, and respiration ( Leff et al., 2011; Lai et al., 2013; Krigolson et al., 2015; Tinga et al., 2019a), making it unclear how well current findings generalize to these other outcome measures. Additionally, most work has focused on perceptual learning and the acquisition of new information or conceptual knowledge ( Lai et al., 2013; Tinga et al., 2019a). Some learning topics should be explored in more detail in order to ensure generalizability across a wide range of tasks ( Krigolson et al., 2015). Therefore, more research is needed examining a wider range of neurophysiological measures on a range of tasks.

Validity and Applicability of Neurophysiology in Learning and Training

Although many scholars have deemed neurophysiology as having high potential in assessing learning and training, some important concerns about validity and applicability have been raised by another group of scholars. Brouwer et al. (2014) pointed out changes in neurophysiology over time during learning might not be specifically related to the learning process itself, but rather reflect unspecific changes. Brouwer et al. (2014) demonstrated that multiple neurophysiological measures (i.e., EEG, ECG, EDA, respiration, and eye-related measures) were sensitive to difficulty level but that there was no interaction between difficulty level and time on task during learning. Similarly, Cowley (2015) argued neurophysiology is able to measure simple cognitive states, but is limited in measuring higher-order cognition. Ansari et al. (2011) even argue that it is shortsighted and unrealistic to expect that results from research in cognitive neuroscience will have a direct and immediate impact in education. Dahlstrom-Hakki et al. (2019) also indicated that we are still far from being able to use neurophysiology to directly measure learning or knowledge, with these tools at best allowing us to make inferences about a complex process such as learning by studying it during carefully structured tasks. Concerns like these raise the question how valid and applicable these measures are in assessing learning, especially outside the environment of the lab. Indeed, most of the literature measuring neurophysiology during task performance focuses on measuring and predicting simple cognitive states such as arousal or attention, without examining learning ( Charles and Nixon, 2019).

Ethical Challenges and Feeling Uncomfortable Being Measured Through Sensors

The collection of a large quantity of psychological data originating from the body of a trainee obviously brings along ethical challenges. Such a large dataset on bodily signals might contain sensitive information in addition to effects of learning. For example, neurological, respiratory, and cardiac disorders are known to influence neurophysiological measures ( Willems et al., 1991; Daly and Wolpaw, 2008). Even someone's physical and psychological health affects neurophysiology ( Lovallo, 2011; Iorfino et al., 2016). One needs to ensure that this data will not be used for unethical purposes.

Additionally, trainees might feel uncomfortable being measured through sensors, with the sensors potentially inducing fear. For instance, Neven (2015) discussed the effects of a home monitoring system for the elderly consisting of cameras and sensors. Neven (2015) described that one woman developed "sensor phobia", an intense fear of sensors. Albeit an extreme example, the fear of sensors alongside the ethical issues pose a threat to applying neurophysiological measures, including those for learning and training.

Non-experts Perception of Warrantability of Neurophysiology

A final threat is one related to the perception of the layman audience. Non-experts tend to regard neurophysiology to convey an objective truth, even if conclusions based on neurophysiology are unfounded. For example, Weisberg et al. (2008) found that a group of non-experts judged explanations of phenomena containing brain-based information as more satisfying than explanations without this information, even when that information was logically irrelevant. Especially bad explanations were rated more satisfying when irrelevant neurophysiological information was added. When it comes to neurophysiology in learning and training specifically, educational concepts based on neurophysiology can be popular even when claims are not backed by scientific evidence ( Howard-Jones, 2009). These results indicate that one needs to be careful that non-experts are not misled by explanations using neurophysiological information. Researchers and practitioners should therefore ensure they are clear about what can presently be concluded from their results and what might become possible in the future ( Brouwer et al., 2015).

Conclusion

Considering the various aspects of the SWOT analysis, it is apparent that the field of neurophysiology in learning and training is developing rapidly with many opportunities existing and emerging. In these developments, weaknesses should be taken into account, such as the fact that neurophysiology responds to a wide variety of functions supported by the nervous system, and technical challenges related to synchronization and analysis of large amount of data. We should also be aware of the threats that are involved, which includes the fact that research has mainly been conducted in controlled lab environments focusing on specific learning topics and specific neurophysiological measures, as well as ethical challenges and trainees possibly feeling uncomfortable being measured through sensors. Yet, neurophysiology in learning and training has powerful and valuable strengths, such as allowing for formative and objective assessment, tailoring the task to the individual, and gaining deeper insights through multi-channel data. Additionally, there are several opportunities for the field of neurophysiology in learning and training.

To start with, as the associated costs to record neurophysiological measures continue to go down and equipment becomes less intrusive for the trainee and can be set-up more quickly and easily, applying non-invasive neurophysiology in learning and training will become increasingly feasible. We expect technological advancements in the upcoming years to substantially reduce the difference in data quality between lab-based systems and wearables. The advancements will allow for focusing on examining effects in more detail outside of the lab to make sure that current results established in the lab replicate in more applied and less controlled environments. Moreover, it will become more feasible to assess a wider range of measures to add to the literature which is currently mostly focused on measures of brain activity and eye-related outcomes.

The important challenge to distinguish learning-specific effects from other effects will become easier to deal with when research has identified under which conditions effects will hold for which neurophysiological measures, in combination with advancements in data quality and dealing with noise. Especially when taking advantage of the highly interdisciplinary nature of the research field, experts can work together in advancing toward solutions to dealing with challenges like these.

As in any other field, ethical issues such as proper data handling should be considered carefully. These issues will especially be apparent as application of non-invasive neurophysiology in learning increasingly takes place outside the lab. One possible direction could be to take advantage of real-time data analysis and visualization of more general changes in neurophysiology, which does not necessarily involve saving a full dataset with specific, and possibly sensitive, responses. This online assessment of neurophysiological effects could already provide trainers, and even trainees, with important information on the learning process and can be used to tailor the learning task in real-time.

Also, one should ensure findings in the field are communicated clearly and acknowledge limitations in order to warrant that findings are not misinterpreted. Here one could also take advantage of the highly interdisciplinary nature of the research field, which allows for addressing a wide range of aspects to guarantee a comprehensive view of new findings.

Developments in including neurophysiological sensors in learning systems and new technologies such as virtual reality systems will open up many interesting possibilities for application. For example, promising immersive learning experiences can be more effectively adapted to the individual when making use of online (automatic) formative assessment through neurophysiology.

Opportunities of neurophysiology in learning and training can be seized most effectively by having insight into the weaknesses and threats of their application as well as their unique strengths. In this way, the field of non-invasive neurophysiology in learning and training can move forward in promising directions getting closer toward valid application of neurophysiology in a wide range of learning and training settings.

Author Contributions

All authors: conceptualization, and writing-review and editing. AT: writing-original draft preparation. ML: funding acquisition.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

Funding. The current research is funded by the European Union, OP Zuid, the Ministry of Economic Affairs, the Province of Noord-Brabant and the municipalities of Tilburg and Gilze Rijen (PROJ-00076) awarded to ML. The usual exculpations apply.

References

Barrios V. M. G., Gütl C., Preis A. M., Andrews K., Pivec M., Mödritscher F., et al. (2004). "AdELE: A framework for adaptive e-learning through eye tracking," in [Proceedings of the I-KNOW'04, Graz. Google Scholar]

Berntson G. G., Cacioppo J. T. (2000). "From homeostasis to allodynamic regulation," in [Handbook of Psychophysiology, 2nd Edn, eds Cacioppo J. T., Tassinary L. G., Berntson G. G., (New York, NY: Cambridge University Press; ), 459-481.Google Scholar]

Brenner E., Chirculescu A. R. M., Reblet C., Smith C. (2015). Assessment in anatomy. [Eur. J. Anat. 19 105-124.Google Scholar]

Cowley B. (2015). Psychophysiology and high-performance cognition - a brief review of the literature. [PeerJ 3:e1373v1 10.7287/peerj.preprints.1373v1 [CrossRef] Google Scholar]

Díaz D. H. ÓC., Pallas-Areny R. (2010). "Heart rate detection from single-foot plantar bioimpedance measurements in a weighing scale," in [Annual International Conference of the IEEE Engineering in Medicine and Biology (Piscataway, NJ: IEEE; ), 10.1109/IEMBS.2010.5627358 [PubMed] [CrossRef] Google Scholar]

Jarchi D., Casson A. J. (2016). "Estimation of heart rate from foot worn photoplethysmography sensors during fast bike exercise," in [Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Piscataway, NJ: IEEE; ), 10.1109/EMBC.2016.7591398 [PubMed] [CrossRef] Google Scholar]

Mullen T., Kothe C., Chi Y. M., Ojeda A., Kerth T., Makeig S., et al. (2013). "Real-time modeling and 3D visualization of source dynamics and connectivity using wearable EEG," in [[Proceedings of the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Piscataway, NJ: IEEE; ), 10.1109/EMBC.2013.6609968PMC free article] [PubMed] [CrossRef] Google Scholar]

Page E. B., Webb E. J., Campbell D. T., Schwartz R. D., Sechrest L. (1966). Unobtrusive measures: nonreactive research in the social sciences. [Am. Educ. Res. J. 3:317 10.2307/1162043 [CrossRef] Google Scholar]

Song J. C., Nam S., Song K.-S., Park S. Y. (2015). Development of a personalized learning system using gaze tracking system. [WSEAS Tranact. Comput. 14 264-271.Google Scholar]

Stobart G. (2008). [Testing Times: The Uses And Abuses Of Assessment. Abingdon: Routledge, 10.4324/9780203930502 [CrossRef] Google Scholar]

Weinstein I. (2001). Testing multiple intelligences: Comparing evaluation by simulation and written exam. [Clin. Law Rev. 8 247-286.Google Scholar]

Articles from Frontiers in Neuroscience are provided here courtesy of Frontiers Media SA

# data[].attributes.attachment_link.title
Non-invasive Neurophysiology in Learning and Training: Mechanisms and a SWOT Analysis

# data[].attributes.attachment_link.description
Although many scholars deem non-invasive measures of neurophysiology to have promise in assessing learning, these measures are currently not widely applied, neither in educational settings nor in training. How can non-invasive neurophysiology provide insight into learning and how should research on this topic move forward to ensure valid applications?

# data[].attributes.body
Rewards- What you can do to motivate students?

# data[].attributes.body
Use Models when presenting the information. Make sure that you differentiate to meet the needs of all students.

# data[].attributes.body
Problem-Based Learning- students work together in a group to solve a problem. This type of learning helps students take control of their learning.

# data[].attributes.attachment_link.provider_name
Center on the Developing Child at Harvard University

# data[].attributes.attachment_link.title
InBrief: The Science of Early Childhood Development

# data[].attributes.attachment_link.description
Early experiences establish either a sturdy or a fragile foundation for all of the learning, health and behavior that follow.

# data[].attributes.body
Network theory- It discusses emotional reactions in stages. It also discusses how emotions help with attention. Emotion also helps with learning.

# data[].attributes.attachment_link.title
Why Ages 2-7 Matter So Much for Brain Development

# data[].attributes.attachment_link.description
Rich experiences—from play to the arts and relationships—fundamentally shape a young child’s development.

# data[].attributes.attachment_link.readable_text
Abstract

Psychophysiological and neuroscience studies of emotional processing undertaken by investigators at the University of Florida Laboratory of the Center for the Study of Emotion and Attention (CSEA) are reviewed, with a focus on reflex reactions, neural structures and functional circuits that mediate emotional expression. The theoretical view shared among the investigators is that expressed emotions are founded on motivational circuits in the brain that developed early in evolutionary history to ensure the survival of individuals and their progeny. These circuits react to appetitive and aversive environmental and memorial cues, mediating appetitive and defensive reflexes that tune sensory systems and mobilize the organism for action and underly negative and positive affects. The research reviewed here assesses the reflex physiology of emotion, both autonomic and somatic, studying affects evoked in picture perception, memory imagery, and in the context of tangible reward and punishment, and using the electroencephalograph (EEG) and functional magnetic resonance imaging (fMRI), explores the brain's motivational circuits that determine human emotion.

The general thesis examined here is that experienced emotions are founded on the activation of neural circuits that evolved in the mammalian brain to ensure the survival of individuals and their progeny. Primitively, these motive circuits were engaged by external stimuli that are appetitive and potentially life sustaining, or alternatively, represent threats to the organism's survival. The psychobiological consequences of this neural firing are potentially twofold: On the one hand, they engage sensory systems that increase attention and facilitate perceptual processing, and on the other, they initiate reflex responses that mobilize the organism and prompt motor action.

Research with animals (e.g., Davis, 2000; Fanselow & Poulos, 2005; Kapp, Supple & Whalen, 1994; LeDoux, 2003) has defined the key neural structures in this survival network (see Figure 1 : Lang & Davis, 2006; see also, Davis & Lang, 2003): The bilateral amygdala e-- two small, almond shaped bundles of nuclei in the temporal lobe-- plays a central role. Each amygdala receives input from cortex and thalamus (sensory) and hippocampus (memory), subsequently engaging through the central nucleus and extended amygdala (basal nucleus of the stria terminalis) a range of other brain centers that modulate sensory processing (vigilance), increase related information processing, and activate autonomic and somatic structures that mediate defensive or appetitive actions.

It is convenient to consider this survival circuit as organized into two motivational systems, one defensive and associated with reports of unpleasant affect and the other appetitive, associated with pleasant affect. Konorski (1967) early conceived such a motivational typology, keyed to the survival role of the body's many unconditioned reflexes. Exteroceptive reflexes were either preservative (e.g., ingestion, copulation, nurture of progeny) or protective (e.g., withdrawal from or rejection of noxious agents). He further suggested that affective states were consistent with this preservative/protective grouping: Preservative emotions include such affects as sexual passion, joy, and nurturance; fear and anger are protective affects. Dickinson and Dearing (1979) developed Konorski's distinction, renaming the two motivational systems, aversive and attractive, with each again activated by a different, but equally wide range of unconditioned stimuli, determining perceptual-motor patterns and the course of learning. In this general view, affective valence is determined by the dominant motive system: the appetitive system (preservative/attractive) prompts positive affect; the defense system (protective/aversive) is the source of negative affect. Affective arousal reflects the "intensity" of motivational mobilization, determined originally by degree of survival need and the imminence or probability of nociception or appetitive consummation. In this regard, it is pertinent that factor analyses of emotional/evaluative language (since Osgood, Suci, & Tannenbaum, 1957; see also, Russell & Mehrabian, 1977; Bradley & Lang, 1994) have consistently found two main factors accounting for the most variance among affect descriptors: Emotional valence (positive/pleasant/appetitive vs. negative/aversive/defensive) and arousal (intensity of activation). Thus, it would appear that, despite the great number and diversity of emotional words, the underlying structure of affective language appears to reflect the dual motive system model-- appetitive and defensive neural circuits that vary in the vigor of their activation.

The defense system is ultimately a fight or flight circuit, but in response to danger cues it also mediates behavioral "freezing" (hiding), increased vigilance, and counter-threat displays in animal subjects. The appetitive system is activated variously in alimentation, sex, and nurturance of progeny. However, as will be seen, although some reactions are uniquely appetitive or defensive, many physiological and behavioral patterns are similar in both contexts of arousal, and are mediated by the same neural structures.

The view to be elaborated here is that these survival circuits in primitive cortex and the limbic brain are the stuff of human motivation, and that motivational arousal is the foundation of emotion. From this neuroscience perspective, human emotions can be described as dispositions to action (e.g., anger>attack; sexual desire>approach; fear>escape-- see also Frijda, 1986). That is, expressed emotions are grounded in motivational circuits that feed back to sensory systems, heightening vigilance and information gathering, and importantly, prompting reflexive autonomic and motor responses that in evolutionary history acted directly to counter threats and escape punishment, or achieve needed rewards.

In humans, affective percepts, thoughts, and images act as motive cues that automatically engage the older limbic circuits and reflex patterns. Of course, with evolution of the cerebrum came our human metamorphic capacity to delay, modulate or inhibit overt actions, providing for more flexible and adaptive responses to achieve survival's aims. Nevertheless, the primitive physiology of action preparation persists, mobilizing muscles and glands and placing sensory systems on high alert-- functions that are the physiological manifestations of emotional expression.

This essay considers research undertaken from this perspective, first examining bodily, behavioral, and brain reactivity associated with threatening or appetitive cues, as they vary in their proximity to, and imminence of, punishment or reward. Subsequently, we describe research with emotional pictures, assessing EEG event related potentials, eye tracking, and changes in pupil dilation that co-vary with emotional arousal. Finally, we consider recent fMRI research that delineates structural and network activation differences for appetitive and aversive cues, and show a common activation of the brain's motivational circuits in emotional perception and mental imagery.

Predator and Prey: The defense cascade

In considering the mortal confrontations between predator and prey, ethologists and comparative psychologists have defined characteristic response patterns that change systematically with the proximity and probability of an encounter (e.g., Blanchard & Blanchard, 1989; Campbell, Wood, & McBride, 1997; Timberlake, 1993; Tinbergen, 1951). When one antagonist perceives the other, responses are sequentially deployed that vary in function and emphasis with the imminence of a physical confrontation: For prey, the sequence enhances the probability that a threat will be survived; for the predator, it optimizes the chance of a final strike and capture. In many ways the required responses (and their mediating brain circuits) are broadly similar, consisting of augmented vigilance (a rising demand for attentional resources), accompanied by increasing physiological mobilization and a readiness for action (fight/flight or strike/capture). In the natural environment, of course, the same organism often plays both roles, foraging for rewards, but wary of being attacked: A small bird fishing for modest game risks the notice of an eagle, a larger, more voracious species that views the fisher and its catch as its proper prey.

In a recent experiment ( Löw, Lang, Smith, & Bradley, 2008), we examined anticipatory brain and reflex reactions in human beings in the laboratory using a simulation that was modeled on features of the predator-prey survival scenario, in which the participant acted both parts. The research was prompted by previous data indicating that viewing unpleasant, emotionally evocative pictures elicits patterns of physiological reflex reactions that vary markedly (in direction and amplitude) with evaluative reports of the intensity of affective arousal (e.g., Lang, Bradley, & Cuthbert, 1997). Thus, some autonomic responses, such as skin conductance or pupil dilation, show a monotonic increase with rated emotional arousal. Heart rate, however, decreases with greater arousal, except when viewing aversive pictures that prompt the reports of highest arousal (e.g., in phobic reactions), which then show a dramatic tachycardia ( Hamm, Cuthbert, Globisch, & Vaitl, 1997). The probe startle reflex also shows directional changes, initially inhibited and then increasingly potentiated with greater rated arousal of individual aversive picture cues ( Bradley, Codispoti, & Lang, 2006; Cuthbert, Bradley, & Lang, 1996; Lang, 1995).

This directional variability within and between measures as emotional intensity increases suggests that different defensive functions are being served along the arousal continuum. One hypothesis is that reports of aversive arousal reflect an atavistic distance parameter, and that the changes in physiology when viewing unpleasant pictures reinstantiate reactions that in prey animals changed as threatened confrontation became increasingly imminent ( Lang et al., 1997). From this perspective it is assumed that a human participant looking at an unpleasant/aversive picture is viewing threat at a distance, analogous to a prey animal (e.g., gazelle) apprehending a distant predator (e.g., a lion) from afar. Increases in rated arousal might be associated with progressive changes in reflex physiology that are similar to those observed if a threatening event is increasingly imminent-the lion approaches. That is, reports of emotional arousal indicate, however imperfectly, the intensity of defense system activation which then deploys different reflex responses for remote threats (where confrontation is uncertain) than for an increasingly imminent encounter. At a distance, perceptual processing and information gathering dominate, but as the distance from a threatening stimulus diminishes, organism mobilize increasingly for coping action-the fight/flight imperative.

The aim of the first research to be described here is to examine changes in reaction patterns as motive cues appear to loom progressively closer, as well as to consider possible differences (and similarities) associated with cues that signal pleasant or alternatively, unpleasant consequences (reward or punishment). In a series of classic studies, Neal Miller (1959) assessed behavioral responses (latency and strength of response) in rodents confined to an alley maze, and defined motivational gradients of approach to food reward vs. avoidance of electric shock. His work showed greater relative approach motivation when the animal was distant from the reward, but that the strength of avoidance rapidly superseded approach when the goal was proximal. More recently Kahneman and Tversky (2000) have reported a similar advantage for aversive motivation in human beings, showing that motivation to avoid loss greatly exceeds motivation to gain new resources.

A computer simulation

The predator/prey research environment took the form of a computer simulation: Participants observed a continuous stream of briefly presented pictorial images (see Figure 2 ). Most of these images depicted motivationally neutral objects, animals, or people, each replacing the other at a constant rate in a continuous flow, mimicking the serial humdrum of daily life. Occasionally, however, a picture appeared that had motivational significance: An offered fist-full of money or alternatively, an attacker's hand pointing a gun at the viewer. These motivationally relevant images were sometimes repeated several times in a sequence, each time larger, as if the gun or money were moving closer to the viewer. This "looming" sequence could continue for a few "foil" trials, ending with naught but a return to the neutral image stream.

If, however, the image continued its approach, the player was metaphorically in the predator-prey strike zone. At some point in this region, the background of the picture (gun or money) changed color, signaling the participant to press a reaction-time-key as rapidly as possible. On money trials, a fast response resulted in a reward of 1 dollar; a slow response produced no gain and an image of the money burning up. If reactions were slow on a threatening gun trial, the gun appeared to fire and 1 dollar was deducted from the participant's stake; with a timely response, however, she/he escaped the potential loss and a steel door slammed shut, blocking the gunner. As a control procedure, neutral pictures also sometimes appeared to loom, but, when their background color changed, a key press (under no time pressure) only served to continue the sequence of pictures.

The experimental task permitted the evaluation of both reflex reactions and brain potentials as they are modulated by increasing temporal, proximity/probability of reward or of a punishing loss. In humans, these measures are held to reflect modulation of three coincident anticipatory processes: heightened attention to motivationally significant cues; mobilization of the body for action, preparatory to avoiding a theft or achieving a reward; and emotional arousal, evoked by the context of threat and appetite.

Autonomic responses

As anticipated, skin conductance increased progressively during both reward and punishment sequences. Its amplitude was already somewhat heightened, compared to neutral cues, early in the sequence when a final confrontation was still uncertain. Then, as shown for the gun cue in Figure 3 -- but also seen for reward-- it rose dramatically during the final steps before the goal, when a fast response would be required. Sweat gland activity is a well-known indicator of general sympathetic activation (e.g., Wallin, 1981), and associated with increased metabolism, visceral changes, and modulation of sensory systems, as in pupil dilation (e.g., Janisse, 1977). The other autonomic measure examined here, heart rate, showed a modest initial increase (perhaps attributable to vagal release); however, as a possible reward or the risk of loss became more certain ( Figure 3 ), heart rate decreased steadily in frequency to a nadir on the penultimate approach cue. This deceleration was then followed by an abrupt acceleration just prior to the final response signal for escape or capture.

In preparing to seize a reward or escape a punishment, the two branches of the autonomic nervous system are co-active: The increase in sweat gland activity indicates progressively augmented sympathetic arousal; in contrast, heart rate is mainly under parasympathetic control, and decreases progressively with closer proximity to the goal. This latter phenomenon has been widely noted in biological studies of prey animals confronting a predator ( Campbell et al., 1997), and is coincident with relative immobility (freezing) and an increasingly focused vigilance. In the context of predator approach, it is described as "fear bradycardia" (and for the predator/prey task, the deceleration was predictably, somewhat greater for the gun cue). Nevertheless, within the strike zone, of course, there is anticipatory parasympathetic release and a sympathetic increase in rate accompanies the final mobilization for goal directed action. Interestingly, brief anticipatory increases in rate can also be seen increasing over the immediately preceding picture cues.

The probe startle reflex

In less complex animals, the startle response can be considered an automatic escape reflex as with an abrupt movement overhead, the sudden flight of a fly. Although this functional significance has been lost in humans, it is clear that startle reflex amplitude is modulated by at least two factors: Hedonic content (most clearly, a potentiated response in unpleasant perception) and focused attention (prompting relative reflex inhibition).

During the predator-prey simulation, startle reactions were tested at various stages of the approach/avoidance sequence, including when viewing the initial cue, midway through the sequence, and just prior to the response cue. As illustrated in Figure 4 , the initial exposure to a motivational cues (i.e., a distant threat or reward) yielded the pattern observed in many studies-significant reflex potentiation when viewing the threat cue and a relative inhibition for appetitive cues (e.g., Bradley et al., 2006; Lang, Bradley, & Cuthbert, 1990). However, as the distance to confrontation closed, reflex magnitude is increasingly inhibited, reflecting effects of task-focused attention and suppression of irrelevant action, facilitating a final functional response to the motive cue. From this perspective, fear potentiated startle appears to be a premature action trigger, subsequently suppressed when the exigencies of a more efficient survival response dominate.

The poetry of the predator-prey interaction can be described as an orchestration of fear and desire. For a natural science of emotion, however, it highlights the impossibility of marrying, in the Jamesian sense, emotional language (fear, anxiety; longing, joy) directly to the underlying dynamic flow of behavior that changes differently across different physiological systems as an anticipated reward or fear of loss passes from the possible to the imminent.

The brain in threat and reward

In addition to autonomic and somatic reflex measures, the electroencephalogram was recorded throughout the predator-prey simulation, and event related potentials were assessed at each stage along the approach/avoidance gradients. Figure 5 shows the electroencephalographic event related potential (ERP) responses to each motivational cue (i.e., gun or money), at their first appearance in the continuing sequence of neutral pictures (upper right panel). Emotional modulation of the ERP is essentially the same as that found when pictures are presented for free viewing (e.g. Cuthbert, Schupp, Bradley, Birbaumer ,& Lang, 2000): A slow positive waveform is apparent beginning around 300 ms and extending past 700 ms that is reliably enhanced over centro-parietal sensors for emotional pictures - either pleasant or unpleasant - that increases in amplitude with increases in rated arousal. As illustrated in the left panel, this late positive potential is dramatically enhanced for all cues at the penultimate (presumably most activating) cue which occurs just before a motor response is required.

Differences in the late positive potential for the looming gun and money cues, compared to the neutral cue, are shown at the bottom of Figure 5 for each step in the looming sequence. It is of interest that the initial heightened potential to motivational cues shows some decrease early in the sequence, when the continuation of the sequence is uncertain (1-3). Subsequently, however, the amplitude of the late positive potential increases progressively, peaking just before the cue that initiates action. Interestingly, the late positive potential is significantly larger for the gun cues (threat of loss) than for money (a possible gain) at this final stage, consistent with Miller's (1959) hypothesis that avoidance gradients are steeper than approach (relatively higher close to the goal, cf. Cacioppo & Berntson, 1994), as well as with Kahneman and Tversky's (2000) view that motivation to avoid a loss is greater than that to achieve a gain.

The late positive potential and motivational relevance

The late positive potential evoked by picture stimuli is a reliable, replicable index of their motivational relevance. Recording from scalp electrodes and with the electroencephalograph (EEG) amplifiers set for a long time constant (allowing one to see sustained brain potentials), Cuthbert et al. (2000) observed an enhanced centroparietal positive signal that began around 300 ms after the onset of an emotional picture and persisted for almost the entire duration of a 6 s viewing interval (see Figure 6 ). Its relationship to emotional arousal was confirmed in a second study in which we found that this positive potential was most enhanced for the pictures rated highest in emotional arousal, regardless of whether they depicted appetitive (e.g., erotica) or aversive (e.g., mutilated bodies) hedonic contents ( Schupp et al., 2004). The amplitude of the late positive potential (LPP) not only correlated highly with reports of rated emotional arousal but also co-varied with sympathetic arousal, as measured by electrodermal response amplitude.

Complexity or arousal

To confirm that the late positive potential varies with motivational significance, rather than with other physical characteristics of these relatively complex visual stimuli, we varied the perceptual composition of emotional and neutral pictures and presented either simple figure-ground compositions or more complex scenes in a free viewing paradigm ( Bradley, Hamby, Löw, & Lang, 2007). Not surprisingly, perceptual composition had strong effects on the event-related potential; however, these effects appeared earlier in time (150-250 ms), were maximal over occipital-temporal sensors, and did not differ for emotional and neutral pictures. The late positive potential, on the other hand, continued to be enhanced when viewing emotional (pleasant or unpleasant), compared to neutral pictures, regardless of perceptual composition. Interestingly, effects of emotion on the late positive potential were somewhat larger for simple figure-ground compositions, contrary to the view that the late positive potential reflects the complexity of information processing. Rather, hedonic content was communicated more effectively by these simple figure-ground compositions, consistent with the finding of stronger affective modulation of the startle reflex for simple, compared to complex, scenes ( Bradley et al., 2007).

Attention and habituation

We originally entertained the hypothesis that sustained positive potentials during affective perceptual processing simply reflected enhanced attention allocation to the motivationally relevant pictures. Previous findings using secondary probes were consistent with this view as the evoked brain response (i.e. probe P3 amplitude) is smaller when viewing emotionally arousing pictures (e.g., Schupp, Cuthbert, Bradley, Birbaumer & Lang, 1997) and probe reaction times are slower ( Bradley, Cuthbert, & Lang, 1996). Furthermore, memory is enhanced for emotional, compared to neutral, pictures ( Bradley, Greenwald, Petry, & Lang, 1992). However, a more recent series of studies conducted in conjunction with Maurizio Codispoti and Vera Ferrari argue that this simple, attention-capture interpretation may be too narrow ( Codispoti, Ferrari, & Bradley, 2006, 2007). In these studies, we presented the same picture for as many as 90 trials in the same experimental session. Although the overall amplitude of the late positive potential did diminish somewhat with repetition, significant modulation of the late positive potential by emotion persisted to the last presentation (see Figure 7 ). Moreover, an enhanced late positive potential is found even when pictures are presented very briefly (25 ms): Given that the late potential persists for seconds after the picture itself is no longer on the screen in this case, the hypothesis that it reflects a continuing perusal of the sensory array would seem to be ruled out ( Codispoti, Mazzeti, & Bradley, 2009; see Figure 7 ).

On the basis of these and other autonomic data involving picture repetition, it was suggested ( Bradley, 2009) that the late positive potential is a broad index of motivational "significance". That is, the LPP is a persistent sign that the evoking stimulus is motivationally relevant, reliably different from neutral cues. Furthermore, of course, it varies in amplitude with increasing motive system activation (appetitive or defensive). Late positive potentials are enhanced at initial orienting, extended with sustained perceptual processing, associated with preferential memory storage, and in considering the predator/prey findings, perhaps further augmented when motivated actions are imminent.

What brain structures underlie the ERP late positive potential?

Determining the neural structures that are primary contributors to any specific component of the event-related potential is a difficult task: Most meaningful features of the ERP waveform are less than 200 ms in duration, and these bioelectric signals are determined by volume conduction across the whole brain at each scalp electrode (albeit from different trajectories). Considering the relatively large amplitude of the late positive potential, however, and its sustained duration (which may persist over several seconds), it is perhaps the strongest candidate-ERP event for which the spatial location of primary voltage sources in the brain might be uncovered. Consistent with the view, Keil et. al (2002) used the minimum norm method to determine dipole sources contributing to the LPP signal, based on the analysis of a 129-electrode array. The most reliable differences in dipole strength that showed enhanced activity when viewing emotional, compared to neutral, pictures, were located in the dorsal and ventral posterior brain. While lacking structural specificity, the findings are consistent with fMRI findings ( Bradley et. al, 2003; Lang et. al, 1998) showing enhanced activation for emotional pictures in secondary visual processing sites in lateral occipital cortex, extending up the dorsal stream to parietal cortex.

A more recent study ( Sabatinelli, Lang, Keil, & Bradley, 2007) provided a more precise answer to the question of the neural determinants of the late positive potential evoked during picture viewing. The experiment was conducted in two sessions that repeated the same procedure on two different days. Participants were seen either first in the scanner at the MRI facility where functional changes in specific brain regions were measured, or first in an MRI simulator that exactly duplicated the scanner environment, and where dense sensor EEG was safely and accurately recorded. Subjects were simply instructed to look at each picture while it was on the screen, and at a fixation cross between trials. Pictures from the International Affective Picture System (IAPS, Lang, Bradley, & Cutlhbert, 2008; Bradley & Lang, 2007) included equal numbers of images rated pleasant, unpleasant and neutral (using SAM, Bradley & Lang, 1994; Lang, 1980), presented in an intermixed order. The same pictures were seen in both sessions. As previously discussed, research has shown that modulation of the late positive potential by emotion is highly stable over repeated trials, even when the same picture is presented multiple times (e.g., Codispoti et. al, 2007). We hypothesized that the determining underlying neural structures would show a similar consistency over the two experimental sessions.

Figure 8 (left panel) shows the scalp voltage maps for the late positive potential and the fMRI activation sites observed when participants viewed pleasant, neutral, and unpleasant pictures. From this posterior view, it is apparent that, compared to neutral pictures, pleasant and unpleasant emotional images similarly prompt greater blood oxygen level-dependent (BOLD) increases in the occipital and parietal brain regions that support perceptual processing (lower left panel). The topographical maps of the late positive potential (upper left) show that emotional pictures (both pleasant and unpleasant) prompt parallel stronger voltage foci for the LPP over the anterior brain than is found for neutral scenes. The right panel compares event related response waveforms from the EEG (positive voltage up) and fMRI sessions: Similar enhanced responses are apparent for emotionally arousing pictures (pleasant and unpleasant), relative to neutral, despite the great difference in the latency of EEG and BOLD signals.

In subsequent analyses, we assessed the relationship between the late positive potential (measured over centro-parietal sensors) and BOLD activity in specific brain regions implicated in secondary visual processing: Significant correlations were obtained between the amplitude of the LPP and activation in lateral occipital cortex (.65), parietal cortex (.61), and in inferotemporal cortex (.55). Importantly, the amplitude of the late positive potential did not correlate significantly with activation in primary (pericalcarine) visual cortex, where the sensory stimulus is first mapped prior to secondary (semantic/affective) elaboration. Significant correlations of the late positive potential with deeper limbic structures (e.g., the amygdala) were also not found, perhaps because the latter, small neural structures generate a weaker and less reliable BOLD response, and being distant from the scalp, their relatively minute neural signal is negligible in the volume conduction recorded by surface electrodes. Overall, these data show a dramatic co-variation between the late positive potential of the ERP and BOLD activity measured using fMRI, suggesting that the late positive potential, evoked during non-task-related picture viewing, is generated predominantly by elaborated perceptual processing in the extra-striate visual system of the cerebral cortex.

Amygdala/visual cortex connections during picture viewing

One interpretation of these ERP and fMRI findings is that the higher activation of visual processing areas for emotional stimuli is attributable to re-entrant projections to the sensory system from the brain's motivational circuits. In detailed primate studies ( Amaral, Price, Pitkänen, & Carmichael, 1992), it has been shown that visual information is processed in a hierarchical fashion along the ventral temporal stream from primary striate cortex to higher level processing areas and proceeding finally from the inferotemporal (TE) area to the lateral nucleus of the amygdala. Subsequently, the amygdala's basal nucleus back-projects "to virtually all levels of the visual cortex' (p.44: Amaral et al., 1992). Thus, it is held that emotional cues receive sustained processing that is mediated through a positive feedback loop that is specifically enhanced for "significant" stimuli.

Presumptive evidence for this hypothesis was obtained in a fMRI experiment in which we presented fear-related pictures to highly fearful individuals ( Sabatinelli, Bradley, Fitzsimmons & Lang, 2005). Emotional and neutral pictures were organized into categories that represented a monotonic increase in rated emotional arousal sequentially, from neutral scenes to neutral people, to non-threatening animals, to snakes, to erotica, to mutilated human bodies. The regions of specific interest were the inferotemporal cortex and the amygdala. Inferotemporal cortex has been implicated in the processing color, faces, and semantic categories ( Bartels & Zeki, 2000; Chao, Haxby & Martin, 1999; Gauthier, Curran, Curby & Collins, 2003; Grill-Spector, Knouf, & Kanwisher, 2004) and, as noted previously, we have consistently found greater activation of secondary visual processing centers when people view emotional, compared to neutral, pictures ( Bradley et al., 2003; Lang et al., 1998). Furthermore, diffusion tensor imaging research has revealed a direct white matter tract in humans between secondary visual and anterior temporal cortex ( Catani, Jones, Donato, & 2003), complementing the findings from primates (cited above).

In figure 9 (right panel), overall differences in the BOLD response between neutral picture contents and the most emotionally arousing pictures (erotica and mutilations) are shown for the bilateral amygdala and inferotemporal cortex. Consistent with hypothesis, both high and low fear individuals showed substantially larger responses when viewing emotionally arousing contents (both pleasant and unpleasant). Also as predicted, there was a parallel significant linear increase in BOLD signal over categories, almost identical in the two bilateral structures (see Figure 9 , upper left panel).

The bottom left panel illustrates the findings for the set of participants who reported high fear of snakes on a prescreening questionnaire. These fearful individuals (N=9) showed a significantly enhanced response when viewing pictures of snakes when compared with the other (unselected) participants. The co-variation between rated arousal of the picture and BOLD activity is impressive, both generally and considering individual differences in fear. Furthermore, these data are consistent not only with the hypothesis that these structures are interconnected, but that re-entrant input from the amygdala prompts enhanced perceptual processing of emotionally arousing cues.

More recently, using a novel fMRI fast-slice methodology, Sabatinelli, Lang, Bradley, Costa and Keil (2009) showed clear differences in the onset latency of BOLD effects in secondary visual processing areas further supporting Amaral's primate model ( Amaral et al., 1992). That is, emotionally arousing pictures prompted significant activation first in structures more closely connected to the amygdala (i.e., inferotemporal "fusiform" cortex) and palpably latter in more distant, posterior regions of the visual system (i.e., lateral occipital cortex). Other findings based on a Granger causality analysis of steady-state EEG potentials visually evoked by emotional stimuli ( Keil et. al, 2009) suggest there may be an even broader re-entrant connectivity to the visual processing system from anterior neural structures, both subcortical and cortical.

The brain begins at the eye

For visually oriented mammals, particularly primates, the eye is the principal organ for gathering information from the environment concerning threat and safety, sexual opportunity, and for successful foraging. The brain begins with the neuroectodermal photoreceptors at the back of the eye, and the optical system as whole-- lenses, receptor bed, and pupillary and orienting musculature-is under fine nervous control. Thus, motivationally relevant signals from the environment prompt rapid adjustment in the brain' primary receptor organ.

In a recent series of experiments we have tracked the eye as it looks at visual cues depicting appetitive and aversive events, measuring both scanning behavior (e.g., number and duration of fixations; Bradley, Miccoli, & Lang, 2009) and changes in pupil diameter ( Bradley, Miccoli, Escrig, & Lang, 2008). Although physical properties of visual stimuli such as contrast, edges, luminance, etc. are known to assist in guiding the eye as it traverses a visual stimulus (e.g. Marr, 1985), it has also long been known that the eye is also more likely to fixate on interesting or salient information in the environment and that the gaze remains longer on interesting or informative events (e.g., Loftus & Mackworth, 1978).

In our research ( Bradley et. al, 2009), participants viewed emotional and neutral pictures that were either simple figure-ground compositions, or more complex scenes. Eye tracking behavior was significantly affected by the motivational relevance of pictures -- regardless of their perceptual complexity. Thus, emotional pictures - either pleasant or unpleasant - were associated with a significantly greater number of fixations than neutral images, and this pattern was found regardless of whether the pictures depicted simple figure-ground compositions or more complex scenes. Moreover, affectively engaging cues were scanned much more broadly, as evidenced by significantly greater average distances between successive fixations than for neutral pictures, which again was found for both simple figure-ground compositons as well as more complex scenes. Taken together, the data suggest that picture cues that contain emotionally arousing content generate increased scanning that sweeps more broadly across the image when motivationally relevant information is detected.

Affective images also prompt significantly larger pupil diameter during picture viewing. Hess ( Hess & Polt, 1960) famously proposed that the pupil dilated for positive cues (e.g., acceptance) and constricted for negative cues (e.g. rejection), but there were numerous methodological and measurement issues in those early studies, and the results proved difficult to replicate. Because the pupil is extremely sensitive to changes in light, it is quite important to equate affective cues in terms of their brightness. Fortunately, the pupil itself proves to be a good index of brightness differences, as the initial light reflex (pupil constriction) following presentation of a visual cue can be easily measured. In our study ( Bradley et. al, 2008), we initially matched emotional and neutral pictures in terms of contrast and brightness, and measured the initial light reflex. As illustrated in Figure 10 , the initial pupil constriction following picture onset was equivalent across pleasant, neutral, and unpleasant pictures, allowing an assessment of pupillary changes during picture viewing that may be specifically associated with hedonic content.

As illustrated in Figure 10 (inset), viewing either pleasant or unpleasant pictures prompted relative pupil dilation, compared to neutral pictures, and the same pattern was found regardless of whether pictures had been judged to be simple in perceptual composition (e.g., figure-ground) or were more complex scenes, showing a within-study replication across different affectively engaging pictures. Pupil dilation also covaried strongly with changes in skin conductance activity. Thus, for both pupil diameter and skin conductance change, reactions were largest for the pleasant and unpleasant pictures rated as most arousing. The close relationship between pupil diameter and skin conductance reactivity was expected, considering that increases in both responses are mediated by the sympathetic nervous system.

In the view presented here, these autonomic reactions begin when a motivational cue is processed in visual cortex and activates the basolateral and then the central nucleus of the amygdala, which subsequently projects to the lateral hypothalamus, prompting broad engagement along the sympathetic chain, including neural connections to the rapidly dilating pupil, as well as to the more slowly responding palmar sweat glands (see Figure 1 ). The evolutionary significance of these responses has prompted much speculation (e.g., pupil dilation increased light gathering in dark environments; palmar sweating reduced abrasion), but there is little support for a specific view. As for the potentiated startle response, these autonomic reflexes appear to be vestigal remainders from evolutionary history. They are still automatically deployed in motivational arousal, indexing emotional intensity; however, their specific survival value is no longer apparent.

Differences in brain circuit activation for pleasant and unpleasant cues

Looking at pictures

Initial exposure to a novel unpleasant picture is associated with sustained cardiac deceleration, facial muscle action (corrugator "frown" muscle), and potentiation of startle probe reflexes, whereas these responses differ dramatically in direction and magnitude during exposure to a novel pleasant cue (e.g., Bradley & Lang, 2007; Lang, Greenwald, Bradley, and Hamm, 1993). Thus, these physiological measures indicate clearly which motivational system is active -- appetitive or defensive. In contrast, skin conductance, pupil dilation, and the brain's late positive potential increase in amplitude as both pleasant and unpleasant pictures are rated more arousing, indicating the intensity or vigor of motivation, but not its goal direction (see Bradley and Lang, 2007.

These findings show, on the one hand, that much of the brain is similarly engaged by motivationally relevant cues, whether they signal threat or reward; on the other hand, it is apparent that some brain circuits must be specific to the hedonic valence of an initiating cue, deploying different responses depending on the goal. We have already seen that the amygdala fits the former case and is highly responsive to the intensity of both appetitive and aversive emotional stimulation, and that it is central in elevating sympathetic reactivity and in enhancing visual processing-- reflexes that are clearly appropriate both under conditions of threat and appetitive arousal. We will next consider research exploring other brain structures and circuit connections-sites in the cortex and limbic brain specific to appetitive motivational processing and positive affect.

Research with animals suggests that reward-seeking behavior is controlled by neural circuits that include the basolateral nucleus of amygdala, medial prefrontal cortex, and the nucleus accumbens (e.g., Ishikawa, Ambroggi, Nicola & Fields, 2008) that function as an interface with motor areas ( Mogenson, 1977). This appetitive circuit might also subserve pleasure-specific emotional reflexes. Neural imaging research with human participants has indeed implicated these neural structures in the perception of motivationally positive stimuli (e.g., pictures of loved ones: Aron et al., 2005, Bartels and Zeki, 2004; addictive drug cues; David et al., 2005); however, other research has suggested that the striatum (which includes the nucleus accumbens) may be active for any salient stimulus that arouses and prompts attention ( Zink, Pagnoni, Chappelow, Martin-Skurski, & Berns, 2006).

To assess whether these mesolimbic circuits are specifically activated for pleasurable cues, or whether their activation instead varies with emotional arousal, we presented highly arousing pleasant (erotic) and unpleasant (mutilated bodies) pictures together with pictures of neutral people in a brain imaging (fMRI) study ( Sabatinelli, Bradley, Lang, Costa, & Versace, 2007). As anticipated, the BOLD response in the bilateral amygdala was significantly greater when participants viewed erotic or mutilation pictures than for quotidian pictures of neutral people, again indicating that the amygdala is equivalently activated by emotionally arousing stimuli, whether appetitive or aversive. In contrast, it was only erotic stimuli that activated medial prefrontal cortex and the nucleus accumbens, consistent with the view that these structures are specifically implicated in the brain's appetitive motivational circuit.

A second, replication experiment expanded the picture content to include clothed romantic couples (e.g, kissing, holding hands) and unpleasant scenes in a dental operatory, snakes (coiled or attacking), and threatening people (with guns or knives). As illustrated in Figure 11 , viewing appetitive pictures of either explicit erotica or scenes with romantic couples prompted significant BOLD increases in the nucleus accumbens. An almost identical result was obtained for the medial prefrontal cortex. However, neither neutral nor unpleasant pictures activated these structures. In fact, BOLD activity when viewing these non-appetitive stimuli appears to actually be below baseline, particularly in the medial prefrontal cortex (a phenomenon also seen in the first experiment). Again, amygdala was significantly activated when viewing either arousing pleasant or unpleasant pictures, compared to the neutral content.

The results are clear in finding that arousing appetitive pictures selectively activate the same neural structures observed in animals anticipating rewards and as seen in human substance users exposed to tangible drug cues. The data do not support the hypothesis that accumbens and prefrontal cortex sites respond to all salient cues that arouse or capture attention. Rather, in contrast to activation patterns found in the amygdala and inferotemporal cortex, appetitive neural structures are silent, perhaps even deactivated in the context of arousing unpleasant picture stimuli.1

Emotional imagery and appetitive motivation

Text-driven mental imagery of traumatic memories or phobic confrontations prompts psychophysiological reactions similar to those observed in humans anticipating an actual aversive experience ( Lang, Kozak, Miller, Levin, & McLean, 1980; Miller et. al, 1987). The bio-informational theory of imagery ( Lang, 1979, 1994) originally proposed that text cues engage an emotional network of representations in the brain that includes sensory, semantic, and response information. The response information (procedural knowledge) is of particular significance for the action mobilization that characterizes emotional engagement, and it is these efferent memories that are most directly connected to the brain's motivational system.

The above conception suggests that motor control systems in the brain should be active during affective imagery, mediating the efferent output represented in emotional scenarios. Brain activation patterns consistent with this bio-informational view were obtained in a recent fMRI study of pleasant and unpleasant imagery: Greater activation was observed for arousing emotional scenarios in supplementary motor area, prefrontal cortex, and cerebellum (areas involved in planning and executing motor action) than for neutral imagery ( Sabatinelli, Lang, Bradley, & Flaisch, 2006). This activation pattern is, of course, consistent with psychophysiological studies showing reflex mobilization in arousing emotions-- increased facial muscle action, heart rate and skin conductance increases, and potentiated startle when imaging arousing aversive content (e.g., Lang & McTeague, 2009; McTeague et. al, 2009).

Perhaps because the evocation of vivid fear imagery in desensitization (e.g., Wolpe, 1961) and exposure (e.g., Foa & Kozak, 1986) have been important methods in the treatment of anxiety disorders, fMRI and PET studies of limbic circuits have focused almost exclusively on imagery activation during aversive emotional imagery (e.g., Shin et al., 2004). In a recent study, however, we included both pleasant and unpleasant emotional contents to determine if the same neural circuits implicated in appetite and defense are activated in mental imagery as in picture perception. Costa, Lang, Sabatinelli, Bradley & Versace (2009) assessed imagery of erotic content (e.g., sexual encounters), as well as scenes of joy (e.g., as winning a lottery), quotidian neutral scenes (e.g., doing laundry), and scenes describing animal attacks, contamination (e.g, a drunk vomits on your hand), survival threats (e.g., an auto accident) and dental exams.

Figure 12 shows the location and extent of BOLD activation in the motive relevant structures and the temporal course of the imagery-evoked waveforms for pleasant, neutral, and unpleasant imagery contents. As for emotional pictures, both pleasant and unpleasant arousing imagery strongly activated the amygdala. In contrast, only imagining pleasant scenes engaged the nucleus accumbens and the medial prefrontal cortex. Furthermore, the reduction in regional blood flow during imagery of unpleasant scenes in accumbens and prefrontal cortex is even more dramatic than it was during picture perception. Because hemodynamic measures are thought to reflect both excitatory and inhibitory processes ( Logothetis, 2008), this mesolimbic BOLD decrease could indicate inhibition of the appetitive system in aversive arousal, permitting the reciprocal activation of defense circuits.

High correlations were obtained between the participant's pleasure ratings of the imagined scene and BOLD activity in both the nucleus accumbens (.75) and the medial prefrontal cortex (.82: see Figure 13 ). In addition, valence-dependent functional connectivity analyses ( Friston et al., 1997) revealed a reliable time series co-variation between the accumbens and prefrontal cortex that was significant over all valence categories. However, connectivity between these sites and the amygdala was present only during pleasant imagery. In effect, although amygdala reacts to any motive cue-- and more strongly with increasing arousal, activation of the appetitive system and a consequent positive emotional states, depends on an active connection with the mesostriatal circuit.

Summary and Conclusions

The research presented in this review is driven by an animal model that defines appetitve and defensive motivational centers-- deep in the older limbic brain and primitive cortex-- that are the foundation of human emotion. In this view, emotions reflect sequenced, somatic and autonomic reflexes, organized by brain circuits that developed over evolutionary history to ensure individual survival and the propagation of genes across generations. The research has shown that in emotional perception some of these reflexes are prompted by appetitive cues, others by aversive cues, and many more are active in response to motivationally relevant stimuli, both appetitive and aversive. It has been further found that these reflex patterns are dynamic, and that individual responses are differently modulated in direction and amplitude along gradients of appetitive approach and aversive arousal.

Reflex reactions to emotional cues that signal rewards or threaten danger are associated with two primary functions: (1) They capture attention preferentially and prompt perceptual processing and information gathering (e.g. Lang et al., 1997; Bradley, 2009); (2) they occasion metabolic arousal and mobilize the organism for coping actions. In this review we considered how emotional cues, both pleasant and unpleasant, evoke an extended late positive potential in the EEG that is-as shown by fMRI evidence-- highly correlated with the activation of visual processing centers in the posterior brain. This event related potential is a persistent, specific reaction to cues of emotional significance, whether perceptually simple or complex in composition. Functional MRI research demonstrates a high correlation between activation of the amygdala and posterior cortical sites that are involved in elaborative picture processing. Furthermore, activation in both sites vary systematically with reports of emotional arousal, supporting the hypothesis that increases in the visual processing of emotional cues are attributable to re-entrant processing from anterior motivational circuits.

Studies of gaze direction indicate that the eye preferentially fixates first on pictures with emotional content over competing neutral images. Furthermore, both complex and simple emotional pictures are associated with significantly more fixations than neutral content, and are scanned more broadly. The pupil also responds to emotional content, dilating more for pleasant and unpleasant images than for neutral pictures, and similarly mediated by the sympathetic nervous system, shows a close correlation with skin conductance and with arousal ratings.

Finally, human fMRI research is consistent with animal studies that implicate the mesostriatum in reward processing. Both in picture perception and during narrative imagery, the nucleus accumbens and medial prefrontal cortex are selectively activated by pleasant stimuli, whereas the amygdala shows increasing activation with greater rated arousal for both pleasant and unpleasant content. Connectivity analyses reveal an appetitive circuit connecting all three structures, uniquely during pleasant picture processing.

In conclusion, both psychophysiological and neuroscience research supports the motivational view of human emotional expression, delineating the mediating neural structures and their autonomic and somatic reflex output . Emotions occur when appetitive or aversive cues activate primitive circuitry in the limbic brain that earlier evolved to promote the survival of organisms and their genetic inheritance.

Acknowledgments

The experiments reviewed here were in the main accomplished at the NIMH Center for the Study of Emotion and Attention at the University of Florida. Many significant contributors to the research reviewed here are now investigators at other institutions, including Andreas Löw (Greifswald University, Germany), Dean Sabatinelli (University of Georgia, USA), Maurizio Codispoti and Vera Ferrari (University of Bologna, Italy), Harald Schupp (University of Konstanz, Germany), Bruce Cuthbert (University of Minnesota, USA), Francesco Versace (MD Anderson Cancer Center, USA), Tobias Flaisch (University of Konstanz, Germany), and Laura Miccoli (Granada, Spain). We also take pleasure in acknowledging the contributions of current Center investigators including Vincent Costa, Andreas Keil, and Lisa McTeague. The research was supported in part by a National Institute of Mental Health grant, P50 MH-72850.

Footnotes

¹The finding of a pleasure-specific increase in NAc during picture processing contrasts with some studies suggesting that increased NAc activity is driven by stimulus salience (cues of significance, whether pleasant or unpleasant), rather than selectively by positive hedonic cues (see review by Leknes and Tracy, 2008). The data-base relating NAc activity to aversive stimulation is complex, however, varying with species (human or animal), specific brain measure (e.g., single unit activity, dopamine release, PET, fMRI, etc), type of aversive stimulation (pain, odor, taste, picture, image, etc.), and importantly, the processing paradigm (perception, anticipation, classical conditioning, operant conditioning, etc.). However, considering fMRI studies that assess direct exposure of pleasant/appetitive cues to human participants, the data are consistent with those presented here: NAc activity does not increase when people view aversive pictures (Meseguer et al., 2007; Phan et al., 2004), are exposed to painful thermal stimulation ( Becerra et al., 2001; Becerra & Borsook, 2008) or process cues signaling loss (Cooper & Knutson, 2008) or unpleasant odors (Gottfried, O'Doerty & Dolan, 2002).

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

Bradley MM, Miccoli L, Lang PJ. Tracking the eye during picture perception: Hedonic content and perceptual composition. 2009. Manuscript submitted for publication.

Bradley MM, Miccoli L, Lang PJ. Tracking the eye: Hedonic content, color, and composition. 2009. Manuscript submitted for publication.

Costa VD, Lang PJ, Sabatinelli D, Bradley MM, Versace F. Emotional imagery: Assessing pleasure and arousal in the brain's appetitive circuitry. 2009. Manuscript submitted for publication. [PMC free article] [PubMed]

LeDoux J. The emotional brain, fear, and the amygdala. Cellular and Molecular. 2003 [[PubMed] Google Scholar]

Mogenson GJ, Hillsdale NJ. The neurobiology of behavior: An introduction. Neurobiology. Vol. 23. L. Erlbaum Associates; New York: 1977. pp. 727-738. [Google Scholar]

Sabatinelli D, Lang PJ, Bradley MM, Costa VD, Keil A. The timing of emotional discrimination in human amygdala, inferotemporal, and occipital cortex. 2009. Manuscript submitted for publication.

# data[].attributes.attachment_link.description
Psychophysiological and neuroscience studies of emotional processing undertaken by investigators at the University of Florida Laboratory of the Center for the Study of Emotion and Attention (CSEA) are reviewed, with a focus on reflex reactions, neural structures and functional circuits that mediate emotional expression.

# data[].attributes.body
Simulations and Role-Playing-students engage in meaningful discussions and role-play about topics as if it was themselves in the situation.

# data[].attributes.attachment_link.readable_text
Department of Psychology, University of Michigan, Ann Arbor, MI, United States

This review takes a historical perspective on concepts in the psychology of motivation and emotion, and surveys recent developments, debates and applications. Old debates over emotion have recently risen again. For example, are emotions necessarily subjective feelings? Do animals have emotions? I review evidence that emotions exist as core psychological processes, which have objectively detectable features, and which can occur either with subjective feelings or without them. Evidence is offered also that studies of emotion in animals can give new insights into human emotions. Beyond emotion, motivation concepts have changed over decades too, and debates still continue. Motivation was once thought in terms of aversive drives, and reward was thought of in terms of drive reduction. Motivation-as-drive concepts were largely replaced by motivation-as-incentive concepts, yet aversive drive concepts still occasionally surface in reward neuroscience today. Among incentive concepts, incentive salience is a core motivation process, mediated by brain mesocorticolimbic systems (dopamine-related systems) and sometimes called 'wanting' (in quotation marks), to distinguish it from cognitive forms of desire (wanting without quotation marks). Incentive salience as 'wanting' is separable also from pleasure 'liking' for the same reward, which has important implications for several human clinical disorders. Ordinarily, incentive salience adds motivational urgency to cognitive desires, but 'wanting' and cognitive desires can dissociate in some conditions. Excessive incentive salience can cause addictions, in which excessive 'wanting' can diverge from cognitive desires. Conversely, lack of incentive salience may cause motivational forms of anhedonia in depression or schizophrenia, whereas a negatively-valenced form of 'fearful salience' may contribute to paranoia. Finally, negative 'fear' and 'disgust' have both partial overlap but also important neural differences.

Introduction

This review discusses the history of motivation and emotion concepts in psychology and affective neuroscience, drawing on both animal studies and human studies, in order to gain a better perspective on recent concepts and debates. In this discussion, I will sometimes combine motivation and emotion together because those psychological categories overlap in phenomena, making it hard to draw absolute distinctions. For example, is fear a stimulus-elicited emotion or a goal-achieving motivation? Or both?

Conversely, I will often distinguish between conscious or subjective feelings and unconscious or objective core processes within emotion and motivation. Although these often occur together, they sometimes need to be considered separately. To aid that distinction I will use quotation marks around 'wanting,' 'liking,' 'fear' and 'disgust' to distinguish those as objective core processes, that can be either merely unconscious or also conscious, from accompanying always conscious feelings denoted by the same words without quotation marks. The issue of whether emotions can exist independently of subjective feelings is important enough that we should deal with it right away.

Subjective Feelings Versus Objective Features of Motivation/Emotion

In the first half of the 20th century, to speak about subjective feelings was considered misguided by behaviorist psychologists and reductionist neuroscientists. To behaviorists and reductionists, subjective feeling measures were irrelevant, and feelings were mere epiphenomena. At best, they thought feelings were old-fashioned relics of early introspectionist psychologists, such as E.B. Titchener, who around 1900 analyzed introspective reports of sensory feelings using trained observers, but found results that were often peculiar to particular observers and in the end amounted to little of general value ( Titchener, 1902).

Today subjective feelings again are of wide interest, and for some the pendulum has moved to the opposite extreme. For example, Joseph LeDoux, a distinguished affective neuroscientist, recently argued that emotion is necessarily always and only a subjective feeling, and that verbal reports of feelings are the 'gold standard' of evidence needed to conclude that any emotion exists ( LeDoux, 2014; LeDoux J., 2015; LeDoux and Hofmann, 2018). For example, LeDoux and Hofmann write " subjective emotional experience, the feeling, is the essence of an emotion," and the only reliable " way to assess conscious emotional feelings is through verbal self-report" ( LeDoux and Hofmann, 2018, p. 67). For individuals who cannot speak about feelings, including animals, human infants, or brain-damaged aphasic human adults, LeDoux reverts to a stance shared with early behaviorists: such creatures are regarded to have mere reactions, without any emotional feelings at all. He suggests that " Infants can react "emotionally" long before they can feel emotion... Similarly, it is possible, in fact likely, that animals can react "emotionally" "without feeling" ( LeDoux, 2014, p. 2876). To react without feeling is not to have a true emotion at all, according to LeDoux, and he assumes that animal reactions lack any conscious feeling : " I thus assume, until proven otherwise, that a defensive organismic state and its constituent components are implicit (non-conscious)" (parenthetical phrase in original) ( LeDoux, 2014).

This is a striking change in position, because LeDoux is well known for earlier work that aimed to map fear in amygdala-related circuitry of rats. LeDoux considers now that the question of " whether animals react but do not feel, or whether they both react and feel, is, in my opinion, not something we can determine scientifically" ( LeDoux, 2014, p. 2876). To be unable to know something is to be agnostic. But LeDoux goes beyond an agnostic stance regarding animal emotions and instead adopts one of positive denial. LeDoux rejects his own former view of amygdala as a fear system, writing " I and others have called the brain system that detects and responds to threats the fear system (in rats). This was a mistake that has led to much confusion. ... I will propose and defend a different way of talking about this research, one that focuses on the actual subject matter and data (threat detection and defense responses)" ( LeDoux, 2014, p.2871). In short, LeDoux currently grants animals to have brain threat detectors and the capacity for defense responses, but neither fear as a psychological emotion nor the brain systems needed for emotional feelings.

LeDoux notes that his view is similar to the dictionary definition of emotion as a subjective feeling. He is also not alone among modern psychologists in insisting that emotions must always be conscious ( Clore, 1994), nor in denying emotional feelings to animals ( Barrett, 2017). For example, the prominent psychologist Lisa Feldman-Barrett in her recent book on emotion raises the question ' does a growling dog feel anger?', and answers " The answer is almost certainly no. Dogs do not have the human emotion concepts necessary to construct an instance of anger" ( Barrett, 2017, p. 269). This rejection of animal emotion resembles LeDoux's, but is on different, more cognitive, grounds. In Feldman-Barrett's view emotion requires the complex cognitive appraisals, language-based reasoning and sociocultural construals of situations and meaning that only humans possess. This position continues a long tradition of earlier cognitive appraisal theories that reinterpreted emotions as essentially just another type of cognition, turning emotions essentially into cultural-linguistic representations of semantic meaning ( Arnold, 1960; Lazarus, 1981; Clore and Ortony, 1984; Ellsworth, 1994; Russell and Barrett, 1999; Ellsworth and Scherer, 2003). Cognitivist academics focus on reasoning and language, and place such a high premium on rationality that they are inclined to see all psychological processes through a purely cognitive lens.

Regarding consciousness, older versions exist in psychology for the notion that emotions must be subjectively felt. The early 1900s position of Sigmund Freud, despite his reputation as father of the psychological unconscious, presaged the modern assertion that emotions are always and necessarily felt. Although Freud held that many psychological processes could be unconscious, he asserted that emotions in particular must always be conscious. Posing the question, " are there also unconscious instinctual impulses, emotions and feelings...?", Freud answers himself no, because " for emotions, feelings and affects to be unconscious would be quite out of the question" ( Freud, 1950, pp. 109-110). Freud did not go on to explain particular reasons for why he believed emotions must always be conscious, when he granted that cognitions, memories and perceptions could all occur in unconscious forms. But after all, Freud's patients were paying him to alleviate their conscious emotional distress.

On the other hand, many contemporary psychologists and affective neuroscientists do believe that affective reactions and emotions can occur unconsciously as implicit processes, as well as subjective feelings ( Zajonc, 1980; Morris et al., 1998; Damasio, 2004; Winkielman et al., 2005; Anderson and Adolphs, 2014; Berridge and Kringelbach, 2015; Frijda, 2016; Greenwald and Banaji, 2017). Many have similarly argued that emotions are shared with animals, and that emotional brain systems arose early in evolution, well before human cognitive abilities such as language-based cognition ( James, 1884; Solomon and Corbit, 1974; Zajonc, 1980; Ekman, 1999; Damasio, 2004; Winkielman et al., 2005; Frijda, 2007; Frijda and Parrott, 2011; Panksepp, 2011; Anderson and Adolphs, 2014; Berridge and Kringelbach, 2015). These views are very well expressed by the distinguished affective neuroscientist Antonio Damasio in a recent book on affect, emotions and feelings ( Damasio, 2018). On whether brain systems for emotional feelings are shared with animals, Damasio writes, " Moreover, the emergence of subjectivity is not recent at all, let alone exclusively human. It is likely to have happened long ago, over the Cambrian period" (p. 238). That is, by his view the necessary neural circuitry first evolved several 100 million years ago, being embedded in ancient wiring patterns that are largely contained in deep brain structures below the cortex ( Swanson, 2005; Strausfeld and Hirth, 2013). And regarding the need to conceptually distinguish between feelings and emotions, despite their overlap in everyday experience, Damasio suggests, " the felt experiences of emotions are unfortunately known by exactly the same name as the emotions themselves. This has helped perpetuate the notion that emotions and feelings are one and the same phenomenon, although they are quite distinct" ( Damasio, 2018, p. 100).

My own view is similar: that affective reactions can occur unconsciously, as well as consciously, and that we share many emotional processes and their brain circuitry with animals. But assertions, pro or con, are only an entry point to this discussion. We need reasons and evidence to form adequate conclusions. What is the actual evidence regarding emotions? Is it an adequate theory for psychology to say that animals don't have emotions or that emotions must be conscious?

Quite a lot of evidence actually contradicts contentions that emotions are always subjective feelings, and instead suggests that emotions can also occur even in people without being subjectively felt - at least under some conditions. These emotional reactions occur as core affective processes that can remain intrinsically psychological and emotional, even without conscious feelings, and which have objective consequences and features that can be detected in physiology and/or behavior. However, before sketching that evidence, I should point out one important point of agreement between my view and those of Damasio and LeDoux, despite differences on whether emotion exists in animals or is always necessarily conscious. We all agree it is important to distinguish between subjective emotional feelings and objective emotional reactions. The difference has important implications for understanding normal psychological processes and for understanding psychological disorders.

Avoiding Double-Standards for Proof

Perhaps the main reason LeDoux gives for denying emotions to animals or infants is that we cannot know with scientific certainty that they have subjective feelings: we have no proof of what they feel. That objection has merit, in the sense that we do lack proof and scientific certainty about conscious feelings of animals and infants. But proof and certainty are the key words here, and we should be careful to avoid double standards in using them. We also lack proof and scientific certainty about the subjective experience of every other human adult besides ourselves. Can I be scientifically certain about your subjective feelings? Can you say with scientific certainty whether your fear feels the same as mine? Do I know with certainty when you are happy or sad? Certainty is not the form of knowledge we have about subjective experience in any other mind than our own ( Nagel, 1974).

Yet LeDoux is willing to accept verbal reports from fellow adult humans about feelings as evidence of emotion on the grounds (1) that " If my brain can be conscious, so can yours" and (2) "because our species is naturally endowed with language, we can share...the amazing sight of the sun setting over the ocean" ( LeDoux J., 2015, p. 49). All of us might agree. But this is a socially convenient agreement based on mutual empathy. It is essentially an assumption we are willing to make for each other (and perhaps evolutionarily compelled to make), but not a scientific certainty. It is mere conjecture to say that only other humans have the crucial brain similarity needed for consciousness but that animals do not. Clearly there is no scientific certainty about the minimum neural circuitry needed for consciousness: neuroscientists disagree among themselves over whether the essential basis is cortical versus subcortical, and so on. Dropping a demand for scientific certainty for emotion in other humans, but insisting on it for animals, is to adopt a double standard on proof. It is entirely understandable that we discard demands for scientific certainty in social interactions with other people. But let's not delude ourselves that we are being consistent about scientific certainty if we apply higher standards to animals. And let's not apply a double standard - there's nothing scientific about that.

However, even most science is not about proof in the strong sense of certainty. Indeed, some philosophers of science have argued that scientific data never fully prove any hypothesis with complete certainty (there is always a potential alternative lurking in the wings), but only can falsify bad hypotheses ( Popper, 1972). In fact, most science is rather a matter of reducing uncertainty by degrees, experiment by experiment, gradually building incremental evidence for a particular hypothesis, and gradually ruling out specific alternatives. Uncertainty almost never goes away completely. Even successful hypotheses remain forever vulnerable to being challenged and replaced. Demanding certainty before taking a hypothesis seriously would stop a lot of valuable science in its tracks.

Resolving Whether Emotions Are Necessarily Feelings

So if scientific certainty is not the way to decide about emotions, what is? Three levels of consideration seem to me relevant in deciding the question of whether emotions are exclusively conscious: (1) a priori definition, (2) available evidence on unconscious emotions, and (3) applications from conclusions based on emotional studies in animals to understanding human emotions and psychological disorders.

First, definition: it is perfectly legitimate to say "By emotion, I mean a conscious feeling." That's just a matter of personal word use. But that doesn't tell us anything about the essence of emotion as psychological fact. To go on to claim that emotion as a psychological process is exclusively a conscious feeling is to make an empirical claim about psychological fact. This claim requires actual evidence, which must be looked for in psychological studies. No one can define facts in advance.

Second, what is the evidence? Quite a lot of evidence exists, and in my view it does not support the idea that emotions are necessarily conscious, nor that verbal reports are the best way to assess emotion. Considerable experimental evidence indicates that some features of human emotion and motivation cannot actually be accessed well via introspection or described in subjective reports ( Nisbett and Wilson, 1978; Wilson and Schooler, 1991; Schwarz, 1999; Dijksterhuis et al., 2006; Schooler and Mauss, 2010; Greenwald and Banaji, 2017; Torre and Lieberman, 2018).

Studies of implicit prejudice similarly suggest that introspective verbal reports may miss some important emotional reactions ( Greenwald and Banaji, 2017). Implicit prejudices can only be revealed by objective measures, such as the emotional Stroop test of reaction time to affective mismatch, sometimes to the surprise and dismay of the person who is subjectively unprejudiced. Writing about implicit prejudices, the distinguished psychologists Anthony Greenwald and Mahzarin Banaji conclude, " When people attempt to report on their conscious perceptions and judgments, they do so not based on valid introspection but by using traces of past (possibly biased) experience to construct (possibly invalid) theories of current data." (italics and parenthetical phrases in original; Greenwald and Banaji, 2017, p. 868). What is subjectively reported may only be a constructed explanation of what we think we should feel or would like to feel, rather than an accurate readout of underlying emotional reactions. In other words, introspective reports can be very far from 'gold standard' evidence about underlying emotional processes.

There is also evidence that basic affective reactions can be triggered unconsciously by subliminal stimuli in ordinary humans, and can remain unreportable as feelings even when they go on to change a person's behavior and subsequent judgments ( Zajonc, 1980; Fischman and Foltin, 1992; Berridge and Winkielman, 2003; Winkielman and Berridge, 2004; Winkielman et al., 2005). For example, cocaine addicts under some conditions will objectively work to take cocaine infusions, and choose them over saline infusions, even when the cocaine dose is too low to produce any subjective drug feelings of pleasure or arousal ( Fischman and Foltin, 1992). Further, in ordinary adults under certain circumstances, subliminally brief visual flashes of happy or angry facial expressions can elicit unconscious affective reactions, either positive or negative, which produced no change in subjective mood reports but nonetheless controlled motivated behavior some minutes afterward ( Winkielman et al., 2005). In that study, before and after seeing subliminal faces, participants rated their emotional feelings. Their subjective ratings of mood or emotion were not changed at all by exposures to subliminal and backward-masked emotional faces, whether happy or angry ( Winkielman et al., 2005). Yet subliminal exposure to happy faces elicited a positive affective reaction that remained unconscious, but which could be revealed by presenting thirsty participants with a relevant incentive: a pitcher of fruit beverage. Those happy-exposed participants poured more drink, drank more of what they poured, and were willing to pay a much higher monetary price for the drink offered than after viewing emotionally-neutral faces. It might be objected that subliminally viewing a happy facial expression merely increased unvalenced arousal, which increased drink motivation, and was not truly affective. However, the test of true affect is whether it is valenced positive versus negative. Valence was evident in that subliminal viewing of angry faces had an opposite negative-valenced impact on the same thirsty participants: they poured less, drank less, and were willing to pay less than after seeing only neutral facial expressions - again without any intervening change in their subjective ratings of emotional mood ( Berridge and Winkielman, 2003; Winkielman et al., 2005).

Finally, regarding language as a gold standard for emotion, talking about feelings evoked by sunsets is not proof even of the emotions that are declared. Linguistic computer programs may soon pass an emotional Turing test by talking compellingly about enjoyment of sunsets, so that a listener thinks the computer has emotional feelings. Even in the 1970s some people poured out their own feelings to ELIZA, a linguistic computer program with scripted responses that simulated the responsive questions of a Rogerian psychoanalyst via text messages, and seemed to human interlocutors to convey great emotional sensitivity (e.g., Client typed: "It's true. I am unhappy." ELIZA responded: Do you think coming here will help you not to be unhappy? Client: "I need some help, that much seems certain." ELIZA: What would it mean to you if you got some help? Client: "Perhaps I could learn to get along with my mother." ELIZA: Tell me more about your family.) ( Weizenbaum, 1977). Modern artificial intelligence is creating robotic companions with linguistic scripts that are far more compelling and flexible, able to talk with their human owners about emotions as though they shared them - even when they don't. In humans too, as already mentioned talking about emotions may cognitively distort underlying processes based on what people think their emotions should be, rather than accurately report emotional content ( Wilson and Schooler, 1991; Dijksterhuis et al., 2006; Schooler and Mauss, 2010). In short, talking about emotions is often not a reliable window or 'gold standard' source of evidence into actual underlying emotions, even when verbal declarations of emotion are available.

Applications of Animal Studies to Human Emotions

In animals as well as humans, psychologists can objectively map the shape of emotional/motivational processes from the outside, by finding the lawful rules that govern their operation in action, even when we don't know what is subjectively felt on the inside ( Nagel, 1974). This aims for a schematic understanding of other minds, rather than an introspective mirroring of phenomenal experience. Schematic understanding is the way we study perception, learning, memory and cognition in animals. It's even how we measure and understand perception in people whose visual world differs from ours (e.g., detecting color blindness). What is a schematic understanding of emotion? As the comparative psychologists William Mason and John Capitanio put it, " Our approach to the ontogeny of basic emotions is based on the phenomenon of component schemas. Schemas are hypothetical information-processing units, closely linked to observed behaviors. Most if not all schemas are affectively charged" ( Mason and Capitanio, 2012, p. 240). Similarly, the psychologists of emotion Nico Frijda and W. Gerrod Parrott, posit that animals as well as humans can express 'ur-emotions,' as objectively detectable psychological processes that combine appraisals, affective attitudes and action readiness. Their "ur" label is meant to imply that such emotional reactions may be relatively primitive and fundamental, which overlap but are not necessarily identical with traditional "basic emotions" categories (e.g., anger, joy, etc.). They suggest: " each animal species has at its disposal some innate primary actions that implement the functions we have assigned to the ur-emotions" (p. 412), including the "ur-emotion of desire in its "urest" or purest or most target-free form: as mere wanting" ( Frijda and Parrott, 2011, p. 413). Mere 'wanting' refers to incentive salience, discussed below. Other psychologists and neuroscientists have expressed similar views about fundamental affective reactions in animals and human infants ( Darwin, 1872; Berridge, 2000; Keltner and Ekman, 2000; Steiner et al., 2001; Damasio, 2004; Panksepp, 2011; Anderson and Adolphs, 2014; Berridge and Kringelbach, 2015; Frijda, 2016).

Of course, one goal of animal studies is to gain insights that have application to understanding human emotions and motivations. For that, we need to posit continuity between those affective mechanisms in animals and humans. Otherwise, it would be impossible to make the jump from discoveries about animal affective processes, to human affective processes or to human clinical disorders of addiction, depression, schizophrenia, etc. This jump is not always easy. Indeed another reason LeDoux gives for denying that animals are capable of fear is because animal fear conditioning studies failed to provide new effective medications for human anxiety or panic disorders. However, this lack of therapeutic development could just as well mean that Pavlovian fear learning (e.g., behavioral freezing to a sound that predicts footshock) was not the best animal 'fear' reaction for assessing the efficacy of fear-reducing medications. It is an unfounded leap to conclude instead it means that rats are incapable of fear. Other animal tests of fear or anxiety using different reactions and situations might be more successful. In any case, a counterargument has been made that many of the current anti-anxiety medications used by human patients actually were developed through various animal models of 'fearful' reactions ( Treit et al., 2010; Bourin, 2015). And there are many other cases where animal studies of emotion have produced results with successful implications for understanding human psychology and disorders. I believe these include a few from my own lab, and one is the brain-based distinction between 'wanting' and 'liking' pleasant rewards (Figure 1), described next.

FIGURE 1. Brain systems of 'wanting' versus 'liking'. Robust and large mesocorticolimbic circuitry can generate intense 'wanting' (green), including both mesolimbic dopamine projections and many of its target structures. By comparison, 'liking' is mediated by a smaller and relatively fragile set of 'hedonic hotspots,' which are distributed across the brain but act as an integrated network.

'Wanting' Versus 'Liking': Implications From Animals to Humans

The distinction between 'wanting' and 'liking' initially came as a surprise to me and to my colleagues. We had believed until the late 1980s that the two concepts necessarily went together, as two semantic sides of the same psychological coin. Evidence from our studies of brain mechanisms for food reward in rats forced us to change our minds. Today the distinction between 'liking' and 'wanting' now rests on 30 years of results in animal studies, and 15 years of confirmatory findings in humans. The distinction has also turned out to have applications to human affective disorders, ranging from addiction to schizophrenia, depression and Parkinson's disease. But for the first decade of this hypothesis, the schematic understanding from animal evidence was all we had, based on objective affective reactions, which stood alone and pointed the way.

Our initial discovery came from studies of the role of brain mesolimbic dopamine systems in reward. In the 1980s, we and most other psychologists and neuroscientists believed that dopamine mediated pleasure or 'liking': the hedonic impact of reward. This was best expressed at the time by the dopamine pleasure hypothesis of the neuroscientist Roy Wise, who had suggested that mesolimbic dopamine signals are " translated into the hedonic messages we experience as pleasure, euphoria, and 'yumminess"' ( Wise, 1980, p. 94). Conversely, Wise suggested an opposite anhedonia hypothesis for the psychological effect of drugs that blocked dopamine receptors (often called neuroleptic drugs or dopamine antagonist drugs), which posited those drugs to cause tasty foods and all other reward to lose their pleasure or hedonic impact. His evidence was that such drugs produced 'extinction mimicry': making animals or people gradually cease to pursue or consume the reward, as though their pleasure had been drained, similar to real extinction procedures in which expected reward are suddenly missing, causing animals to extinguish or gradually cease working on tasks that formerly produced the reward. To give credit where due, some other neuroscientists, such as John Salamone and Jaak Panksepp had also expressed early doubts about the dopamine-as-pleasure hypothesis. Salamone had disputed Wise's anhedonia hypothesis around 1990, on the grounds that neuroleptic extinction mimicry did not fully mimic all features of real extinction, and because he noted that mild-to-moderate dopamine suppressions often reduced effort exerted for reward without necessarily also reducing reward consumption ( Salamone, 1991; Salamone et al., 2015). Panksepp suggested what he called an 'expectancy hypothesis' for dopamine, in which it caused the " heightened energization of animals searching for and expecting reward" (while also contributing to positive affect in his view, as a combined motivational/hedonic incentive mechanism) ( Panksepp, 1986, p. 91).

But I must admit myself to having been mostly convinced by Wise's elegant evidence for mesolimbic dopamine as pleasure until we began to probe dopamine ourselves in a schematic approach to specifically measuring 'liking.' Dopamine then failed to live up to its hedonic reputation. To my initial surprise and disappointment, our results on sweetness 'liking' reactions in rats indicated that dopamine did not mediate pleasure 'liking' after all. We were using an objective measure of affective 'yumminess' in rats similar to that used by human parents for thousands of years to ask their newborn infants if they 'liked' the foods eaten by the families: affective facial expressions of 'liking' elicited by a small taste of the foods ( Steiner et al., 2001). In human infants, sweet tastes typically elicit relaxed and rhythmic mouth and tongue movements, and licking of the lips. Bitter tastes oppositely elicit 'disgust' reactions of mouth gapes, headshakes, arm flails and so on. Some of these affective facial expressions to taste are shared by apes and other primates, and even by rats and other non-primates ( Grill and Norgren, 1978; Berridge, 2000). Using this schematic measure of 'liking' versus 'disgust,' we found that drug disruption of dopamine failed to reduce facial 'liking' reactions to sweetness, contrary to our anhedonia-based prediction ( Treit and Berridge, 1990). Further, even complete removal of nearly all dopamine by neurochemical brain lesions left rats' sweetness 'liking' completely normal and unimpaired, despite eliminating all 'wanting' to eat ( Berridge et al., 1989). Conversely, we found that stimulating mesolimbic systems with an electrode to promote dopamine release turned on intense 'wanting' to eat, but failed to increase 'liking' at all ( Berridge and Valenstein, 1991). Later studies in our lab went on to identify various psychological signature features of dopamine-mediated 'wanting' or incentive salience, which will be described later ( Wyvell and Berridge, 2000; Peciña et al., 2003; Smith et al., 2011; Pecina and Berridge, 2013). In short, the early animal evidence showed that 'liking' was different from 'wanting,' and that dopamine mediated 'wanting' but not 'liking' (Figure 1). But only animal evidence existed for these hypotheses until nearly 2000. That made ours a lonely position, because most affective neuroscientists still wrote of dopamine as a pleasure mechanism throughout the 1990s and often into the 2000s.

However, gradually a number of human studies began to emerge around 2000 that confirmed our dopamine-based liking/wanting distinction also applied to people's feelings of pleasure produced by cocaine, heroin, and food reward. Studies began to report that blockade of human dopamine receptors by antagonist drugs, or dietary-induced reductions of dopamine release, did not reduce people's subjective liking ratings for food pleasure, or for drug pleasures of cocaine or heroin ( Brauer and De Wit, 1997; Leyton, 2010; Sienkiewicz-Jarosz et al., 2013). However, those dopamine suppressions did reduce people's subjective reports of wanting to consume more of the drug or food reward.

Similarly, it is now increasingly clear that increases of human dopamine do not reliably cause enhancement of subjective ratings of pleasure (although many rewards do induce dopamine release as a consequence and correlate, but not the cause, of pleasure) ( Leyton et al., 2002; Evans et al., 2006; Liggins et al., 2012). For example, giving ordinary people the medication L-DOPA, which produces surges in brain dopamine levels, does not increase their subjective ratings of pleasure feelings ( Liggins et al., 2012). Dopamine surges in nucleus accumbens evoked by addictive amphetamine or by L-DOPA correlate poorly with human subjective liking ratings of the drug - but do control their subjective wanting ratings to take more of that drug ( Leyton et al., 2002; Evans et al., 2006).

Thus, human confirmation began finally to amass for the 'liking' versus 'wanting' distinction we originally found in rats a decade earlier. Was our animal-based conclusion about human liking and wanting simply a lucky guess? We must have been very lucky indeed 10 years before the human confirmation began to emerge - unless we were accurately tapping in all along to 'liking' versus 'wanting' processes shared by animals and humans, which made it not a guess at all. Instead it was a schematic inference about hedonic and motivation mechanisms of reward gained from animals, with direct applications to human psychology and clinical affective disorders.

Clinical Applications of 'Liking' Versus 'Wanting'

The implication for human addiction is that sensitized drug addicts could have intense cue-triggered 'wanting' for their drugs, especially when encountering drug cues in emotionally aroused states, even if their 'liking' for drugs declined ( Robinson and Berridge, 1993; Berridge and Robinson, 2016). Originally we applied this incentive-sensitization theory only to drug addiction, because only drugs were then known to induce mesolimbic neural sensitization. However, it has subsequently become clear that it is also possible to induce mesolimbic sensitization without drugs in susceptible individuals, such as by exposures to traumatic stresses, exposures to strong specific appetites, etc. Recently it has even become plausible that some people, who may be especially vulnerable to developing mesolimbic sensitization, may become 'spontaneously' sensitized in particular situations via endogenous mechanisms, resulting in behavioral addictions.

There are strong individual differences in sensitization vulnerability, due to genes, hormones, previous experiences, etc. Presumably those who develop behavioral addictions are the most vulnerable individuals, able to develop mesolimbic sensitization via endogenous mechanisms, without need of drugs. Evidence that sensitization happens in behavioral addictions is that fMRI studies have reported individuals with gambling addiction, shopping addiction, internet addiction or sex addiction, to exhibit brain signatures of mesolimbic sensitization to appropriate stimuli: that is, neural hyper-reactivity to their own addictive cues, higher than non-addicted people show to the same reward, and higher than the same addicted individuals show to cues for other (non-addicted) rewards ( Davis and Carter, 2009; Gearhardt et al., 2011; O'Sullivan et al., 2011; Hartston, 2012; Linnet et al., 2012; Ray et al., 2012; Voon et al., 2014). This sensitized brain reaction evoked by addictive cues should produce stronger cue-triggered psychological surges of 'wanting' than faced by other non-sensitized individuals. Those addictive urges would be further amplified when the cues were encountered during stress, emotional excitement, intoxication or other states that prime the reactivity of mesolimbic systems. This could produce extra-strong 'wants' in behavioral addictions that sometimes surprise even the addict by their intensity.

Beyond addictions, the liking/wanting distinction also has been applied to nearly the opposite psychological condition: namely, a motivational component of 'anhedonia' syndromes in schizophrenia, depression, and unmedicated Parkinson's disease. These conditions have traditionally have been described as involving anhedonia or incapacity to experience pleasure (similar to the original Wise hypothesis that dopamine blockade reduced pleasure). However, recent studies suggest that many patients with schizophrenia or Parkinson's, and possibly some with major depression, are not actually anhedonic after all: they report normal sensory pleasure ratings of ice cream or other sensory reward in the moment, even though they attach little value to any sensory, social or other pleasant reward in life. Some investigators therefore have suggested their deficit really reflects a selective loss of incentive motivation or 'wanting,' rather than loss of pleasure 'liking' ( Sienkiewicz-Jarosz et al., 2005; Dowd and Barch, 2010; Treadway and Zald, 2011; Barch et al., 2014). They call this motivational deficit 'avolition,' 'anticipatory anhedonia' or 'motivational anhedonia,' to contrast it with true 'anhedonia' or 'consummatory anhedonia' that would reflect loss of pleasure per se. Like addiction, these clinical applications draw on a psychological understanding of the separability of 'wanting' from 'liking' that originally came from animal studies.

History of Motivation Concepts in Psychology and Neuroscience

Now we will shift gears, and turn to a very different question. Namely, the nature of motivation processes themselves, and how motivations actually control behavior. Motivation concepts have changed a lot over the last few decades. For example, from 1900 to the 1970s, much of the thinking about reward motivation in psychology and neuroscience was dominated by two concepts: drive and drive reduction ( Hull, 1951; Miller, 1971). Drive was typically conceived as an aversive state that goaded behavior into action to reduce the unpleasant drive (hunger, thirst, sex, drug withdrawal, etc.). We'll focus on here natural hunger and thirst, and on drug addiction, as clearest examples.

Drive reduction served as reward in this motivational framework; not as a positive pleasure or incentive, but in a negative reinforcement sense of eliminating the aversive drive. The notion that motivation is mostly to escape unpleasant states is a bit reminiscent of William James' quip in a 1901 letter on happiness (published with other letters in 1920) written shortly after his exhausting experience of delivering the famous Edinburgh Lectures that he subsequently revised into a book ( James, 1902). " Happiness, I have lately discovered, is no positive feeling, but a negative condition of freedom from a number of restrictive sensations of which our organism usually seems the seat... When they are wiped out, the clearness and cleanness of the contrast is happiness. This is why anesthetics make us so happy. But don't you take to drink on that account" ( James, 1920, p. 158).

Natural motivations were viewed by drive-reduction theory as generators of aversive states, triggered by signals for physiological homeostatic needs (low nutrient reserves for hunger, dehydration for thirst) or of other forms of deprivation (lack of sex, withdrawal from addictive drugs, etc.) ( Hull, 1951; Miller, 1971). Their respective rewards were all viewed to reduce the unpleasant drives, by consuming the drive's goal object until the aversive deficit state was gone (food, water, copulation, addictive drug, etc.). Today, drive reduction theories can still be found in a few cases, such as theories of drug addiction based on withdrawal symptoms, or in the conclusions of a few recent neuroscience studies of hunger or thirst.

For example, in drug addiction, the opponent-process theory posited negative hedonic states of withdrawal and dysphoria to be the chief force driving addicts to take drugs ( Solomon and Corbit, 1974), a view that still has adherents ( Koob and Volkow, 2016; Keramati et al., 2017). Modern proponents suggest that downregulation or loss of brain dopamine D2 receptors (one of the two main types of neuronal receptors for dopamine) make addicts experience less pleasure in their lives than other people ( Koob and Volkow, 2016; Keramati et al., 2017). Consequently, these theorists argue that addicts must take addictive drugs to achieve normal levels of dopamine stimulation and pleasure. However, this hedonic deficiency view has been critiqued on grounds that dopamine receptor downregulation is probably a consequence of drug taking rather than the original cause of addiction, on grounds that dopamine does not actually cause pleasure nor does dopamine downregulation cause pleasure deficits, and on grounds that addictions persist after withdrawal syndromes go away ( Berridge and Robinson, 2016).

Recent studies of hunger and thirst neuronal circuitry provide a few more examples that drive reduction ideas persist today. Hunger neurons are thought to include those in the hypothalamus that release agouti-related peptide (AgRP), and stimulating those neurons cause mice to eat. However, a mouse may subsequently avoid a flavor that was paired with the AgRP neuronal stimulation that made it eat, leading some authors to suggest that stimulated hunger is necessarily an aversive drive that the mouse later associates with the flavor ( Betley et al., 2015). However, other neuroscientists have found that mice will actually work to turn on their hypothalamic AgRP neurons that previously made them eat, indicating that the increased appetite can be dissociated from negative valence, and even take on a positive valence in the right conditions ( Chen et al., 2016). This makes a pure drive reduction theory of AgRP hunger less plausible, similar as to what happened for hypothalamic reward electrodes discussed below. Recently, a thirst drive-reduction hypothesis was suggested by authors of a study similar to the first AgRP one for hunger ( Allen et al., 2017). These thirst-study authors found that artificial stimulation of neurons in the hypothalamus made mice drink, but that mice would work to turn off the stimulation if given a choice, thus leading the authors to suggest that thirst is essentially aversive.

Yet if history repeats itself, the aversiveness of the hypothalamic neuronal activation may in future turn out to be separable from its ability to trigger drinking ( Toates, 1986; Berridge, 2004). If so, aversiveness above could in part due to relatively extreme or unnatural parameters of the earlier neural stimulation. The crucial test will be if future studies with other stimulation parameters can eventually tease apart drinking elicitation from aversive effects - possibly even finding that activation of the drinking neurons can become sought under some circumstances. That could help dissolve these new drive-reduction explanations into a future incentive motivation explanation for the same hypothalamic circuitry. This doesn't deny that intense hunger, thirst, etc., can be unpleasant. Of course they can, giving drive-reduction theories an eternally intuitive appeal. However, the brain's motivation circuitry may be mostly oriented toward incentive processes, even for hunger, thirst, and the other biological motivations that drive reduction theory originally was invented to explain.

The Fall of Drive Theories

Perhaps the single most compelling piece of evidence that moved motivation theory away from negative drive theories to positive incentive theories of motivation came from 1960s studies of reward and hunger motivation caused by brain electrode stimulation - mostly at sites in the lateral hypothalamus that indirectly also activated mesolimbic dopamine systems ( Olds and Milner, 1954; Valenstein et al., 1970; Miller, 1971). It quickly became clear that brain reward electrodes were not only rewarding (in the sense that rats and people worked eagerly to activate them), but the same electrodes were also often powerfully motivating in the sense of apparently turning on natural motivations. The electrodes may actually not have caused much pleasure 'liking' after all, but they certainly activated 'wanting' for a variety of natural incentives ( Berridge and Kringelbach, 2015). That is, simply giving free stimulations of the electrodes, without making them work for it, caused rats to suddenly start eating, or start drinking, or begin engaging in sex, or parental behaviors, etc. ( Valenstein et al., 1970). Similarly in human psychiatric patients who had been implanted with similar brain electrodes, activation of a reward electrode caused sudden sexual urges, or urges to drink, or other intense motivations, in addition to supporting button pressing to self-stimulate their electrodes ( Moan and Heath, 1972).

The finding that the same electrode was both rewarding (in the sense of being sought after) and motivating confounded expectations based on drive theories. According to drive theories, the intense motivations had to be aversive. According to drive reduction theories, reward was produced by reducing motivations, not by increasing drives. Thus behavioral neuroscientists of the time expected that an electrode that increased sex or hunger or thirst motivations, would be a punishing electrode. They expected that a reward electrode would reduce drive, and so stop any ongoing eating, drinking, or sex behavior - never stimulate those motivations. To drive theorists, a reward electrode that also increased motivation is an inexplicable paradox ( Miller, 1973; Olds, 1973; Stuber and Wise, 2016).

How can that paradox be resolved? The fact that an electrode that caused eating behavior (or drinking, or sex, etc.) was also a reward electrode meant that the reward could not be due to drive reduction. Since the rewarding electrode actually increased the apparent drive to eat, reward needed to be understood as a phenomenon that was independent from drive reduction. The paradox was eventually solved by incentive motivation theory described below.

Other evidence has indicated similarly that the aversiveness of drives does not actually motivate much behavior, even for hunger and thirst. Nor is the reduction of an aversive drive actually the chief target of those motivations. For example, when in a place that has been repeatedly associated with hunger, or paired with thirst, one might expect a re-encounter of that place to trigger Pavlovian conditioned (cue-triggered) hunger or thirst again, and increase eating or drinking. That should happen if re-activation of the aversive need-state or drive motivates eating or drinking behavior. But that does not happen. For example, rats typically fail to increase eating in a place previously paired with hunger drive, nor to increase drinking in a place paired with thirst drive ( Huston, 1972; Mineka et al., 1972; Toates, 1986). Pavlovian cues for hunger fail to elicit ingestive behaviors, that is, unless those cues were additionally paired with the opportunity to eat while hungry, or the opportunity to drink while thirsty. However, if that extra pairing with food consumption while hungry is given, then later the Pavlovian food cue can evoke conditioned eating even if encountered when the rat is no longer hungry ( Weingarten, 1983; Holland et al., 2002; Petrovich, 2011). That is, a food incentive cue elicits eating, but a hunger cue does not. Similarly, for thirst,: encountering a place paired with thirst drive does not alter behavior, but encountering a cue for water will evoke intense increases in activity in thirsty rats, as though searching for water ( Campbell, 1960). In these cases, cues for the incentive stimulus evoke motivation in a state, whereas cues for the drive state by itself do not. Hunger acts to enhance the incentive value of food, and thirst enhances the incentive value of water - but in the absence of incentives or their learned expectation, neither need state effectively motivates behavior as posited by aversive drive theory.

Not only is drive a weak motivator by itself, but drive reduction turns out to be surprisingly impotent as a reward by itself. Reduction of a hunger or thirst drive is usually not enough to reinforce behavior, unless an incentive stimulus is also involved. For example, rats that can deliver nutrients directly to their stomach via feeding through a gastric fistula in some studies fail to bar press for the nutrients at all ( Holman, 1969). However, if opportunity to taste a mouthful of saccharin at the same time as a gastric nutrient infusion, then the rats do learn to bar press for the combination (the sweet taste by itself is similarly insufficient, revealing an importance of interaction between incentive stimulus and physiological state, captured by Bindra-Toates rules of incentive motivation described below) ( Holman, 1969). Even studies that succeeded in getting rats to work for intra-gastric delivery of nutrients reported only weak effects: such rats bar press to earn only about 30% of their normal intake of calories ( Nicolaidis and Rowland, 1975). Similarly, hungry rats merely walked slowly to a place where they expect intra-gastric milk, but eagerly ran to a place where they could drink and taste the milk while hungry ( Miller and Kessen, 1952). And drive reduction is not even satiating: reducing a physiological drive often does not effectively reduce the motivated behavior. For example, delivering daily calories intravenously to dogs, who were also allowed to eat actual food normally if they wished, did not suppress their daily eating intake: the dogs continued to eat customary amounts of food in addition to their full doses of intravenous nutrients, and so soon became obese ( Turner et al., 1975). All these observations indicate that the role of physiological 'drive states' is really to amplify the incentive value of their relevant incentive targets, and magnify their reward properties, rather than act as independent drives ( Toates, 1986; Berridge, 2012).

Incentive Motivation: Relating 'Liking' to 'Wanting'

What is the alternative to drive theories? Incentive motivation theories posit that motivation is directed toward affectively positive incentives, and that brain motivation systems modulate those incentive values ( Toates, 1986; Berridge, 2004). Hunger, thirst, and other motivation states primarily act to enhance the incentive value of their particular reward, increasing 'wanting' and 'liking' for foods, or for water, and so on, rather than acting primarily as aversive goads ( Toates, 1986; Cabanac, 1992; Dickinson and Balleine, 2002).

Incentive motivation is focused on reward, which involve three categories of mechanism: wanting, liking and learning. 'Liking' reactions reflect the hedonic impact of pleasant reward, and is often considered the essentional kernal of reward. What in the brain is responsible for 'liking' reactions? Our research findings have indicated that pleasure 'liking' is generated by an anatomically small, neurochemically restricted, and functionally fragile brain circuit - which leaves out the large dopamine projection system of 'wanting' ( Berridge and Kringelbach, 2015). The relative fragility of 'liking' circuitry, compared to the robustness of 'wanting' circuitry, may be one reason why intense pleasures are fewer and farther between in life than intense desires. Only a network of small 'hedonic hotspots' is able to amplify 'liking' reactions to a sensory pleasure, such as sweetness.

Hedonic hotspots are anatomically small pleasure-generating islands of brain tissue, tucked within larger limbic structures, such as nucleus accumbens and limbic cortex (Figures 2, 3). The size of each hotspot discovered so far is only a cubic millimeter or so in volume in the brain of a rat. In the brain of a person, a hotspot would be expected to be about a cubic centimeter in volume, extrapolating from the size difference between whole brains.

FIGURE 2. Synergy of cues and internal state in generating intense 'wanting.' A Pavlovian cue lever for disgusting Dead Sea saltiness becomes negatively repulsive during learning (left). But in a novel salt-appetite state, the cue becomes immediately 'wanted' and attractive, by activating mesocorticolimbic circuitry for incentive salience ( Robinson and Berridge, 2013). In Pavlovian-Instrumental transfer (PIT; right), a Pavlovian reward CS+ cue (sound) elicits small peaks of 'wanting' during extinction test (bottom). An amphetamine microinjection that releases dopamine in nucleus accumbens selectively magnifies only the CS triggered peaks of cue-triggered 'wanting,' without raising baseline effort based on act-outcome expectations or the effect of an irrelevant CS- control sound ( Pecina and Berridge, 2013).

FIGURE 3. Motivational salience: positive 'wanting' versus negative 'fear.' The nucleus accumbens can generate either incentive salience or fearful salience. Microinjections of the same glutamate-blocking drug (DNQX) elicit opposite motivations at different sites. Both incentive motivation and fearful motivation require local mesolimbic dopamine. Many individual sites can be flipped back and forth between generating 'wanting' and 'fear' by changes in environmental ambience, as described in text. Based on Reynolds and Berridge ( 2008) and Richard and Berridge (2011).

Neurochemically, these hotspots respond with hedonic amplification to opioid, endocannabinoid, orexin and related neurotransmitter signals - but never dopamine. When stimulated with those neurochemical microinjections, the hotspots increase 'liking' reactions to double or triple normal levels. But no 'liking' enhancement occurs if the same drug microinjections are moved outside the boundaries of the hedonic hotspots, even still within the same structure (e.g., nucleus accumbens). Instead the microinjection at those sites outside the hotspot (or dopamine within the hotspot) still will stimulate only intense 'wanting' in the remainder of the structures, without enhanced 'liking' and sometimes even while suppressing 'liking.'

Psychological Features of Incentive Salience as 'Wanting'

Normally, mesolimbic 'wanting' and cognitive wanting often go together. Incentive salience 'wanting' gives a motivationally compelling quality to a cognitive desire, and helps motivate action to obtain the goal. Incentive salience is often thus a spur to action, reflecting the overlap between dopamine functions of motivation and of movement ( Salamone et al., 2015, 2018). Thus incentive salience may proactively facilitate action and engagement ( Ryan and Deci, 2000; Carver, 2009; Kruglanski et al., 2014). In people, also, strength of mesolimbic incentive salience has also been linked to personality traits of impulsiveness and sensation-seeking (e.g., Gray's BAS [behavioral activation system]) ( Gray et al., 1999; Beaver et al., 2008; Hickey and Peelen, 2015).

Incentive salience mechanisms look for potential targets and triggers in the world, either in physical stimuli or in imagination. Many reward in life are learned, so that their 'wanting' is activated by associated cues, but interaction with world targets can also be seen in people who suddenly have their mesolimbic dopamine systems artificially activated for the first time, such as by a newly-implanted brain stimulation electrode. For example, a woman whose subthalamic electrode (activating an input to dopamine systems) was turned on for the first time suddenly " was excessively talkative and it was not possible to interrupt her while she was speaking," and subsequently with further stimulation " was in love with two neurologists, and tried to embrace and kiss people" (p. 1383) (at least, in love according to the neurologists who published the report) ( Herzog et al., 2003). Similarly, a man in Germany whose nucleus accumbens electrode was first turned on, " spontaneously reported that he realized that he was in the (city of) Cologne, that he never visited the famous Cologne Cathedral, and he planned on doing this in the immediate future, which he indeed did the day following the operation" (p. 372) ( Schlaepfer et al., 2008). The flexibility of such suddenly enabled motivation in seeking targets overlaps with psychological concepts of general motivational engagement ( Higgins, 2006). However, with repeated activation, learning mechanisms tend to focus amplified incentive salience more narrowly on to a particular 'wanted' target, with addiction being an extreme example of a too-narrowly focused 'want.'

Theoretical Roots

Incentive salience follows psychological rules described originally by a useful incentive motivation framework posited successively by the psychologists Dalbir Bindra and Frederick Toates ( Bindra, 1978; Toates, 1986). Dalbir Bindra posited in 1970s that Pavlovian incentive cues or conditioned stimuli (CS), which predict pleasant reward as unconditioned stimuli (UCS), take on some of the motivation features of their associated UCS reward (food, drink, sex, drugs, etc.). These cue features include the ability of CSs to trigger motivated 'wanting' to seek and consume their reward, and to a degree also trigger hedonic 'liking.'

Frederick Toates suggested in the 1980s that reward cues additionally participate in a CS-state interaction, similar to a reward UCS-state interaction called alliesthesia. Alliesthesia refers to the phenomenon that food becomes more hedonically rewarding when we are in a state of hunger, water is more rewarding when thirsty, nasty seawater becomes a rewarding taste if we are sodium-deprived, warmth is more rewarding when we are cold, coolness more rewarding when we are hot, and so on ( Cabanac, 1971). Toates suggested that a learned cue or CS similarly interacted with an individual's current internal state to generate incentive motivation. That is, an incentive percept (food or associated cues) would generate intense motivation when perceived in a physiologically hungry state, but not so much when perceived in a satiated state.

This interaction can be very powerful, able even to reverse the affective valence of a learned cue. For example, is it possible to suddenly intensely 'want' something that you've learned is always disgusting? Yes, says incentive salience theory, because of this interaction.

Such reversals in the incentive salience of cues is a powerful confirmation that mesocorticolimbic incentive salience mechanisms obey Bindra-Toatesian rules. In ordinary life, shifts in cue-triggered 'wanting' may be only incremental within a single valence, merely varying between zero and high. However, the motivational power of cues is modulated by a host of physiological factors, ranging from appetites and satieties, to drug intoxication, stress and emotional arousal states, etc. Reward cues alone are relatively powerless at producing motivation without an appropriate brain state. It's the brain's reaction to a cue that matters for 'wanting' motivation via incentive salience ( Berridge, 2012; Dayan and Berridge, 2014).

Dopamine Sensitization of 'Wanting' in Addiction

Incentive salience synergy between a reward cue and a hyper-reactive mesocorticolimbic brain state can sometimes work against our interest. This is why it can be unwise to shop while hungry. And some 'wanting'-enhancing states of hyper-reactivity are more long lasting, such as dopamine-related neural sensitization. Addiction is the most extreme example, as neural sensitization makes brain dopamine systems more permanently hyper-responsive to drugs and drug cues, triggering strong urges to relapse and actually take drugs. Sensitized hyper-reactivity can be further amplified to even higher levels at certain moments by states of stress (whether unpleasant or celebrational), emotional excitement - or by taking a hit of drug again - creating a special window of heightened vulnerability to relapse. Sensitized 'wanting' in addicts creates a probabilistic form of cue-triggered compulsion, in the form of exaggerated temptation that can come by surprise, be amplified further by stress, and be hard to resist ( Berridge and Robinson, 2016).

Counter-Intuitive Sharing Between 'Wanting' and 'Fear'

Both the 'wanting' and the 'fear' generated by these brain manipulations require mesolimbic dopamine signals in the nucleus accumbens. Simultaneously blocking all dopamine receptors at the microinjection site is functionally equivalent to removing the glutamate-blocking drug contents from the microinjection: no intense motivations are generated at all ( Richard and Berridge, 2011). However, positive versus negative valence generated by glutamate antagonist microinjections depend on slightly different types of dopamine neuronal receptors. Positive 'wanting' required only D1 types of receptor activation by dopamine, whereas negative 'fear' required activation of both D1 and D2 types, which implicates additional neuronal circuitry ( Richard and Berridge, 2011).

In people, this 'fearful salience' may be responsible for drug-induced states of paranoia in psychostimulant users, such as when euphoria flips to paranoid fear or aggression after high doses of methamphetamine or cocaine. In schizophrenic patients, a similar negatively-valenced role of dopamine in nucleus accumbens has been suggested to produce motivational paranoia that can accompany delusions of persecution ( Howes and Kapur, 2009). This hypothesis is based on evidence of mesolimbic dopamine hyperactivity in schizophrenia, and also on the ability of anti-dopamine drugs that block D2 receptors to treat schizophrenic symptoms. By this 'fearful salience' hypothesis, medication-induced dopamine blockade may quickly reduce the motivational salience or compelling quality of perceptions and delusions, thus reducing the paranoia and providing a psychological distancing that more gradually leads to dissipation of cognitive delusions (although dopamine dysfunction is probably not the mechanism of the delusions themselves) ( Howes and Kapur, 2009).

Adding 'Disgust' to 'Fear' in Nucleus Accumbens

Disgust seems quite a different psychological emotion from fear ( Rozin et al., 2008; Chapman and Anderson, 2012). In animals, both 'disgust' and 'fear' reactions could be called negatively-valenced defensive reactions by LeDoux's recent terminology based on evolved functions ( LeDoux J.E., 2015). But the functional 'defensive' label does not capture their psychological and neural distinctiveness. This points again to the need for emotion-based labels for these core psychological processes,even in animals, such as 'fear' and 'disgust'. 'Fear' defends against external bodily danger threats, while 'disgust' defends against oral and internal toxic threats. However, neural mechanisms that generate 'disgust' and 'fear' reactions are mostly distinct, though they do partly overlap in the posterior shell of nucleus accumbens ( Faure et al., 2010; Maren et al., 2013; Ho and Berridge, 2014; Perusini and Fanselow, 2015; Paré and Quirk, 2017).

We've studied core 'disgust' reactions such as gapes and headshakes that are naturally elicited by bitter tastes in rats ( Steiner et al., 2001). In humans, disgust and fear expressions have nearly opposite statistical patterns of facial configuration, giving rise to the suggestion that these facial expressions may oppositely modulate sensory feedback: fear expressions may enhance relevant sensory perception (e.g., eye widening), whereas disgust may suppress relevant sensory perception ( Susskind et al., 2008). Original sensory taste disgust may have been extended in human evolution to include also conceptual forms of disgust, such as in perceived physical contaminations (e.g., by germs, mutilation or putrefaction), or moral contaminations (e.g., particular acts of violence or sex or prejudice) ( Rozin et al., 2008; Chapman et al., 2009). Human neuroimaging studies of disgust have often implicated the insula in cortex and subcortical striatal circuitry (which can include the nucleus accumbens) ( Calder et al., 2007; Chapman and Anderson, 2012). Our studies in rats have further revealed that the only brain site where small lesions are able to cause sweetness to become perceived as nasty, and to elicit excessive 'disgust' reactions, is the hedonic hotspot of ventral pallidum, a striatal target which receives dense nucleus accumbens projections ( Ho and Berridge, 2014). In short, despite both being defensive and negatively valenced fear and disgust are quite different emotional reactions.

Yet our animal brain-manipulation studies show one interesting overlap in neural generators for intense 'disgust' and 'fear' reactions in nucleus accumbens, involving a shift in neural mode of function at the same brain location. The posterior portion of its medial shell has two 'fear' generating neurochemical modes (glutamate AMPA receptor blockade versus GABA receptor activation), and only the GABA mode also generates excessive 'disgust'. In this GABA-related mode, the normally 'liked' taste of sucrose instead elicits 'disgust' gapes and headshakes from rats, as though the taste were bitter quinine. This 'disgust-plus-fear' mode is produced when the posterior shell zone receives microinjections of a drug that activates GABA receptors, such as muscimol ( Faure et al., 2010; Ho and Berridge, 2014). The GABA drug probably produces a stronger electrophysiological inhibition of shell neurons than the glutamate blocking drug (e.g., DNQX), which produces only a mode of pure coping 'fear.' Thus conceivably this incremental change in neural inhibitory signal produces a qualitative change in psychological emotion that adds a categorically new 'disgust' reaction to 'fear' reactions. Because nucleus accumbens neurons normally inhibit their targets (by themselves releasing GABA as neurotransmitter), this stronger inhibition could release downstream neurons in ventral pallidum and other targets into higher excitation levels, to produce the additional emotional reaction. Of course, an incremental difference at one overlapping stage of a neural circuit can still be translated into qualitative neural and psychological differences at other stages to recruit different anatomical patterns of activity. But exactly how that is accomplished remains to be understood.

Would similar neuronal manipulations of human nucleus accumbens neurons induce subjective fear versus disgust motivations in people too? We don't yet know, though these are clear predictions from this schematic understanding of 'fear' and 'disgust' mechanisms. Someday, less invasive techniques of brain modulation, or more sensitive techniques of brain measurement, may allow such predictions to be tested in humans. People may turn out to have desire-dread keyboards in nucleus accumbens similar to those already found in rats that produce core 'wanting' and 'liking' versus 'fear' and 'disgust.' If so, such findings would indicate another continuity between human and animal mechanisms of emotion. Just as when the original dopamine-based discovery that 'wanting' versus 'liking' mechanisms were different in rats pointed the way to teasing apart subjective wanting and liking feelings in humans, conclusions first gained from objective affective reactions in animals can sometimes give us new and important insights into human emotions. In such cases, animal findings at the very least tell investigators what to look for in people, shaping the questions to be asked so that the right answers are found.

Conclusion

Emotional and motivational processes occur in people most notably as subjective feelings, but also have underlying core psychological processes with objective features. These core affective processes can be schematically understood, so that their psychological features become objectively identified, and allowing them to be mapped to underlying brain mechanisms. In many cases, conclusions can transfer from affective neuroscience studies based on animals to human emotions and motivations. Admittedly, not all conclusions will transfer. But those that do often carry important implications for understanding human clinical affective disorders. Best progress in building an integrated psychology and affective neuroscience of emotion and motivation requires combining human and animal approaches together, and consideration of both subjective and objective features of affective reactions.

Author Contributions

The author confirms being the sole contributor of this work, wrote the article and approved it for publication.

Funding

Research in my lab has been supported by NIDA and NIMH grants (DA015188 and MH63649).

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This manuscript was originally accepted by an American Psychological Association journal for a special issue. But it was withdrawn from that journal when APA issued legal demands to United States universities in June 2017, insisting that faculty authors of articles in APA journals remove reprints of their own articles from university websites. Consequently, submission of the manuscript was switched to this open access journal. I thank Tory Higgins and Arie Kruglanski, who edited its previous incarnation, Morten Overgaard, who edited the current version, anonymous reviewers of the former journal and RS and RT, for helpful suggestions on earlier versions of this manuscript, and Hannah Baumgartner, Ileana Morales, Erin Naffziger, and Jeffrey Olney for proofreading a final version. All procedures used in my laboratory were approved by the Institutional Animal Use and Care Committee of the University of Michigan.

References

# data[].attributes.attachment_link.keywords[]
reward                addiction                   emotion                        fear 	                         disgust history of psychology             psychological science             psychological method

# data[].attributes.attachment_link.title
Evolving Concepts of Emotion and Motivation

# data[].attributes.attachment_link.description
This review takes a historical perspective on concepts in the psychology of motivation and emotion, and surveys recent developments, debates and applications. Old debates over emotion have recently risen again. For example, are emotions necessarily subjective feelings? Do animals have emotions?

# data[].attributes.attachment_link.title
Why Educational Neuroscience Needs Educational and School Psychology to Effectively Translate Neuroscience to Educational Practice

# data[].attributes.attachment_link.description
The emerging discipline of educational neuroscience stands at a crossroads between those who see great promise in integrating neuroscience and education and those who see the disciplinary divide as insurmountable. However, such tension is at least partly…