Neuromodulation

Twenty-five Lessons from Computational Neuromodulation

Peter Dayan

Neural processing of brain is constrained by three problems. A) Computation is is radically distributed, yet amount of point-to-point connections are limited. (B) Bulk of these connections are structurally uniform, lacking target specific physical differences, (C) The brain's structure is relatively fixed, unable to explain the multifarious modalities under different conditions of neural ensemble.

Neuromulators address these problems by their multifarious and broad distribution, By enjoying and specialized receptor types in partially specific enough hey anatomical arrangements, and by their ability to mode activity and sensitivity of neurons and the strength of the plasticity of their synapses. 

As we should see later, architectural and neuromodulatory specifications are frequently integrated. We address Pavlovian (habitual) and model-based learning by reviewing acetycholine (Ach), dopamine (DA), norepinephrine (NE), and serotonin (5-HT). Phasic (transient) dopamine signal resulted from predictive cues provides a formal underpinning for the theory of incentive salience, which is concerned with motivational influences over the attention garnered by such stimuli. We also have noted substructure in the dopamine systems such as the mesocortical dopamine neurons that are excited rather than inhibited by punishment.

Goal-directed control, working memroy

Pavlovian mechanisms that lead to approach or withdrawal to external appetitive and aversive outcomes an predictors might inluence the way theenumeration works. States associated with reward could be more likely to be enomerated under the influence of dopamine and serotonin. Control over working memory can bave both instrumental and Pavlovian components. Basal ganglia could acquire policies that control the gating of information nito working memory using reinforcement learning. 

Uncertainty

The representation, updating and use of uncertainty have become major foci of computational treatments of neural information processing.

Learning

Once case of the link between ignorance and learning arises in the context of auto-associative memory models of the hippocampus. If an input is assessed to be novel, then it is stored; if the input is familiar, the recall process should remove noise from it and/or recall relevant context or associated information. The ability of neuromodulators to control the course of activity by regulating  pathways determines the activity of neurons is a common scheme.  ACh helps regulate oscillations that simultaneously affect multiple sub-regions of the hippocampal formation. It has been suggested that different pathways between theses regions are dominant at different phases of theta.

Model-based predictions and plans are dependent on learning about the structure of the environment and circumstances and outcome contingencies. This should be regulated by predictive uncertainty.

There is structure in a loop connecting cholinergic neuclei to sensory processing and prefrontal cortices.

There's evidence that tonic activity or levels of norepinephrine do indeed increase with predictable reversals in a simple reaction time task, and boosting no norepinephrine (NE) can speed the course of reversal learning. Reversals we shot popular way of inducing change, are normally signaled when actions and choices that used to be rewarded become unproductive or less productive; and actions that were formerly punished or nugatory become worthwhile. One might expect dopamine and serotonin to be involved directly in assessment and realization of reversals. Rapid change is normally the feature of a model based or goal-directed system. With NE, the projection of serotonin and dopamine to the striatum and prefrontal regions having implicated in form of behavioral flexibility such as reversal learning and set shifting.  Mechanism by which bonuses arise in model-based calculations is unknown. 

NE has long been linked with anxiety. NE helps to organize response in stress notably in conjunction with cortisol, a steroid hormone that acts as another neuromodulator. This involves everything from changing storage and usage via glucocorticoids to change the actual style of information processing.

Inference

Selective attention the impact of uncertainty should be governed by the utility associates with what can be discovered; one might expect that much of the inferential certainteed should be highly specific to circumstances of the task. There's evidence for involvement of both ACh and NE in controlling critical aspects of inference at both timescales mentioned above. Cholinergic neuromodulation ass also been implicated in aiding signal detection.

 For normal animals, ACh was substantially released over a short timescale on trials on which animals successfully detected the cue. At fine, sub-second timescale presentation of the cues the phasic release of ACh is related to expectation of the changing circumstances associated with the upcoming reward.

Discussion

For the first, we have seen common motifs such as heterogeneity in space (i.e. different receptor types with different affinities some localize on different systems) and heterogeneity in time ( with phasic and various timescales of tonic release) there's number forms of controls including self-regulation by autoreceptors, complex forms of inter-neuromodulator interaction. Also, loops between prefrontal areas and neuromodulator nuclei which exert mutual influence uptown each other. These, and indeed other functions of the neuromodulators, maybe complicated by corelease of other neurotransmitters and other neuromodulators through the same axons. 

Given the focus on decision-making, the key neuromodulators were dopamine, serotonin, acetylcholine, and norepinephrine, which you presented information about reward punishments and expected and unexpected uncertainty.

Concerning neural processing, the two most important systematic effects are controlling plasticity, and controlling whole pathways. Acetylcholine's influence on thalamocortical versus intracortical interactions. They are so dynamic effects, searches change the gain of competitive, decision-making circuits.

For future, measuring their local concentrations at target zones and selectively manipulate their activity or that of particular receptor types. 

BOLD signal my not would be very effective in helping understanding neuromodulation because the neuromodulators might be able to affect local blood flow directly themselves. There is much work to do to  understand overall networks and systems affects of changes that we know different neuromodulators lead to individual elements in those circuits. This may help understand the aspect of various sorts of heterogeneity. - e.g. What is achieved by the subtle differences within families of receptors and also the rich intertwining of neuromodulators. Model free and model based systems, with the former substantially based on neuromodulators such as dopamine and serotonin, whereas the latter depends on cortical processing.

My thoughts

The effects of neuromodulation is almost everywhere on every function of brains. They are the main mediator of signal processing across multiple spatial and temporal scales. In the review, the paper addressed four principal neuromodulators that implicated on model free, Pavlovian control, model-based, goal-directed control, uncertainty, inference, working memory, and learning. From signal transmission at synapses, to looping between striatum and cortex, their intertwining corelease and heterogeneous targets and receptors allows complex functions of brain be possible with such a limited, and fixed dendritic structures. This paper explained how the three limitations mentioned in the abstract are solved by neuromodulations.  

What do neuroscientists think of computational neural networks and deep learning?

As a student of neuroscience, I have never heard of computational neural networks. I guess you mean artificial neural networks. I can just speak for myself.

Many theorists believe there is a link between ANN, DL and brains, but to most neuroscientists only believe what evidence tell them. There is not much proof showing ANN and deep learning are how brains work.

The connection between ANNs and brains are nearly non-exsitent. The fact is that ANNs and DL only looks like real neural network from a layman’s perspective. There’s nothing they are alike when you scrutinize the details. For example, the ‘cell’ in ANNs stores values. However, in real cells there are just electrical potentials and the information are full of noises. Besides, the connections in our brains are quite random, and full of recurrent loops in different levels, while in DL the network structures are feed-forward and fixed. In another words, it seems the more successful the ANNs, the less it resembles real neural networks. Even many vision scientists use probabilistic models for visual cortex, it is not explained how a global cost-function minimization, back propagation are achieved neurologically. For deep learning such as deep belief networks, the training is not on-line but off-line (batch training, 2 phase training), which is also very different from our brains.

On the other hand, neuroscientists have fairly clear idea of how grid cells and place cells in hippocampus work to encode spacial memory based on experiments, and the machenism is very different from ANNs. Other successful models for cerebellum are very different from ANNs, either. If ANNs and DL are generic models of brains, then the mechanisms should not be so different.