What do neuroscientists think of computational neural networks and deep learning?

As a student of neuroscience, I have never heard of computational neural networks. I guess you mean artificial neural networks. I can just speak for myself.

Many theorists believe there is a link between ANN, DL and brains, but to most neuroscientists only believe what evidence tell them. There is not much proof showing ANN and deep learning are how brains work.

The connection between ANNs and brains are nearly non-exsitent. The fact is that ANNs and DL only looks like real neural network from a layman’s perspective. There’s nothing they are alike when you scrutinize the details. For example, the ‘cell’ in ANNs stores values. However, in real cells there are just electrical potentials and the information are full of noises. Besides, the connections in our brains are quite random, and full of recurrent loops in different levels, while in DL the network structures are feed-forward and fixed. In another words, it seems the more successful the ANNs, the less it resembles real neural networks. Even many vision scientists use probabilistic models for visual cortex, it is not explained how a global cost-function minimization, back propagation are achieved neurologically. For deep learning such as deep belief networks, the training is not on-line but off-line (batch training, 2 phase training), which is also very different from our brains.

On the other hand, neuroscientists have fairly clear idea of how grid cells and place cells in hippocampus work to encode spacial memory based on experiments, and the machenism is very different from ANNs. Other successful models for cerebellum are very different from ANNs, either. If ANNs and DL are generic models of brains, then the mechanisms should not be so different.

Information theory, a misleading name.

Today information theory has been widely used in artificial intelligence, machine learning, and being suggested that information theory is how our brain acquire information.

For example,  they define -log(P(x)) be the the amount of information in a pool of data, where P(x)
is the possibility event x occurs. The formula says when a event happens  frequently, there is no information; when a rare event happens, it contends a lot of information.

It appears to be reasonable at the first glance. However, it is totally irrelevant to the definition we know of information. What we think about information is something “meaningful”, not something “rare.”

Meaning has two meanings, the first one is inference of a word, concept, or action. The second one is significance, or importance of something.  What we mean information is something meaningful, something gives inference, not something important.

 We see sun rises from east everyday, but it is as informative as sun rise from west, which rarely, or never happens. They are equally informative because the former means the Earth spins from west to east, and the latter mean the opposite. Besides, not all new things are information, only those making senses are. You don't consider a misspelled word ‘demacrosy’ more informative than the word ‘democracy’ although the latter is more common to see.

Therefore, it will be more appropriate to call information theory “news theory” to avoid misleading
association.  Something happens rarely would be on the news, while something happens everyday is not a news because unexpected information are apparently more valuable a information than those expected to happen. So, it is safe to say information theory can be a measure of the surprise value of information instead of the amount of information itself.