Note: A general familiarity with multiple machine learning methods is assumed here. Understanding on a broad level how neural networks¹, decision trees², and linear regression³ work should be sufficient to understand the content.

When explaining machine learning methods, analogies to the human mind can prompt intuition for what part of human cognition the learning method is re-creating. On the broadest level these analogies are words like “learning,” and “training,” which describe processes general to arguably all machine learning methods ⁴. More thoroughly defined sub-fields of machine learning are described with increasingly specific analogies. Supervised and unsupervised learning, for example, describe different methods of pedagogy shared between humans and two types of machine learning algorithms. This process of iterative specification continues to the point of specifying an individual algorithm’s form and function.

Certain neural network structures are described with these metaphors, which helps people learning about them determine those structures’ use cases. In the space of recurrent neural network structures, processes like “attention,” (transformers) and “short-term memory,” (LSTMs) provide insight into the intended and expected features of those networks. Generative Adversarial Networks are another type of neural network, named as such because they train by pitting two antagonistic networks against each other. Having an idea of what a network is doing on a high level— understanding what about them is similar to the human epistemic process—makes it much easier to judge their efficacy and understand their limitations.

While analogies between specific neural network structures and the human mind exist, on of scope of neural network architecture as a whole, comparisons are usually to the mind on a materialistic level, e.g. computational neurons are compared to human neurons (though that isn’t very accurate ⁵). It’s rare to see explanations of what thought processes deep neural networks imitate for what seems like a good reason: the lack of pre-defined structure in the sub-field of neural networks appears to allow different types of them to train towards any solution necessary. What would it mean to describe an architecture that is so flexible and universal? While this blank-slate argument appears to end the discussion on neural network-human mind analogues, it can serve to obscure the subjective reality of neural networks.

Grant Sanderson, in his video “But what is a neural network?”, described how one might expect the first layer of a neural network training on the MNIST dataset to identify lines and dots, then the second to identify intersections and primitive shapes from the prior layer’s output, and the third to compose shapes into more complicated structures, but, when the network finishes training, in actuality the layers’ contents are a mess. This is true of almost all neural network architectures which don’t heavily rely on very specific directions from the programmer—the internal contents of neural networks are unintelligible by consequence of their inherent structure. As neural networks update their internal weights and biases, there is no value associated with intelligibility, which demonstrably results in their disorganization.

One explanation for the lack of subjective cognitive analogies here is a distinct difference between neural networks and other types of machine learning. Most other types of machine learning model a task we can perform consciously, or at least on command. Clustering 2D and 3D data is a stunningly easy task for humans. Polynomial regression is a conscious process we learn to perform in algebra and statistics. Creating decision trees is a task for storing information so common there are jokes about their overuse (https://xkcd.com/518/). Iterative backpropagation on the other hand… isn’t so easy. Unlike other methods, there isn’t a conscious way to invoke the human analogue to deep neural networks, even though they’re based on our neurons…

When I look around my room, I see my shoes, a computer, some pillows, a desk. This I understand effortlessly, and comes without the need to reason through the steps from raw visual data to a conceptual object. I can attempt to reproduce my thought process, focusing on the parts of the shoe that my eye used to assess its shoe-ness, but I have no way to verify that analysis’s accuracy; it’s purely speculation. The outputs of our visual processing are intuitively available to the seer. With effort, the seer can become aware of the raw inputs to their visual process, but the steps in between input and recognition are consistently a mystery. This mirrors the conceptual structure of neural networks surprisingly well, with messy inputs, a clear output, and an inscrutable process between the two.

Logical analogue for deep classification NNs This analysis can be extended to logical reasoning. Neural networks that map from large feature sets to a set of outputs have the same pattern of clarity as the visual neural networks described above, but serve the function of categorizing the data into relevant concepts. When we perform patterns of thought frequently enough, our brains create heuristics to make those patterns faster. Walk outside just before a rainstorm, and notice how you can tell it’s going to rain even before you choose to evaluate why you know that. Watch your dog start to salivate at the sound of the dinner bell. These black-box associative processes are very powerful, and incredibly fast once trained⁶.

With the power of associative learning methods in mind, it’s important to understand their limitations. Because they have no structural intermediates, they can be subject to invisible bias, where certain combinations of inputs which were biased in the sample set are transferred to the final model without anyone knowing. They also don’t have the inherent structural capability to simplify and convey their conclusions to a human or other machine learning structure. One method of human reasoning is the establishment of facts upon evidence and higher order facts upon other facts. Neural networks, in having fixed complexity⁷ and no way to fix a useful set of neurons (analogous to a lemma here) in place, are not structurally designed to perform this task, and so will only arrive at reusable logical intermediates to their conclusions on chance. Unconscious learning models certainly have a place in artificial intelligence, but are structurally inadequate to model conscious thought processes. Some problems are simply not suited for neural networks.