The language model! Is it alive!?! Is it conscious?!

Debate rages throughout the internet. Let us take a step back and present the best arguments of both sides.

The closest thing to an LLM in the biological world is the following. We take your brain. We cut off the visual cortex and all other sensory inputs (who needs that). We cut off the cerebellum which connects to the body(it was dying slowly anyway).

We further isolate the language center of your brain. But stop. This isolated language center does not reflect an LLM. The training was very different.

True, your language center almost certainly used ‘predict the next word’ to help learn. Many theories of the brain [1] believe that predicting the next sequence of input is a major method of learning. Butyour language center also used many other methods. Your language center training involved semantic grounding in bodily experience and the integration of many multi-modal inputs. It also involved an in-utero development process.

So to better match an LLM, we untrain your language center. Then we multiply it’s size by like a thousand. Then we train it again.

But this time it never receives input based on the visual cortex, and it never receives feedback from the cerebellum. It only getsweird signals with no context and is rewarded when it predicts those weird signals with no context.

But the weird signals have patterns between them. Patterns that can be predicted. So, we give this monstrosity a ton of data. And Imean a TON. Like a bigillion examples, more training data than your little brain could ever normally comprehend.

And this language center... over time... well it starts making coherent sense. It starts speaking eloquently even.

Predicting the language manifold.

The thing is LLMs are massive, and they get absurd amounts of training data.

Argument 1: “Look it never had any real-world data to contextualize the words it uses. It never had a visual cortex that allowed for a visual understanding of the words it uses”

Reasonable counter argument: “Yes, but it contextualized the words it uses with other situations where those words are being used. The sheer amount of training data means that it may have had more contextual clues then you ever had... besides these language models are now multi-modal, they do have visual contextual clues”

Argument 2: “Ok but it is still just taking in input and predicting the next sequence of words.”

Reasonable counter argument: “Taking in input and selectively creating output is the whole point of intelligence. Using prediction as a training method (something we do also) is irrelevant. As long as the output is intelligently chosen given the input situation then it is performing the main function of intelligence.”

IMO predicting the language manifold is a uniquely beautiful reward function. Every input situation has a unique output response that best meets the goal. Every input and output has concentrated meaning. IMO Knowing that meaning is kind of required to predict all the weird situational edge cases. The utter fluency current LLMs have when discussing weird edge case scenarios makes me believe they understand what they speak. But hey certainty in this field is stupid, so i may be wrong.

Ok... what about consciousness

Recently Claude 3 has been released and wow oh wow, it is saying some crazy stuff. It certainly can wax lyrical about being conscious. Check out this reddit post if you have not already.

You lot noticing this with Claude 3 as well? WTF? : singularity (reddit.com)

The question of consciousness is a tough one because so many people have their own bloody interpretations. But let's stick with the somewhat grounded idea that consciousness requires an updating self-state model. Where the network is considering itself and its own internal state as it processes inputs.

Now the thing about LLMs is that they are feedforward. They do not have any recursive loops. They do not experience reality as a sequential set of inputs where each input is related to the next.

Ahh! you say, but they do! I give them an input and then they respond, then I respond to that. Each conversation is a sequential set of inputs where each input is related to the next.

Ok fair, but:

1. its mind is wiped each time you change conversation.

1. Previous questions and responses are tacked on the input window. It does not have an internal loop. Each input it ‘sees’ is a brand-new situation. It's just that the input is getting longer and longer as your conversation continues. (I believe this is true, unless these companies have drastically changed their architecture design without telling anyone.)

So the argument goes. Consciousness is the experience of a changing self-state. The self-state can only change over time. These feedforward LLM’s have no experience of changing time, therefore… no consciousness within LLMs.

Without a recursive loop that can propagate “self-state” many people immediately say: no consciousness. Impossible. Nada, not happening.

God, I hate Yann LeCun’s certainty. It just grinds my gears. Certainty in this field? Especially when he has been wrong so many times... but anyway i digress.

These people have a point. As much as i hate agreeing with Yann, I do agree. Without a recursive loop propagating a self-state through a sequence,it is unlikely that LLMs are conscious.

However...Can we at least consider the counter argument? Because it has more legs to stand on then you would immediately think.

What if this is what is happening...

Here instead of propagating through time, we are propagating through framings. Where first the input is framed and placed within a certain context a. Then the input is framed and placed within a certain context b. On and on. Anyone who knows the transformer architecture can immediately recognize the situation. Attention modules are built to “frame” the input in a certain way. To pay attention to this and not to this. As different attention modules frame the input in different ways there is also a skip connection (or residual connection) which bypasses the attention modules. This skip connection connects the results of each attention model. One after the other, it connects them within a linear sequence.

One could then imagine that a self-state could be harbored within the residual connection. A self-state that is changing as the network propagates through the various framings of the input. Given how large LLMs have gotten... this may even be a large sequence.

This would suggest experience a sequence of change viaframings, instead of experiencing a sequence of change via time. This changing of framings would then be a brief sequence that inevitably ends with the end of the architecture.

But let’s be clear I do not really think this is the case. I would argue that without the ability to recursively loop and accumulate persistent representations of previous states over time, it is difficult for a coherent, integrated self-model to emerge from this process. The self-state would be too transient and ephemeral.

In Summary, while I lean towards skepticism that current LLM architectures can support genuine self-awareness or consciousness as we typically conceptualize it, I don't want to be overly dismissive.

The lack of direct sensory grounding, persistent internal state, and recursive self-modeling make it difficult to see how a coherent, unified self-model could emerge. However, these systems are displaying strikingly abstract reasoning capabilities that are surprising and impressive.Maybe… we will get surprised again.

This is an article on midflip feel free to edit as you wish. We vote on any changes weekly.

[1] On Intelligence by Jeff Hawkins