Are LLMs Creative?
Nivi: I just started this Twitter space to talk about why LLMs are and are not creative. Mostly it’s going to be about why they’re not creative. But I’ll also be providing some defenses of LLMs.
And if I say anything interesting here you can assume that I stole it from David Deutsch. Also, a lot of this I’ve learned from Brett Hall as well. So let’s jump into it.
LLM outputs are implicit in the weights
A lot of people think that LLMs are creative, and I’m going to start by explaining why they’re not, and there’s really two good reasons why they’re not creative.
One of them is pretty straightforward, and the other one’s a little more complicated. Let’s start with the simple one.
First, for any given prompt, the LLM’s response was already determined during training. So it’s not creating anything new when it makes its output. The output that you’re getting is implicit in the program and in the weights of the neural net. So the same prompt and seed and temperature and whatever other pseudo-random inputs that they’re going to put in there are going to give you the same output every time.
So every single possible output was already created during training, and it’s already reflected in the weights. You can think of the neural net as a giant and also compressed database of outputs that you can look up with the right prompt, or you can think of it as a paintbrush that is made useful by the creativity of your input.
So the only creative aspects of the LLM are in the training data that was made by a human, the program that was also made by a human, the human feedback, which was also done by humans, and the prompt, which was also made by you, a human. So all of these things are creative acts, but all the creativity is coming from the human interaction with the program.
I don’t consider a program that looks things up in a giant database to be a creative program. The people that made it were creative, not the program. So that’s the first reason that LLMs are not creative: Everything was determined during training.
All programs are inherently limited
So there’s a second deeper reason why there is no program or programming language that can be as creative as a human. You can’t make a programming language that can capture all human creativity because humans can always take that language and make a different language or a bigger language that can do things that the first language can’t do.
So humans can go in and redefine all the words in the language. They can invent new words for ideas that were previously inexplicit, meaning things that people don’t know how to put into words yet, like how to ride a bike. They can take the existing words of the language and put them in new orders that don’t follow the rules of the language but are still useful and meaningful. Humans can make new kinds of media like music, dancing, drawing or they can extend the language by adding undecidable propositions as in Gödel’s Incompleteness Theorem.
But the bottom line is that programming languages give the same result every single time, their outputs are bounded while human creativity is unbounded and limitless.
So if you want to argue the other side of it, you could say: Okay, maybe the program could modify itself like a human would do and extend the formal system or programming language that it’s in. But if the program that is doing the modifications is also deterministic, which it is going to be, because that’s all we know how to write, all the extensions of the language that you’re trying to modify are also going to be deterministic. By the way, when I say deterministic, just to define it, it’s a process where the outcome is already determined by an earlier state of the process.
There is an AGI program
There is a problem with this argument that I’m making about the limit of programming languages, though. The problem is that there is actually a normal computer program, a deterministic computer program that has the same output every time that can simulate a person.
And this is coming from the Church-Turing Principle and from the assumption that quantum mechanics is correct or close enough. And what the Church-Turing Principle says is that any quantum mechanical system can be simulated to arbitrary accuracy with a universal computer. I think the Church-Turing thesis is actually stronger than that. It says that any physical laws can be simulated to arbitrary accuracy with a universal computer. Someone should double-check on that.
But if you can simulate a human, or a human brain, then you have an AGI that can simulate human creativity. AGI: Artificial General Intelligence. So we just made the argument that there actually is a computer program that is as creative as a human because we can simulate a human, because we can simulate anything that follows the laws of quantum mechanics.
There are some problems with this argument though. First, when people say that something can be done in principle, and this is an example of an argument where the argument is in principle, not that we actually know how to build the computer program that can simulate a human, but we can do it in principle. What we mean is that it follows logically from our best explanations of the phenomenon, but it also means that we don’t know how to do it. And there may be reasons why we can’t do it.
So the computational resources required to do the simulation would be larger than the size of the universe, for example. I’m not saying that’s the case here, but it’s something to consider whenever people say something can be done in principle.
You would also have to have a way to find the state of every relevant aspect of every single particle in the system to enough accuracy to get the result that you want with enough accuracy. So basically you’d have to figure out a way to scan a human brain, find out what every single particle is doing, and then load that into the computer program. That may not even be possible to do in principle.
Then you have the issue that this computer program is going to give you the same answer every time if it’s loaded with the same initial conditions, but somehow that’s supposed to simulate human creativity. And the way we think of human creativity is that people can change their minds. So if you ask somebody to do something, they might give you a response and then come back the next day and say they changed their mind, or they might decide that they want to go play tennis instead of dealing with your problem.
So those are the issues with the idea that a deterministic computer program that is simulating a human can be done. I don’t think anybody has a good way to resolve the conflict between these two ideas that, on the one hand, a programming language could never encompass the full range of human creativity, but, on the other hand, there apparently is a program that can encompass the full range of human creativity. If you figure out a way to solve this problem, your name will be in the history books along Turing, David Deutsch, and Einstein, and Darwin.
LLMs are not ‘just math’
I do want to speak briefly in defense of LLMs.
I see people on two extremes. On the one hand, they’re like: Oh, all AI is just math, so there’s nothing special about it. On the other hand, some people imagine that there’s some unknown intelligence going on inside there. I think both of these extreme and common viewpoints are wrong.
One: saying that AI is just math is like saying everything is just physics. It’s not a useful explanation for anything. And then imagining there’s some unknown intelligence in there is also misguided because we designed it. We understand the design, we know how it works and we can make another one if we want to. The interesting question to me is if there are other explanations for what’s going on inside the neural net. And I suspect there are, and there’s probably a lot to learn from the structure of the neural net.
LLMs are more than just the training data
I also think it’s a mistake to say that LLMs can’t come up with new ideas or new theories. They just can’t do it in a creative way. So even a calculator can come up with an idea or a number that no one has ever seen before, if you press the buttons on the keypad correctly.
I think it’s also a mistake to say that everything is in the training data of the LLM because the program, which is a creative expression of the author, the prompt is a creative expression of the author and the human feedback, which are also creative expressions, can all transform the training data in novel ways. So there’s more to it than just the training data because the program, the prompt, and the feedback that was given to it during training all transformed the data in creative ways.
Aside from that creative transformation, everything that the LLM does is not creative. Even when it’s coming up with a new idea, it is doing it in a non-creative way.
Can LLMs be as creative as DNA?
I think there’s one way you could have a creative LLM, at least in the sense that evolution is creative. If you connect the LLM to a true random number generator and have that LLM spit out ideas and then use something like the environment or a human to evaluate those ideas, you do have a creative process because that is how evolution works, and evolution is considered to be creative.
In evolution, you have random mutations in the DNA and the environment determines if that mutation was useful and that is a process that can create and build things like the wings of a bird, which was nowhere implicit in the Big Bang.
So evolution relies on randomness, but it doesn’t reduce to randomness because the environment determines what constitutes knowledge. So it’s the combination of the random change in the mutations of the DNA combined with the environment providing a, I guess, a utility function that can create things that were not implicit in the Big Bang, like your eyeball or the wings of a bird.
So, in this case, connecting an LLM to a true random number generator, which is guided by a human or an environment that can determine whether those outputs constitute knowledge, that LLM would then be engaged in a creative process.
This is just a sketch of an idea and I’m not totally sure it makes sense. So let me criticize it a bit. First, is this any different than just generating random numbers and trying to determine whether those constitute knowledge.
I think it might be because a random LLM output is more likely to be useful than a random number. Essentially, guiding the random output through the weights might reduce the cycle time to finding a random sequence that can replicate or maintain itself in the environment.
Next, if a human is evaluating whether the LLM output constitutes knowledge, is the human the one that is conjecturing the knowledge. My guess is: yes. So if you need a human in the loop, there’s probably no input to the creative process coming from the LLM. But if the environment and the LLM output together are determining that the output of the LLM constitutes knowledge, that might work.
In any case, I’m going to think about this more.
So those are my defenses of LLMs. One, that it’s not just quote unquote math. Two, it can come up with ideas in non-creative ways. Three, it’s not just the training data. And then four, connecting it with a true random number generator and an environment that can evaluate its outputs can create.
Q&A
Okay. That’s all I have prepared. Anybody have any questions?
Question: Hey. Hey. I also was thinking like something similar maybe a couple of weeks ago, like it’s in my pinned post, but I’m curious why you think it’s important to think of LLMS as creative.
Nivi: I think they’re not creative. I think people think they’re creative and I think I have, demonstrated why they’re not, mostly using the ideas from David Deutsch.
But I do think it’s wrong to say that they are just the training data, which I hear people saying all the time, they’re more than just the training data because the training data has been transformed through creative processes. And those are the program, the human feedback and your prompt.
Those are all creative inputs that transform the training data in a way that produces something that is more than just the training data.
Okay. Thank you everybody for joining. I’m going to do more of these. I recorded this and I will post it later. Cheers to everybody.