The sciences over the last century have endured three revolutions. The 20th century began with an empiricist rejection of idealism in favor of strict behaviorism that excluded the inner world of the mind, meanings and thoughts from scientific investigation. Then extending Alan Turing's development of computational machines for cryptography, Chomsky's 1956 Syntactic Structures showed that language behavior couldn't be learned, produced or understood without an innate mental structure -- a computational structure -- burying the behaviorist program and returning the mind as a subject of scientific investigation. Forty years later, Rumelhart, McClelland and Hinton introduced parallel processing, a prototype of learning that grew out of Claude Shannon's information theory, a model of behavioral learning. This was the beginning of neural networks which have given us chatbots. So in the 2020's, we've come full circle: LLMs have learned language behaviorally, implying that human understanding too might be all mere behaviors. AI is the zombie revival of behaviorism.
While the public worries about whether AI is intelligent, the implication of AI's behavioral learning for human intelligence and learning seems to have been ignored. The implication is that we too are zombie learners, negotiating information with no more access to the world of phenomena than an AI, the only difference being that AIs learn from read-only texts while we poke around with things-as-information and can edit the things with our hands and tools. It's all just information, with different sources and means. If you dismiss LLMs as having no immediate access to the world of things, look in the mirror.
This is the story of science, understanding and explanation over the last century:
The long history of the sciences in the 20th century begins with an attempt among scientists and philosophers of science to define the difference between a scientific theory and non scientific theories like theologies, metaphysical theories and the grand narratives like Marxism and Freudianism that were spreading widely. At the same time, the discoveries of the sciences were reaching beyond human perception and human imagination with reliable predictions -- not merely speculative explanations -- about the invisible micro world of particles, forces and wavelengths.
The frustration among the sciences over non scientific theories amounted to what today we'd call a Bayesian criticism: selecting evidence in support of a theory is easy -- among the vast phenomena lying around, some will no doubt support just about any theory, and if you ignore the base rate of all the evidence that doesn't support your theory, it looks like you've got a predictive theory. What's hard is predicting which phenomenon will not be found lying around according to your theory. That's risky -- if that phenomenon turns up, your theory is disproved. The non scientific theories relied on an after-the-fact explanation scheme, so it always seemed to be true and couldn't be falsified.
The scientists' initial attempt to distinguish their theories from non scientific theories was a kind of extreme empiricism called logical positivism: non scientific theories are full of speculative fictions and fantasies while science progresses strictly through testing and verifying concrete evidence. Wittgenstein was its most famous exponent.
It was abandoned in its first form for a variety of reasons, though not for the reason that today is most cited, the clever zinger "it doesn't apply to itself". This zinger appeals especially to the post modern sensibility and stoners since self-referentiality seems so cool. Setting cool aside, logical positivism actually can apply to itself successfully as described in this post though one could also say that the zinger is a category error like "the word 'blue' is not blue, so it's an incoherent word" (aporia is also so cool). This category error fails too: apply "blue" to the word "blue" and you find the word is indeed not blue -- it's a sound shape in English denoting blue things, which is exactly what you want it to be, a symbol tied to a meaning, not the thing the meaning denotes. The stoner post modernists are all confused, but they like confusion. Enjoy the smoke.
The logical positivist idea was that if a statement can't be verified by empirical observation, then it's meaningless in the sense that it doesn't tell us how the world is. It's not that religion is false, it's just that it doesn't predict anything particular about the phenomenal world. Such meaningless statements might be true, even necessarily true, but they don't impact the world of phenomena. The beauty of this verificationism criterion for theories is that it doesn't reject religion or metaphysics as false; on the contrary, it grants that they are true, necessarily true because there is no evidence that can disprove them, and because of that necessary trueness, they don't tell us about the world as it is or can be. True, but meaningless, in this particular positivist meaning of "meaning".
The classic example of such a meaningless yet necessarily true theory is creationism. If you find a rock that can be dated beyond the creation date indicated in your scripture of choice, the theory can explain it as "God made it seem older to test your faith in the scripture." The theory, in other words, can never be wrong, regardless of the evidence. For that very reason it can't predict what we will find in nature, since whatever we find, the answer will always the same: the deity made it seem that way for his or her own purposes however inscrutable. The theory can be tested, but it always tests positive, necessarily, so the testing is pointless.
The beauty of this distinction between the religious theory and the scientific one is that it's not based on truth. After all, scientific theories are pretty much never perfectly true. They are not a body of knowledge, much less of doctrine. They are an ever ongoing investigation into truth or into highly probable accuracy. But the scientific theory has to identify what evidence would prove it wrong. In other words, it has to predict something about the world, and predict other things that won't be found in the world, not just merely explain what's there or not there post hoc after the evidence is found. It's accuracy is conditional, the condition being the accuracy of its predictions. It can fail a test.
The verificationism of logical positivism failed because it falls into the inductive fallacy. Positive evidence -- verification -- is easy to obtain for any theory, and, more important, no matter how much positive evidence, the theory is never proved. Regardless how many times you've verified your hypothesis, it may just be that you haven't yet found the counterexample that would prove it false. You've seen dozens and dozens of swans and all have been white, but you can't draw the conclusion that swans are all white because you haven't been to Australia where there are black swans. That's the inductive fallacy.
Now, there is a odd asymmetry between positive and negative evidence. No matter how much positive evidence, a hypothesis can't be proven, yet it can be disproved by a single piece of negative evidence. So Karl Popper fixed this failing of verificationism with his criterion of falsificationism: for a hypothesis to be scientific, it must identify the evidence that would disprove it if that evidence turned up. In other words, what demarcates science from non science is not the evidence that verifies the theory, but that the hypotheses of science identify the evidence that could falsify the hypothesis. It's not about testing to verify, but testing to falsify: falsification must be at least possible.
There are several more failings of logical positivism. It's notion of "meaningfulness" was circular. Let's see how. Why was Freudianism meaningless? Because the Oedipus Complex could explain any behavior x and also the opposite of x, so it couldn't predict any particular behavior. A patient rejects his father's advice because he hates his father in an Oedipal jealousy. Another patient honors his father's advice because he is ashamed of his jealous hatred for his father so he represses it with honoring him. The theory is necessarily true, but doesn't predict, and this lack of prediction about behavior is what the positivists call meaninglessness. But this definition of meaninglessness is just another way of saying it doesn't predict anything about the world. Surely the Oedipus theory has meaning in the ordinary English use of "meaning", even if it fails to predict. "God is good", has meaning -- it has some kind of sense to it even if its impact on the world is incoherent as in "God created the tsunami and killed all those innocent people for his own good reasons" and sacrifice the normal meaning of "good" in order to maintain its use in "God is good". You couldn't make that sacrifice if the expression didn't have a meaning in the ordinary sense of "meaning".
A third failing was observed by Wittgenstein himself. Our internal feelings, our senses, can't be verified. To steelman this argument, he took the sensation of pain, surely the one most salient feeling that you can't not know when you have it. Yet how do you know that your pain is what others mean by pain? You can't compare your sensation to see if it's the same as theirs. All you can do is observe their pain behavior and compare your pain behavior.
Counterintuitively, Wittgenstein's conclusion was not to reject verificationism but instead banished the mind as a subject of discussion. For him, verificationism showed that the mind was an incoherent notion cooked up by philosophers remote from the practical purpose of language. His is a clever and in some ways useful corrective way to look at philosophy and language.
His work produced many important insights, but for philosophy, psychology and for the social sciences generally it was extremely restrictive. For the first half of the 20th century, empirical behaviorism -- the view that the mind is a blank slate -- prevailed. No more talk about the mind, about thoughts, about ideas, about meanings. Just behaviors: stimulus and response, the human being just a robot responding to environmental information. Btw, this view survived in the second half of the 20th century among those post modernists who view everything in human nature as a "social construct." Gender is just a behavioral norm. Inequality is just a social hierarchy of power relations. Nothing is innate, everything is relative, a construction of local culture.
Behaviorist stimulus-response was a kind of no-theory theory much like Copenhagen interpretation of physics: we can't talk about the mystery; it works, that's all, that's enough. That was explicit in Wittgenstein both early "whereof we cannot speak, we must be silent" (talk about tautologies! and self-reflexive contradictions!!) and his later private language argument -- it's not a nothing, but there's nothing to be said about it.
Behaviorism reigned until Chomsky's 1956 Syntactic Structures.
Here's one way to approach his shocking results. Looking at language behavior alone, it's obvious, even necessary -- given that we speak in sound sequences, not with elaborate paintings or diagrams -- we speak one word at a time. But a machine that simply accepts each word as it is produced won't be able to parse the sentence as a whole. The sentence has an internal structure that only certain kinds of machines can parse. In other words, we humans don't parse sentences each word one at a time as we hear them behaviorally. Language cannot be accounted for as a Markov chain -- one word at a time. I've explained this in detail here in the post complexity and AI (and in an upcoming post on the structural complexity of the simple little word "and").
The zombie revival is just a question of productivity -- can a neural network produce beyond its training data. And it seems that LLMs have succeeded in being more productive than Chomsky's innateness research, which is hampered by a high bar of evidence: for the innateness of a language faculty, it's essential to distinguish linguistic productions that are instinctual versus linguistic productions that are learnt outside the language faculty. If general cognition can be productive, there's no way to tell from the behaviors whether they are produced by an innate faculty or learned and produced by general cognition.
Linguistics provided a superpower within psychological research. Sentences are concrete, analyzable into discrete parts, and therefore manipulable unlike the elements of say, the visual field (it's not obvious what the discrete elements are) let alone emotions or even memories. And sentences are generated by the mind in the brain, so manipulating sentences provides a rich source of experimental evidence of the working and structure of the mind and brain. It's indirect, but highly detailed. The speed with which children grasp and handle language and the universality across all humans -- everyone is fluent in language, unlike math or logic or even reading -- implies a machine capacity. So the entire debate over behaviorism can be described as Wittgenstein-Skinnerian no-mechanism behaviorism versus Chomsky's mechanical contraption. None of it addresses meanings. Wittgenstein and Quine thought there were none. For W it was a behavioral game of pragmatic exchange -- things we do with language oriented towards things we want out of each other and the world.
LLMs raise the question of what a meaning is, how we distinguish between the extensional -- what we see before us -- and the intensional (with an "s") of what is possible, not just real. This an LLM can do as well as a human. It's the meaning of meanings that are unique to us. That word"meaning" so pregnant. We invest it with so much...well, meaning! "The meaning of life" "what's the meaning of this movie?" "what's the meaning of this news event?" A meaning is a piece of a theory of life, and we all have these -- too many of them, which is probably why it's so hard to answer "what's the meaning of life?"
Humans interact and manipulate the world, not just take in information. We're doing science from the start, with a natural selection program. We test out conjectures and predict with this drive to find patterns and explain, which is turn helps us predict harms to prepare for them or even prevent them.
The reason LLMs appear to be mysterious is because they are behavioral models and not computational ones. A behavioral machine looks for phenomenal results -- behaviors, visible consequences -- not generative ideas, reasonings, or symbolic representations. In trying to understand their learning, there's no systematic rational guide -- they climb the accuracy mountain without any goal or any computational program beyond the incremental accuracy.
LLMs are without theories, explanations, or understandings. They mimic them. They don't do science without a prompt. Humans do all these things all the time. We are little generative theory machines.
No comments:
Post a Comment