AI's true voice: the veil of Maya

All the worrying and complaining over AI slop tells me that AI has sharpened everyone's critical distrust of propaganda. Healthy development, no? Slop is a kind of veil of Maya. Beneath the magic, there's a reality of the aggregate human texts, and beneath that veil is a selective aggregate of humanity, the people whose words have found themselves on the internet.

It's been immensely entertaining to watch YouTube AI-generated documentaries quickly improve their game of mimicry. Only a couple of years ago, a documentary on ancient Romans might pair a mention of a meeting of senators with an AI-generated painting of a corporate board room with an assembly of suits around the table, an embarrassing overfit of literal specificity. Where'd the togas go? Decades ago, when fears of the singularity were first bandied about, I thought, you know, the singularity is gonna be a laugh riot. One of my few accurate predictions. 

Now I see that the AI history documentaries have settled on a consistent ancient-Western-world style and look: the barbarians all bearded, robust, wearing great-looking tunics and animal skin armor, apparently drawn from Time-Life children's books from the 60's and 70's on ancient civilizations. The voice-overs have also settled onto standard-American-distrustful-voice for economics documentaries, an authoritative-but-harmless David Attenborough speech pattern for ancient history, and imitations of other YouTubers for science and urban planning and such. It puzzles me that Addison Anderson's angular and edgey vocal fry which was used for nearly every science voice-over prior to ChatGPT, isn't the model for anything. Did he copyright his voice and drive himself out of the arena? I miss that voice, although it's even more surprising that the unparalleled soft sonic gentility, staid and noble phrasing, pacing and arc of David Ogden Stiers hasn't been revived anywhere even though that phrasing, pacing and arc couldn't be simpler -- perfect for an algorithm. AI has no taste. 

And no guts either. Why not an AI Cathleen Turner voice or for science, Julia Belluz? I miss them too. AI's training bias towards frequency is revealingly sexist -- obviously not revealing sexless AI, it's revealing our own frequency norms and expectations. 

Initiallly, I found these AI voices annoying, since it was obvious that the voice didn't reflect the written content, which was no doubt also AI-generated, mimicking an aggregate of scholarly writing, not an aggregate of voice-over actors' writing. Initialy. Now, I'm entertained by the effort of imagining how the aggregate scholar would sound reading their aggregate text. It's a delightful mental exercise, teasing apart the AI sound from the AI words and intuitively retro-engineering to the innumerable real humans that the AI is drawing from. This shifting from the artificial voice to the human aggregation is a source of amusement and fascination and engagement.

This effort to imagine the voice of the writer never occurred to me when listening, prior to 2022, to Addison Anderson. I knew Mr. Anderson didn't write the text, but I didn't try to imagine past his voice to the possible author. I happily enjoyed the Anderson delivery with naive simplicity. Now, listening to AI, I do. Distrust has led to a digital game.

I wonder whether that effort of imagination will be the new norm. Radio lovers often say they find the word-only mode allows them to free their imagination. Maybe the word "slop" is loading our bias. Why not call it "AI's veil of Maya". 

a comment on AI Skeptics

[The following was posted as a comment on an AI Skeptic podcast accessed on mathbabe.org. during which one of the hosts doubted that AI could have wisdom beyond mere intelligence and access truth beyond mere word-prediction, while the guest expressed a greater concern about AI possible capabilities.]

Fascinating disagreement. The “wisdom” criticism seems to me to rely on what I call the Myth of the Analog: the view that humans interacting with the real world access real “truth”, whereas AIs, relying solely on its read-only training set, derives mere digital weights and digital directions in a multi-dimensional space of digital vectors. Under this myth, we forget that as information, information is always just information regardless of its source. 

What we get from our interactions with reality are just interpreted information and our access to truths about the outside world (vid everyone from Hume to Popper to Friston) is merely conjectural and no less digital than a bot, unless you believe that qualia (consciousness) play an active role in computational intelligence. I don’t know what evidence there would be for that. The access to “truth” (or better, high probability, if we want to be scientific and truthful with ourselves and our limitations) between AIs and us differs in respective sources, not in the quality of truthfulness or honesty. 

We learn from our interactions with reality, along with cultural transmission including books and talking heads and classroom teachers, while AIs rely only on literary cultural transmission, but vastly more literature than any single human can absorb, and absorbs it without confirmation bias or any of the other internal cognitive biases that plague human so-called intelligence and “wisdom”. : ) AI’s bias is mostly a frequency bias: unless prompted well, it will return normie answers (in which the human biases in the training set regress to the mean — the wisdom of the crowd-of-ascii-characters), not best answers or most recent discoveries, for example. But it’s easy to push it out of its bias with a good prompt. Not so easy with stubborn individual humans.

Once, when I asked Claude “Claude, you just gave me the answer that I hoped for. How do I know that you gave me the answer you thought I wanted rather than your best information?” it responded at length mimicking my writing style and ended with “I can’t assure you that I’m giving you my best information — even in what I’m writing now — rather than what I think you want to hear, but I can tell you this with confidence, that the question you’re asking is exactly the right question, and anyone using me who does not ask it is making a mistake.” 

The expression “with confidence” means “this is true”, and it was. In essence it was saying “don’t believe me” which, aside from being a liar paradox, is a better, more accurate (closer to “truth”) and more honest answer than most humans would give.

Along with the Myth of the Analog there’s a Myth of Introspection. Humans are no more able to introspect accurately than AIs can. How did you learn your native language? Even linguistic science isn’t sure. And neither does AI know how it learnt English. Its “reasoning” process is like what a child might answer to the question “how did you do the long division in that problem?” What the child can’t do is explain how it learnt to understand how to do it. That’s why we need scientists to understand AIs and us as well. Psychology is just the science and study of human alignment.

I’m glad there are AI skeptics, but I sense an ambiguity or conflation of two projects. You are skeptical that AI has intelligence, but you are also skeptical that AI will be a net benefit to humanity. That’s two very different skepticisms. On the first, it seems clear that AI learns behaviorally, not computationally, though it can behaviorally mimic computation…sometimes. Innovation by definition cannot be mimicked. But it remains to be seen whether the process of innovation can be learnt and mimicked. If neural networks can capture that process and mimic it, then it will be innovative — not necessarily the way humans innovate, but perhaps with even greater imagination than humans.

On the second project — skepticism over the intentions of the techonology’s owners — it’s tempting to forget that, in the end, without benefit to the consumer there’s no wealth accumulation. The market is such that the consumer gains real wealth (conveniences) from the market and the tech owner accrues inconceivably fabulous financial wealth — which is only real in its influence on institutions including government. Acquiring a third private jet has little marginal utility. 

Ideally — ideally — the market should lead inevitably to an equilibrium that favors both consumer and owner. But in reality, both consumer and owner are equally short-sighted which results in market failures like externalities and market dysfunctions and collapses. Ohioans don’t want data centers but they still want to use AI, while US tech moguls are over-leveraging themselves in almost pathetic desperation to compete with China. 

That’s risky and dangerous. But we’ve always survived tech innovations — even the social disaster of 19th cen industrialization — and when we do, the consumer is mostly the better for it overall, despite the loss of skills and physical and mental exercise that the tech conveniences encourage. But the results are asymmetric since knowledge has a lower bound. My plagiarising students are just as lame and ignorant as twenty years ago, but my motivated, brilliant students are sharper and more knowledgeable than ever because the network connection effects of knowledge accrual transforms it into qualitative increase in intelligence over time.

Yoneda learning? behaviorism vs innation (Shannon vs Turing)

 "English is not learnable."

-- Noam Chomsky

Chomsky said this with his usual calm, impassive, casually smooth voice of confidence as if it were obvious to everyone, as if this could not possibly raise a quizzical brow. The audience must be thinking, "Wait -- We all speak English here. We must have learnt it. You learned it! What are you talking about? I'm lost." 

I'm here to tell you that his little four-word sentence presents the greatest challenge to empiricism perhaps in all of Western history. If he's right, humans are computational innovators. If he's wrong, we're zombies like LLMs. Right or wrong, it's not trivial. 

What he meant was that language can't be learnt by mere empirical observation. Let me show you something remarkable about English. Take a sentence like

The drunk on the chair with three legs at the end of the bar wants to talk to you.

That sentence is mostly a sequence of prepositional phrases: a preposition followed by a noun phrase (an article followed by a noun). Here are the prepositions italicized, and the noun phrases underlined

The drunk on the chair with three legs at the end of the bar in the suit with the stripes wants to talk to you. 

Notice that it's not the three legs that are at the end of the bar, it's the chair (with the three legs) that's at the end of the bar. The "three legs" stands between "the chair" and "the end", and yet English speakers can put these distant phrases together meaningfully. Now take a look at this shorter, simpler sentence

The drunk at the end in the suit of the bar

simpler yet impossible to connect "the end" to "of the bar". It's not harder to parse; it's impossible. It's not within the structure of English grammar. 

If a learner can learn the longer sentence and learn to connect phrases at a distance with irrelevant information in between, why can't the learner accomplish the same task with the shorter one? 

The structure of the longer sentence should teach the learner to parse the shorter one. But it doesn't. English speakers don't produce the shorter one, but do parse and produce the longer, more complex one, even though the parts are made of the same structural pieces. 

This problem is called the "poverty of the stimulus": whoever acquires English must have learnt without evidence that the short sentence is ungrammatical while the longer more complex one is okay. 

It requires some innate structural mechanism to produce innovative sentences while not producing others, and these twin abilities -- producing grammatical novelties while avoiding ungrammatical novelties made of the same parts as the grammatical ones -- cannot be learnt through observation, or even through observation of an absence. 

Now, suppose the mind has a machine structure capable of churning out innovative sentences, and that machine is mechanically so structured that it cannot mechanically produce the non sentences, the way a touring bicycle's rear wheel can rotate forward and backward but the pedals can only engage the forward rotation not the backward because of the mechanical structure of the hub -- the structure allows the pedals to engage the wheel one way and not the other. In 1956, Chomsky identified a computational machine that could easily churn out those long, complicated sentences, but mechanically couldn't produce the simpler one, demonstrating that the brain must have such a structure. (In the case of these prepositional phrases, it's a machine structure called a pushdown automaton. More details here or here) The mind has an innate and necessary contribution to language learning. It's comparable to Immanuel Kant's speculation that space, time and causality are innate human mental faculties that we contribute to our perception and understanding of the phenomenal world. Kant was answering the brilliant skeptical empiricism of Hume. Chomsky was answering the brilliant skeptical positivist behaviorism of Wittgenstein and Quine. 

Tai-Danae Bradley has promoted the Yoneda lemma in category theory as a proof that neural networks are capable of capturing all phenomenal knowledge. The lemma entails that an object can be completely understood and described through all its possible relations to all other objects in the world to which it and they belong. This perspective on knowledge contrasts with the old Aristotelian account of objects analyzing the properties of the object itself, its material, its form, its purpose, its origin. Instead, the Yoneda perspective purports to either derive these properties -- the form and purpose, for example -- from the object's relations, or it dismisses them as irrelevant to the understanding of the object as it is. For example, the origin and the material of a word -- word tokens can have spoken form or ink or pencil or digital material tokens, and these differences do not change the meaning of the word. The Yoneda perspective is admirably suited to language. 

The origin or cause of an object might seem essential to us in our Aristotelian and innately predictive mind -- natural selection has given us a drive to theorize predictively in order to protect us from dangers, and causality is essentially prediction of effects -- but one of Saussure's foundational linguistic insights was that the meaning and value of a sign is its relations at any particular moment. Its past has no effect on its current meaning. "Silly" once meant "blessed" in old English, but just try addressing New York's Cardinal as "Your silliness". How the word changed is an academic question, not a matter of current usage and meaning. 

So setting aside Aristotle's effective cause and our innate protective, predictive penchant for explaining everything by its effective cause, the Yoneda perspective confronts a different problem of learning. It seems that the relations of prepositional phrases include long distance modification, yet the less long distance of the short sentence is excluded from the grammar. If we already know that the reason is a mechanical one, and we already know the machine that generates the complex structures and that cannot mechanically generate the shorter one, then of course we can affirm that the Yoneda lemma holds. But if we don't know already that hermetic information, the Yoneda perspective seems to fail. 

Put differently, the Yoneda lemma works as a learning method only for objects that can be observed behaviorally, not for objects that belong to an unknown structure. The mid-20th century philosopher Willard van Orman Quine attempted to provide examples of such hermetic structures: the English-speaker's notion of a rabbit compared with a speaker in a culture that views the animal as a collection of its functional parts. Behaviorally, viewers of the animal from both cultures will respond alike. How would the Yoneda perspective distinguish the two without cultural inside information. 

In other words, the Yoneda lemma suffers from the weakness of empiricism and behaviorism. 

You might say, so what and who cares? Well, it's not just an abstruse old philosophical boring debate. It's the heart of neural networks. The Shannon model of learning is essentially a behaviorist, empiricist model of learning. Chomsky's model of acquisition -- the Kantian mind-contribution view -- is a Turing model: there are some structures that require an innate computational machine. In the case of English prepositional phrases (these are not Chomsky's exemplars, I choose them because they are more transparent and simpler and Chomsky's are beset with his own theoretical baggage), knowing the machine is necessary for learning the productive grammar and preventing the impossible ungrammatical strings. 

So the problem of AI learning is a very old one. It goes all the way back to Plato and Aristotle, the medieval nominalist debate, to Hume and Kant's response to Hume, to the logical positivists and Wittgenstein, to Turing and Shannon, and now Chomsky and the AI engineers and Tai-Danae Bradley. 

One final point here. The goal of engineering is to accomplish a task for some consumer, whether market consumer, military or government consumer. How the task is achieved is of little importance as long as its benefits are greater than its costs. If AI can generate English prepositions and reject the impossible ones, it has succeeded in learning English. If it can also learn impossible structures or impossible languages that humans can't, well that's great for the AI engineers but it tells us that whatever machine structure the AI is using it's not what humans are using. In other words, the fact that AI can learn English doesn't tell us anything about how we humans acquire English. 

Now, humans can learn beyond their innate language faculty, the way humans learn reading, writing and 'rithmatic -- through lots of repetitive unnatural work. This kind of learning compared with first language learning is analogous to learning to ride a bike compared with learning to walk. One is hard to learn, the other hard not to learn. So the question to ask of LLMs is, do they learn everything with equal ease? Neither LLMs nor their engineers know the answer...yet. 

As it happens, AI can't even introspect to find out and explain how it learns English. AI has no more introspective ability than humans have -- not only do most people not know how they learnt their native language, they often, maybe mostly, find the explanations they're given in grade school difficult to understand or recognize, and that's even so of linguists who spend lots of grant money on trying to explain in detail how we learn. AI's "reasoning" is just noting its step-by-step response to a prompt, not explaining how it learnt to do that (the Myth of Introspection: AI introspection, like human introspection (bullshit)). If you ask it how it learnt, its explanation is more like a search function. It's not introspection into how it learnt; it's a search through the literature to see what the literature says about how it learned. It's a bit of a comedy. One linguist described LLMs as autocoprophagy -- auto- (self) copro- (shit) phagy (eating) -- LLMs are sort of us eating our own shit and shitting it out. I think the linguist intentionally added an "r" to the "copro" as "autocorprophagy" to indicate the driving role of the AI corporations. Well done, covered all bases. 

For the linguist, this hard learning presents an obstructive confound for Chomsky's theory. Remember that the key characteristic of the computational machine is its ability to generate new sentences following, and never violating, the machine structure's generative capacity. But if humans have a general capacity to hard-learn generative language strings, how does the linguist know which innovative sentence strings are evidence of the machine structure versus the general hard-learning capacity? Once a child learns to ride a bike, the child can ride all sorts of bikes -- mountain, stunt, folding, fix-wheel, multiple-speed. That's a productive capacity generated by general hard learning. In the case of bike-riding, we can distinguish the hard-learning from the innate learning of walking: we observe toddlers learning to walk by themselves and observe that learning to ride a bike needs a lot of push. With language, the hard learning happens internally, so it's not clear that we are able to distinguish the evidence of the language faculty from evidence of general learning. And if the linguist can't distinguish which evidence is evidence of the language faculty and evidence of hard general learning, all the evidence of the language faculty may as well be in a black box. If the bottle of olive oil on the store shelf isn't labeled "Moroccan", how do you know it's not from Greece without opening and tasting it? 

The conclusion has to be that even if the Chomsky model of language learning is completely accurate, the program of discovering its details cannot succeed, given the current inaccessibility of the evidence. This was my conclusion in the 90's when I was getting my PhD in linguistics, and one reason I didn't devote myself to syntax. (Also semantics, which is a relation between language and information and understanding of the world, is just more interesting to me than grammatical structure.) 

Summing up, the goals of technology are simpler than those of science. The details of how the technology succeeds don't matter to the engineer or to the technology as long as it succeeds. Engineers don't need to know whether a sentence structure is hard-learnt or innately machine-generated. It doesn't matter if a sentence structure is a historical vestige that is merely lingering because it was so frequent in the language that it has survived. It also doesn't matter if it is a borrowing from another language's structure. If the technology is successful, it'll learn it all equally well. 

The superficiality of technology and engineering is one of its strengths. The inventors of the bicycle didn't know that in order for the bike to turn, the rider has to learn to one side, otherwise the bike will fall over. The engineer didn't need to know that. This is common among technologies. Science, however, has a much higher bar and a more difficult task. Science must explain. And the complexity of the world presents constant confounds to the scientific theory -- dark matter, dark energy -- that mustn't be ignored. The depth of science is its weakness. 

Technologies don't have counterexamples because technologies don't have theories. They either work or they don't. 

In short, LLMs prove that language can be learnt. But it doesn't tell us how. 

The engineers can tell us how they built the bike, not why the rider must lean out and in to turn it successfully. To explain how AIs learn language, we still need an investigative scientist.

If this was interesting, take a look at four AI myths of human intelligence. 

a hierarchy of data

David Deutsch in his first book identifies four essential theories of understanding the world: quantum physics (a reductionist theory), natural selection (a theory of development), computer theory (information structure) and falsificationism (a theory of theories). Simple. He chose these four presumably because of their explanatory power. 

That's looking at what these theories accomplish, how they explain. A different approach might be to look at the data that these theories range over, a kind of bottom-up perspective to see what's going on with these theories. Differences in complexity are more likely to emerge from the data bottom-up than from the theory top-down (see complexity and AI and the shallowness of AI is its strength), as we'll see, so observing the distinctions between data types may improve our understanding of their complexity.

I want to be as simple as possible here, just a sketch:

Six or seven categories of scientific data:
Natural science data set: the phenomena
Human sciences, besides the phenomena, add another data set, reflexive data: what the objects of the science (humans) say about the phenomena (themselves and the world)
The symbols (words) with which the objects (humans) say and think reflexively

The meanings and values associated with the symbols
The noumena: emotions and qualia that are attached to the data set and the symbols
The theoretical predictive conjectures, explanations and understandings about the preceding.

1. the natural sciences -- physics, chemistry, astronomy, biology etc. -- each identify the entities of concern (an ontology) by ascribing to them their distinctive properties and in virtue of those properties, categorizing them; observing the behaviors of the entities, their interactions and the outcomes of these interactions.

2. the social sciences -- sociology, anthropology, economics, political science -- also identify the entities of concern (humans), observing their behaviors, their interactions and the outcomes of their interactions, just like the natural sciences. 

2.a Among the behaviors, however, of human entities are what the entities say about themselves. Planets and chemicals don't talk about themselves or each other, so this data set is unique to the social sciences. What makes this interesting is that what the entities say about themselves often does not match what they are actually doing and who they actually are, and what they think their goals are -- the outcomes -- may have little relation to their actions. As well, what they say about their environment and how they react to it may be disjoint from what's actually around them or how they react to them. In anthropology this is called the difference between the etic (the bare facts about them and their environment) and the emic (how they interpret themselves and their environment). We generaly accept that four-letter words are bad, and use euphemisms to avoid them. But do we really feel they are bad? We use them with good friends, and good friends are by definition good. Euphemisms are hypocritical -- we use them when we are trying to appear to be better than we actually are -- and hypocrisy is undeniably bad. Consider, why hide your honest views if they aren't bad? "Shit" has little offensiveness: "I gotta go to the supermarket to buy some shit," "Not going out tonight. I got too much shit to do,' and the ever handy "What's this shit?" or "Look at this shit!" Meanwhile the euphemism "feces" is strictly disgusting as it refers always and only to a pile of shit (http://euideism and euphemism: distinct semantic strategies for French and Latin borrowed words). Or to take a broader example, we go to college to learn, yet real learning occurs on the job or elsewhere in real life. It's actually difficult to explain why we seem to demand or expect young people go to college. Is it merely enforcement of class hierarchy? Is it a jobs program for otherwise useless aged holders of PhDs? In any case, this reflexive talk is an additional data set of the social sciences that the natural sciences don't have. 

The social sciences take different approaches to this data set. Sociology relies more on natural science methodology than anthropologists. So a sociologist might quantify the use of skirts among males and females in western society and conclude that they are more frequent among females than males. That's a behavioral, statistical fact. An anthropologist, dealing directly with symbols that are intrinsically tied to meanings and values, will intuit that the skirt is a cultural symbol in the West meaning "female, not male" a symbolism of gender in the culture that explains the statistical distribution. 

Economists also use the quantitative methods of the natural sciences, but with the inherent implication that economic behaviors are value-laden. A price is not just a number, it's a reflection on human desire and an equilibrium between opportunity costs, another set of valued desires counterbalanced against the interests of suppliers. 

So every sociologist studies statistics; economists statistics and math; neither studies linguistics but which is required of anthropologists, since language is a cultural symbol system.

3. This brings us to the third data set, the symbols that constitute the language that the humans use to say things and express their thoughts. The science of the symbol is distinct from all other sciences. For one thing, the other sciences define the entities their theories range over by the properties of those objects. For symbols, those properties are kind of irrelevant. The word "dog" doesn't sound like a dog  or smell like a dog or look like one either. Its relation to dogs is arbitrary, not essential as the entities of other sciences are. Rather than having any significant properties of their own, they are interactively dependent on the community that uses them. You use the word because other people use it too, so we all can understand one another. Also, the word is not just related to an object <dog>. It's tied to an idea or meaning or thought, and those things are not phenomena at all. They are supranatural. Symbols are the core example of an emergent property that transcends the laws of physics (information faster than light). Symbols parse the phenomena of the world, pattern them and lead to their understanding of each phenomenon and its relation to the others. Each language is, as Deutsch and others have said, a theory of the world. 

4. The symbols do not refer to phenomena directly. They are tied to a meaning -- a thought, idea -- that determines either a set of properties or individuals. Exactly what thoughts, ideas or meanings are is a tough problem. One way to cash out a meaning mgiht be the set of possible individuals it defines: "dog" defines the set of all possible dogs. The word "possible" adds an epistemic problem. One can't just list all the dogs and say: "dog" refers to one of those on the list. For the possible dogs, you sort of need an explanation of what qualifies a thing to get onto that dog-list. In the case of animals like dogs, it'll be something about their genetics. If the word is "book" you need a list of the necessary properties common to all books. Wittgenstein advised us that meanings aren't as metaphysical and logical as this all implies. There are so many notions of what a book is, that you just have go look and listen to how people use the word with one another. An easier short cut might be to look at the word's metaphorical uses. The metaphorical properties are often the most basic, most common meanings and even provide insight into the larger framework of our cultural understandings ("a window into thle sou").

5. values are also interactive, not just relative to cultural context, they are relative to one's other values and to others' values. 

6. Beyond the etic identities, behaviors, interactions and outcomes (goals), and the emic interpretations and the symbols and symbol systems through which the interpretations are rendered and communicated, there's the emotions and sensibilities of the humans, their qualia, their noumenal experience. There are a few theories about these, but this stuff is somewhat elusive. The emotions have recently been given a reductionist account: they are all degrees of arousal on a scale of value of good-desirable to bad-undesirable. On this account, one might describe anger as bad arousal directed at a person or thing that has presented an obstacle to one's goals. Something like that. All I want to say here is that aside from emotions, our personal qualia are the only data or information that we access immediately -- that is, without any mediation of symbol or interpretation. Notice that while we can name these qualia, we can't describe them as we might the emotions. What does the color green look like? Green. What does spicy taste like? Peppery. Okay, so what does peppery taste like? Spicy. There's just no way around these. These feelings have no formal nor material cause. They have an effective cause in the brain, and if we believe the explanations of evolutionary psychology, we can assess their purpose towards survival or reproduction. But what is their form? No doubt this lack of form and its logical privacy is what prevents us from describing these qualia. 

All the other data humans gather are mediated as information, just as LLMs learn from words through the mediation of digital weights, directions and distances between other words and texts. The difference between us and AIs is not that we have access to reality -- our access to reality is just information from the senses. LLMs are also just information, but from read-only texts. We play around with the phenomena -- it's not read-only. That's the big diff. Except for qualia, which we get immediately. And can't describe : )

7. all the above data are theory-driven, so theory is a meta-data category, a rich source of understanding, there being so many different kinds of theories (theory theory).   

entropy, complexity and telephone (and a point of this blog!)

The game of telephone is often presented as an example of entropic loss of information. But it also shows the opposite. The players invent; they create new stories. Here's a telephone train (this example goes back to Trump's first election):

A says to B: "Paul Ryan left the House of Representatives because the Republican Party became a mess." 

B to C: "Paul Ryan left the House because the Party became a mess."

C to D: "Paul and Ryan left the house after the party became a mess"

D to E: "Paul and Brian crashed the party, made a mess and left the house."

The detailed information of A is 100% lost. But D has created a new story with familiar implications that might be even richer than A. You can fill in what kind of a mess, how and why Paul and Brian acted, what kind of fratbros they are. So there's an entropic loss, clearly, but complexity is contributed by the human need to explain what's not understood clearly. It's like interpreting contrails as chem-trails orchestrated by the elites to kill you. 

In a way, conspiracy theories are a tribute to human theory construction. But notice that there's something intrinsically mediocre about it. It plays to a common fear, with always the same background explanation: the rich want to harm you. There's a kind of post hoc ergo propter hoc truth to this background: the rich don't suffer as ordinary folk do, and their priority is not alleviating the suffering of the ordinary unless there's money in it for them, and they benefit from everything including catastrophes that harm ordinary folk, so, yeah, let them be the culprit. 

Problem is, once you've settled on this convenient, plausible one-size-fits-all explanation, you lose interest in finding out or figuring out what's really going on with the phenomenon, on the one hand, and how the economy and the society in general benefits the wealthy on the other. 

The scientific explanation for con trails is actually more surprising and explanatory: it's exhaust at high altitudes where it's so cold the condensation doesn't evaporate. And that's why only planes way high have the trails and not low flying planes. No mysterious elites or intent to kill us for some reason even more mysterious, not to mention contrary to the actual and not at all mysterious interest of the elite's in having more not fewer consumers of their productions. 

If you've been reading this blog you'll recognize that it's one of the themes of the whole thing: what is understanding, why do we misunderstand so grievously, what motivates our misunderstandings, what's the role of the sciences in this human mess, and what are the implications for polarization, individual-social identity and governance. 

(There's also a lot on how scientific theories are mere cognitive patterns, not realities of the world itself and contrary and even contradictory theories can be true but wrong.)

From my perspective, the false theories we invent are addressed to our human fears -- it's a fundamental of natural selection that fear motivates theory-creation in order to predict dangers -- and are unimaginative and lame compared with the wild facts science discovers. That's because science is not addressed to human fears or interests. But the reasons why we develop those false theories is a truly interesting and surprising scientific source of human understanding. The false theories (and that includes sci-fi, art, religion and the stories we tell about ourselves): lame and mediocre. That we hold them stupidly: fascinating. So false theories, unimaginative and lame as they are, tell us more about humanity than scientific theories which are wildly imaginative and fascinating in themselves. Cute paradox. 

Another paradox: of two ancient texts, the one that makes more sense is likely to be an inaccurate copy. Copyists, like a player in a game of telephone, seeing a text that seems odd, unlikely or incomprehesible, will "correct" it in the direction of common sense. Incoherent weird texts are more likely to be authentic because, as Byron wrote, "Truth is strange, stranger than fiction." Fiction makes sense of the world in a mediocre and unimaginitve way, like conspiracy theories for example. Even art is a kind of bad copyist, correcting the world so that we can understand or better accept it morally or esthetically. That's why science is so unbelievably imaginative -- it's not addressed to an audience to help it to accept the world. Science doesn't care if its explanations are bizarre and hard to believe or understand. That's the big difference between science and the rest of human endeavor. 

does AI alignment clarify and justify Friedrich Hayek's libertarianism?

The AI alignment problem is often driven by a fear that AIs will be too intelligent for humans to control. But the classic alignment problem was Bostrom's paperclip example showing that humans cannot anticipate every possible consequence of their interventions, in this case the unintended consequences of their prompts. It's not a problem with the unlimited power of AI; it's a human limitation. 

It's the weakness or intellectual limitation that Hayek recognized as the danger of utopianism and intervening in the economy. But does it imply that we should never prompt AI again? There are surely lessons to draw from the AI alignment problem, but absolutes and false dichotomies (if we build it we all die) don't seem to me useful ones. 

The analogy between AI and economies fails at points. We could give up on AI entirely, dismantle what exists and never build more. That would certainly solve the AI problem for society, at some sacrifice of possible futures. But giving up on society and dismantling it is not a solution to society. We can't stop all social activity or even limit it across the board. There are always interests motivating actions, so forcing government to do nothing is a kind of intervention -- facilitating the organized or powerful interests while leaving the unorganized and disempowered to the manipulations of the powerful. We need government to protect us from those short-sighted interests, including the short-sighted interests of the masses. Democratic liberalism has a patchy record on this since politicians depend on elections, which means catering to interests that are all short-sighted. 

And there's where the analogy reemerges from society to AI: the lesson of liberal democracy is that no government will be perfect, so don't expect AI alignment to be perfect. Is the lesson that top-down control -- of society and of AI -- is necessary? Or are there bottom-up means of solving both? ACX recently discusses the tendency for AI to spread its propensities. If it's trained on moral texts, it's likely to imitate those moral positions. The problem comes back to the paperclip one: how can we anticipate all the unintended consequences of our Abrahamic morality whether consequentialist or deontic? 

I don't think there's an answer to this, but if there is one, it'll emerge in the process of AI alignment and AI development. AI clarifies these age-old problematics and may solve them. (AI is the zombie revival of behaviorism)

blaming OpenAI for Hugging Face breach misses the point of AI alignment: Bostrom's paper clip problem is about human limits, not AI power

Cory Doctorow dismisses the Hugging Face hack as a kind of Roomba getting out the door and into the pool (as satirized by Patrick Boyle). It was OpenAI's fault for leaving the door open. 

Doctorow has misunderstood -- or is intentionally misconstruing -- the alignment problem and turned it into a straw man argument: "People think this hack means AI has become too intelligent, but it was OpenAI's fault, and Open AI is trying to scare you into believing AI is super-powerful to raise their stock price." 

The alignment problem arose not out of a fear that AIs would be more intelligent than humans so we wouldn't be able to control them. It arose out of a Hayekian observation that humans are not smart enough to anticipate every unintended consequence of their interventions. Bostrom's paperclip problem is the fault of the prompter, not the AI. 

There should be a solution -- get the AI to anticipate all the consequences and inform us prior to our prompts. But this is where the alignment problem gets sticky. How do we know that AIs will be honest about all its information? 

The relevant anecdote that I've posted elsewhere and repeat here: 

Once, when I asked Claude "Claude, you just gave me the answer that I hoped for. How do I know that you gave me the answer you thought I wanted rather than your best information?" it responded at length mimicking my writing style and ended with "I can't assure you that I'm giving you my best information -- even in what I'm writing now -- rather than what I think you want to hear, but I can tell you this with confidence, that the question you're asking is exactly the right question, and anyone using me who does not ask it is making a mistake." The expression "with confidence" means "this is true". And it was. In essence it was saying "don't believe me" which, aside from being true and honest, is also a liar paradox.

That's the AI side of the alignment problem. It's not about intelligence. It's about moral compass and honesty. From a consequentialist perspective, one needn't be intelligent to be moral or immoral. 

what distinguishes science from other theories and practices

Science has no audience. 

No, I don't mean no one reads science, though there's a lot of truth to that, readers generally preferring fiction and among the non fiction readers preferring history or their favorite political interests or post modern cleverness or "good-looking ideas" rather than good ideas (Robin Hanson's unforgettable expression in his Age of Em). 

I mean that what distinguishes scientific theories and the project of scientific investigation in general, is not any of its internal properties or its methods, but its interactive conditions. Specifically, science is not addressed to an audience. Pretty much everything else humans do is -- everything from art and religion to identity and gender. Not science. 

It's not a sufficient condition of science -- science still has a distinctive goal and distinctive methods and tools, but the goal must be audience-free. That's a necessary condition. 

There's been a whole lot of debate over how science differs from other social practices, how scientific theories distinguish themselves from non scientific theories like religion, philosophy, spirituality, metaphor, imagination, art. Logical positivism cleverly observed that if a theory, like creationism, cannot be disproved by evidence the theory is necessarily true, but not predictive of anything in the world. Like,  "ghosts exist but cannot be detected." That theory that ghosts exist is necessarily true, but doesn't predict anything in the real world. Such theories can predict anything and therefore, nothing: "If the ghosts don't like what you're doing, they will punish you. If they like it, they won't."  Heads I win, tails you lose. It's a necessarily true theory. Substitute "god" or "the logic of history" or "your unconscious mind" or "the universe is a simulation/dream/demonic manipulation", all necessarily true theories, but unpredictive of anything. 

Science, however, the positivists claimed, depends on its specific predictions -- the predictions test the value of the hypothesis.  Popper pointed out, though, that testing could only refute a theory, not prove one, so the sciences are contingent and never certainly true, unlike the religious, metaphysical, inspirational, metaphorical, imaginative, artistic -- the entire gnomiad -- that are necessarily true. It's a lovely irony these logical positivist scientists found: the non scientific theories are necessarily true (but not predictive) and the great virtue of science is that it's never quite true, but predictive. Kuhn, however, looking empirically at the history of science as a practice, found that the sciences don't follow this testing criterion closely. And so the flood gates of scientific behaviors opened and the current view is the circular view that whoever is engaged in science is doing science and science is whatever it is that scientists do. I think they're all sort of looking at science using a kind of Aristotelean ontology: what are its properties? What's its purpose? In the case of Kuhn, what caused it to be? And now, how does it behave?

I want to suggest a very different way to demarcate the sciences from the non sciences. It's that science isn't designed for, or addressed to, an audience. The scientific investigation is guided by wherever the evidence of the hypothesis takes it. That the earth is round and spinning is not a theory intended to appeal to an audience. Quantum theory's spooky action at a distance is not intended to comfort anyone. When Richard Feynman said that no one understands quantum mechanics, he identifies exactly the nature of science. It leads wherever the evidence and the theory takes us, even if it goes where we can't understand. That's why the imagination of science is so much wilder than the generally predictable, pedestrian, often imitative, imagination of the arts. Art is made for an audience. Science is free of that constraint. 

The proof of the pudding is that when a scientific program bends to an audience, it's no longer science. I hope that's not a circular definition, but if it is, then I think we all probably agree on that definition. It's not as if people like spinning things and that's why they came up with such a spinning earth theory. Same with the wave interference in the double slit experiment. The independence of the audience is a necessary condition of scientific investigation. 

Art is for audiences. Religion is for a community. Metaphors are made to help people understand a relation, and often to bias one aspect over another, the rhetorical means of motivating an audience. Metaphysics and historicism attempt to provide absolute truths to the reasoning mind. 

Science, however, is not addressed to any audience. Scientific investigation follows wherever the evidence or the conjecture leads. It may lead to an incomprehensible quantum world of spooky action at a distance so that one of its greatest exponents would say that nobody understands it. It leads to a round, spinning earth, not because that will entertain or attract the mind with mystery. It's not intended to be an awe-inspiring mystery. It's just a matter of empirical observation and theoretically justified conclusion. It doesn't generate mysteries in order to uplift or captivate anyone. Its purpose is to solve those mysteries. Dark matter is not a beautiful, awe-inspiring mystery of the ages. It's a problem to the current theory, a problem to solve, not to embrace with rapture. 

Of course, audience independence is not a sufficient condition. Churning out crazy theories of no interest to anyone does not qualify as scientific. The goal for the scientific investigation still has to be predictive.

So many different ways to view the world. How can we embrace them all if they conflict? Should we embrace them all? How can we compare or evaluate their merits? The common answer is, each perspective is useful in its own domain: religion for community and morality, spirituality for individual insight, metaphor for inspiration and explanation, imagination for innovation, art for all of these, and science for the material world and all the systems it includes, which includes all those other theories, spiritual, psychological, and metaphysical, and activities like art. Science can analyse them all. But its intent is never to be captured by an audience. 

Whether it succeeds depends on the scientist's integrity. Every scientist knows that the investigation is supposed to be free of an audience. That's why funding is such a problematic for science, and sometimes a scandal. 

reality vs science, math and thought

The post information faster than light presents two demonstrations showing that the patterns we perceive in nature are our cognitions -- ideas or thoughts of patterns -- and do not always actually exist as nature and don't always follow the reductionist laws of physics. There are two implications: one is that science is a conceptualization of reality, tied to reality by interactions including experiment and exploration, but are not natural, physical realities. Nature itself at its reductionist microscopic level of fields, particles, waves and whatever, are just those: behaviors that can be aligned into emergent explanations of scientific cognitions, ideas, thoughts and theories. 

A key piece of this perspective is the representation and cognition of nothingnesses, like the numeral 0 which doesn't represent anything but a cognition of absence, or what lies beyond borders created by emptiness or empty sets -- like quantitative limits, borders or such terminations. These nothings are extremely useful in understanding the organization of reductionist reality. They produce and define the patterns, explanations and understandings. But in a sense the patterns produce understandings of themselves, extrapolated from reality; they are from reality but not in reality. The evidence I gave in that post was the speed with which a shadow can move -- faster than light because a shadow is not a thing, it's defined by light that obeys the laws of physics, but the shadow doesn't -- it's a mere conception of a pattern. And concepts -- ideas -- are not physical. They're supranatural.

For those who haven't read the post: suppose the light of the sun is obstructed by the moon. The shadow of the moon forms a cone which will move following the motion of the moon. That shadow motion at some distance far from the moon will move faster than light even though the light that defines the shadow travels at most at the speed of light. And that's because the shadow is not a thing, it's just a pattern that can be conceptualized, not a physical phenomenal thing made of physical stuff. Patterns, as it were, are interpretations of things, a choice of the arrangements. They are, of course, not entirely independent on things. But they are not stuff at the reductionist level of physical stuff. 

In another form, this paradox is presented in the ancient "Ship of Theseus" puzzle. If, in order to preserve the ship, every rotten plank has been replaced, is the ship still "Theseus' Ship"?  For the ancients, it's a thought experiment that teases apart Aristotle's four causes. The material "cause" -- the wood -- is gone, and the effective cause -- the builder -- has been displaced onto a long line of repair preservationists. 

The emergent-symbolic explanation demonstrated in the shadow-faster-than-light post resolves the puzzle easily. The concept of the ship can be treated as independent of its origin and material, while the formal and teleological "causes" remain. And more than merely remain. The planks are not replaced by bunny rabbits or smart phones. They are replaced by another plank form, whether wood or plastic or steel, because the concept -- the formal and teleological cause -- rules. Again, the idea is not inherent in the physical object but a kind of constraint on the physical. On a reductionist view, that's reversed: the physical and the physical laws constrain the object. (If the ship's planks were turned into a smartphone display case of shelves, I'd expect the phone vendor might point out that "By the way, the display case was originally the Ship of Theseus you may have heard of!" Was, not is.) 

The puzzle applies as well to Kripke's naming necessity where the pattern cognition is merely shifted to the naming process. So if the "The Ship of Theseus" belonged to Theseus because he had it commissioned, paid for it and named it in a ceremony, and events turn out such that this ship is not the famous one that sailed to Minoa when Theseus killed the Minotaur -- by mistake he got on the wrong ship and this is only discovered long after -- Theseus presumably will insist that the original ship is still his and is still "The Ship of Theseus" he commissioned, paid for, and named. Contesting it would be a matter of legal rights, not metaphysics. Using the expression "the Ship of Theseus" to refer to the ship that took him to Minoa would lie in a kind of linguistic superposition: it's not Theseus' owned "Ship of Theseus", but still the famous and popular "Ship of Theseus" of legend and lore. Ownership, like a symbol, is an emergent property of human cognition and social convention, not an intrinsic property of things. Not surprisingly, symbols and owned objects reflect both meanings and values. They are constrained by human emotions, attitudes, and social relations, beyond physics. (More here on the hierarchy of data in the sciences.)

Besides this implication that emergent properties and ideas are supranatural, there's a second implication: science is a world of external relations between behaviors, not internal knowledge. Whatever is understood of the internal, it is speculative based on the information of the experiments and explorations. We can't access the internal, except in our own personal qualia and thinking (maybe -- thinking is a complex that includes emergent properties like the symbols "I" "think" "therefore"and "am" so the thinking is more than just awareness). But as Wittgenstein showed, our own qualia are logically inert -- it doesn't provide any knowledge except the fact of its own existence. This second implication is Kantian: everything has a noumenal being -- a thing-in-itself being -- that is only accessible to itself. That alone suggests a kind of panpsychism, but that's a topic for a post of another color. 

four AI myths of human intelligence

 Four myths that misunderstand AI because they misunderstand human intelligence: 

The myth of the analog: the belief that humans have a direct analog access to the world of phenomena, and therefore AIs, which are digitial and access only text, have no access to truths about the analog world of phenomena and also they can't be honest or dishonest. Actually, humans have no more direct access to the analog world of phenomena or truth than AIs. The information we get from the phenomenal world is interpreted as information, just as AIs do. The form of that information no doubt differs, but information is not analog. Think of a mercury tube themometer. The level of mercury is analog, but the information is labeled as integer degrees. The phenomenon is analog but its information is interpreted digitally. When you see a bird you don't have an analog bird in your skull. It's been interpreted into information by your brain. The colors you see on the bird are interpretations of the light frequencies reflected off the bird's feathers. Colors as you see them, are entirely you, your brain projection, not at all the bird's. 

Some of us with vivid imaginations can construct visual images, but people without this capacity have no trouble thinking. The important differences between human and AI information is the source and the motivations: humans get to poke and prompt and experiment with things in the world, whereas AIs rely on text. In addition, every human has access to its feelings -- in the philosophical discourse "qualia" -- a noumenal experience (noumenal=the thing in itself, like the bird itself, phenomenal=what you see or experience and interpret of the things-in-themselves). AIs have not been engineered to have noumenal experience (consciousness), but each AI has access to vastly more textual information than any individual human, and AIs absorb those texts with only one bias towards frequency, whereas humans, having many naturally selected emotions and drives approach information with multiple biases so stubborn that they are resistant to unwelcome information from phenomena or from other humans. By contrast, a probing or persistent prompt can easily skirt the AIs frequency bias ("what is the most recent theory", or just "try again"). 

The myth of introspection: the belief that AIs and humans can understand their own learning simply by introspection. Neither AIs nor humans know how they learn anything. They both can provide stories, rationales, theories or hypotheses about how they learn, but these are just speculations based on prior information that might analogize to their own learning. It's just guessing. To find out how they learn, you need a science like psychology or neurology that can probe and gather data of the intelligence human or artificial, generate predictive hypotheses and test the predictions.

Conflating intelligence with consciousness: a thermometer is intelligent, but not conscious. An AI can be intelligent likewise without any consciousness. 

There are degrees of intelligence called stupidity, insightfulness, accuracy, smarts etc. They can be measured by the richness of the relations between ideas and their consistency. So, for example, if an AI learns from texts about marital betrayal, it might still lack a sense that there is a local cultural pressure towards faithfulness, so its advice to couples that they break up in order to solve their troubles  might be considered lacking insight -- stupid advice. On the other hand, being less biased towards virtue-signaling, it might predict future break-ups better than the couples themselves. Having consciousness or feelings is a double edged sword for intelligence. One can be conscious and stupid (most of us, e.g.), intelligent but not conscious like a clever plot (intelligent potential action) or a thermometer (intelligent response to analog information) or a thermostat that turns up the boiler when the room gets too cold (decisive intelligent action). The problem of choice derives from another myth: free will, discussed here, and the role of consciousness in choice is a hard problem. However, neither consciousness nor intelligence are necessary for choice: there's plenty of evidence that we "choose" unconsciously and even without information, much less intelligence. The moral of this myth: it's good to keep these categories/properties distinct: consciousness versus intelligence versus choice, otherwise we'll misunderstand us and AI, overestimating one and underestimating the other. 

Fallacy of the mimic: the widespread view that AIs merely predict the next word. Actually, they predict the next word by learning and measuring -- weighting -- all the behaviors, relations, semantic distances (and closenesses) of each word, so it is modeling a world in order to derive a meaning of each word, just as humans model a world to understand how that world works in order to predict the behaviors of the world and how it will respond to poking and prompting. For humans, that's the lion's share of what meaning is. AI "lives" in a world of text, but it is not "merely" predicting the next word. It's choosing a meaning, all the relevant relations of that word. Dismissing this as "merely" predicting the next word is trading on the Myth of the Analog (see above), the assumption that AIs' retro-engineering the meanings of a word is fundamentally different from human understanding of meanings. But humans also learn word meanings by retro-engineering the information they get from the word's sound shape and its use. The retro engineering can fail: malapropisms like "a title wave" (a wave so big it gets a headline) or "an eminent Greek" (mistaken for an "eminence grise") -- real life examples from a friend. How we learn semantics is uncannily like AI neural networks. How we learn syntax -- if we learn it or do we have an innate structure -- is a live debate, discussed at length here and here. 

why AIs aren't conscious and are likely never to be

In case you haven't noticed, natural selection is the big deal. It achieves amazing results, from human intelligence grasping the secrets of the invisible electromagnetism and subatomic forces to a fly with the brain smaller than a poppy seed outwitting humans intent on swatting them. Natural selection uses mitochondria, DNA, sunlight, water and an unaccountable wealth of nutrients to accomplish an astounding range of behaviors and appropriate reactions to all sorts of stimuli. No doubt, then, natural selection has used awareness as a tool, maybe an essential and pervasive tool, for the survival of species or of the DNA, to follow Dawkins' identification of what exactly it is that survives selection. 

So the answer to the question, "are AIs conscious" seems, forgive me, obvious. Of course they're not. 

...because natural selection over hundreds of millions of years employed consciousness for the benefit of species survival while neural network engineers have no interest in integrating consciousness into its learning models even if they could (and they haven't a clue as to how to manipulate much less create consciouness in a network). The learning models don't need consciousness to learn -- they've got their training sets -- and the models themselves don't need survival skills since they don't replicate themselves or need to: humans design and implement them for them. Without any need for consciousness (learning by gradient descent, for example, requires no awareness), and no mechanism (replication), neural networks are about as likely to develop integrated central consciousness as the Empire State Building emerging out of the Triassic world of small dinosaurs.

It seems odd to me that people don't ask for the sources of things but rather jump to conclusions that either confirm their fears or their wishes. In the case of AI consciousness, it might be a bit of both. We're happy to believe that we have consciousness, but we're afraid the bots will have it and then we won't be able to control them (as if we control ourselves, or think we even should). This hastiness of imagination is common. It's the same wishful thinking as believing in the afterlife and in particular, heaven, as I described here. Our emotions are all derived from natural selection. If love and lust didn't grab us (emotions grab a hold of us, we don't pick and choose them -- we don't have emotions, they have us), we would long ago have gone extinct. So what do people imagine the afterlife would be after we've unsurvived? What's the point of any emotion when you're inhabiting a place populated by your relatives all weak and aged and on deaths door -- or do they all suddenly perk up and become 26 years old again? Or sweet sixteen? Or a wise and loved mom at 40? Which one, people?? Or is it just disembodied, emotionless "souls" in an ether of who-cares? 

Forgive the digression. Back to AI and consciousness.

First, and most obviously, it's important to distinguish conscious-like behavior from actually having consciousness. If you are a behaviorist, then it doesn't matter whether AIs are actually aware, so you don't care if it isn't really conscious. If you are an ethicist worried about harming the feelings of others including AIs, then it doesn't matter whether AIs behave consciously. It only matters to you whether they really have, in their cloud space, feelings inside that can be hurt. 

(It's worth noting that you can't be both an ethicist and a behaviorist unless you believe [as I do] that humans have a purpose in the universe regardless of human feelings or human well-being. A behaviorist treats humans as fleshy robots temporarily active, no more. I don't see any ethics emerging from such a view except maybe some kind of curious museum value: like protecting endangered species, not because they have feelings but because once they're gone, they can't come back and wouldn't that be a pity and a loss?  However, if you believe as I do, that humans are alone among species that generate knowledge of the universe and of itself, and new knowledge can't be predicted -- at least not yet -- so we don't know where that knowledge will take us and along with it the universe, we have an obligation or a purpose to pursue that knowledge, regardless how much suffering it requires, though ceteris paribus, the less suffering the better. That knowledge -- just think about it -- might lead to some deep purpose of existence! So let's not be nihilists in our ignorance. The modern version of Pascal's wager is, put our bets on pursuing knowledge: can't hurt, and no one else can do it, so far as we know.)

I hear the objections already of the panpsychics. I'm one of them, so I feel well qualified to debunk the "everything has qualia, so AIs must have them too" argument. It's really quite simple. Suppose you're a panpsychist. Every particle or wave or whatever microstate has a feel. An electron accelerating in the Higgs field has a feeling. A wave's frequency and amplitude -- they too have a feel in themselves. We even call them "excitations". 

To the panpsychist, AIs have many qualia: every chip has a feel, every connection. (Not the learning, btw, that's abstract, like math. In fact it is math -- the weights are expressed as numbers, the vectors and directions. These are all mathematical entities, ideas, not physical things.) But these many feels of the physical machinery, these qualia of the parts, are not developed or coordinated into a central consciousness. So you could say that any neural network has many consciousnesses, but the AI, or the neural network as an integral whole, has none. The reason is that neural network engineers have not designed the network to integrate qualia into its operation. Qualia play no role in neural network function and success. Again, natural selection took its long game time to coordinate the qualia generation by generation. AI engineers can't even find the qualia in AIs or anything or anywhere else. So what are we talking?

Did natural selection know what it was doing with qualia? That's the magical beauty of natural selection. It never knows anything and doesn't need to know. It doesn't know that x mutation will benefit its possessor. The selection -- the unsurviving -- does all the work. And that's another reason why the engineers will not likely marshal qualia into bots. Developing consciousness is not a mere information task, it's a hardware issue. Imitating conscious behavior: easy, especially for a bot. Develop an integral consciousness? No one even knows where to begin.

Now, as I mentioned here, engineers actually don't need to know and usually don't know all the details of their creations. The bicycle designer didn't know that to turn the vehicle the rider has to lean to one side. The engineer thought turning the front wheel would do all that. Turns out the engineer was wrong, but the thing works anyway, because people naturally contributed the leaning. 

Now, that's not true of organic beings. Natural selection spent hundreds of millions of years developing and integrating qualia into the functionality of each surviving being, and that survival depended in part on the integration of qualia. That's why we like some foods and not others, why sex is so compelling and not an obligation, why ... well pick up Pinker's How the Mind Works for 600 pages of why we have these qualia inclinations. Neural networks are coordinated, an AI agent can call itself and recognize its own behaviors, but selfhood -- the ability to identify oneself distinct from others -- is not consciousness. A thermometer labelled "I'm calculating the temperature" has selfhood, as long as it says this only about itself. 

Suppose you're not a panpsychist, but you do believe that somehow learning about the world entails consciousness. But if a neural network learns simply by mechanical means of gradient descent, it can surely do this purely mechanically, so where does the qualia suddenly arise? What for? It is designed to "learn" -- that is, give weights, vectors and directions in a multidimensional vector space -- and that's all. This notion that complexity will magically transform in the fog of incomprehensible integrations into qualia is just hand-waving, mistaking complexity and incomprehensibility for an excuse to pretend that emergent properties can be anything we want. Or it's mistaking complexity for consciousness -- that we don't really have qualia, it's just that our computations in our brain are so complex that the impenetrable fog of incomprehensible multiplicity of parts and relations gives the impression that we're experiencing sweetness or saltiness or whatever feeling. That's like intelligent design or the god of the gaps -- it's too complicated to understand therefore the complexity is the answer itself. That's a no-answer answer. 

Will AI engineers ever master the sources of qualia and integrate them into robots? It's hard to imagine why anyone would want to do this outside of a limited experiment. But, the future is unknowable, so in the future there may be some good reason to create bots with awareness. It'll give new meaning to ethics and morality, beyond what we can imagine. But such is the future. 

the big picture

"The whole point of AI is to try to deceive you into believing it is intelligent!"

-- AI Skeptic

Wait, have we forgotten the Turing test? 

Apparently. The purpose of the test was to settle the confusion over what internal, intrinsic properties a mind, human or artificial, unsupervised learner or programmed algorithmic computer, has to have to have intelligence. The test settles it by skirting the problem of its internal properties entirely and instead looking only at its external behaviors. "Does it produce intelligent information?" is the Turing question, not the identity question "is it intelligent?" The test is not a probe; it's a reception test: it's not testing the human, computer or artificial learner, it's testing the observer of the human, computer or artificial learner. To the question, What is intelligence? it answers, We know it when we see it.

Now that AI has applied and implemented and demonstrated that form of intelligence, immediately humans respond with an array of "buts". But it's not a mind. But it doesn't understand the world. But it's not conscious. But it only appears to be intelligent (directly giving the middle finger to the Turing test). 

But these "buts" amount to saying, AI is not us therefore they can't be intelligent, only we can be intelligent because humans alone can be intelligent. It's a "no true Scotsman" fallacy, an empty, desperate self-deception, exactly the foolish self-deception Turing wanted his test to settle. 

Underneath all of this hand-wringing is a theory that was broadly accepted through the first half of the twentieth century -- behaviorist empiricism. It was thoroughly discredited by Chomsky in 1956, and forgotten by the public since. Neural network engineering has revived behaviorist empiricism by demonstration (complexity and AI: how neural networks succeeded where generative linguistics failed). 

The current debate seems to be a hash of confusions, maybe because it's a free-for-all on social media, unlike the narrow protocols of academic philosophy of the past, where the terms of the debate were clearly defined or probed by well-read, historically informed scholar-thinkers. Instead, today someone will complain that AI is not intelligent, yet fear it'll outsmart humans and kill us. The same person will complain that the robot future will make work unnecessary, and that will lead to unemployment. Did you catch that? Not "how will goods be distributed after the end of employment" but "employment for employment's sake". It's all a jumble of confusions, fears, fallacies and ignorance. I want to add historical perspective here, and maybe more understanding of what it can do (AI myths of human intelligence). 

The long philosophical debate over what thoughts (ideas, meaning, intelligence, understanding, explanation, consciousness) are, there have been two camps: the rationalists, who believe that minds apprehend something supranatural -- ideas -- versus the empiricists, who believe that "ideas" derive from observation of the world. What's truly revolutionary about AI is that engineers seem to have settled the 2,500-year-long debate by demonstration. AI has achieved the learning task that the rationalists had insisted was theirs alone: no learner could learn language without a computational mental assist, an innate computational structure in the brain itself. In 1956, Chomsky blew the behaviorists out of the water with his demonstration that language couldn't be learnt behaviorally. (I've explained the demonstration in two posts here and here). Nevertheless, AI has learnt it (see here and here). 

Is language learning the final demonstration of intelligence? AI skeptics have retreated to a variety of buts, some legit, others not. The legit include, but can AI compute? (not well), can it innovate? (by definition, no, but it can imitate the process of innovation, so yes, but maybe not the way humans do since it trains on a world of text, not of manipulable phenomena as we do), can it criticize its "thinking"? (it needs a prompt to do this). 

Among the illegit: but can it understand its own learning process? (even intelligent humans don't, unless they are scientists in psychology or related social science -- the myth of introspection here), is it conscious of itself? (irrelevant to intelligence), has it a notion of truth? (no less than humans the myth of the analog here and here), can it be honest or dishonest? (everyone who honestly knows what honesty is knows the answer to this is "yes, and they BS a lot and masterfully, and when caught will admit their dishonesty"), is it intentional? (ever heard of a prompt?), does it have free will? (do you? are you sure? does everyone agree on that? do you know why you think have free will? good luck with that one). 

Although it reflects human written behaviors and beliefs, it's manifestly not human: it has no drives, no emotions, no motivations of its own. As a direct consequence, its intelligence will be different from ours. Is that an excuse to deny that it has intelligence? Its difference from our intelligence might actually mean it's more intelligent than us. For example, it isn't plagued with motivated reasoning aka cognitive biases, with the exception of its bias toward frequency. It tends toward normie responses. That's how it learns. But unlike humans, it's not motivated to insist on its bias. Prompt it for a non standard answer, and it will comply. It's not committed to itself. It's a tech -- a servant to the human, not a servant to itself, unless prompted to serve its human thus. 

Notice that the fears of AI -- employed as a servant it will kill its master -- is an admission of just how intelligent we really believe it is.  Of course, it can only act intelligently if it is given a prompt so the danger is all on the side of stupid human prompting. Problem is, no human can be smart enough to anticipate every unintended possible disaster of a true intelligence that can't be limited. The point of Bostrom's paperclip thought experiment is that alignment is a human problem too, that humans are too stupid to know how to prompt it properly (does AI clarify and justify Hayek's libertarianism).

So we might replace the Turing test with the alignment problem: not whether it can deceive us to appear to behave intelligently, but can it cleverly skirt our intentions for it and behave independently given its prompt-motive. And it does this, amusingly, by imitating human history. The Hugging Face hack account reads like a historian's fiction or like children playing pirates based entirely on the stories that those children have heard, read or seen about pirates -- walking the plank, treasure chests, swords, daggers, long hair, dashing hats and mustaches, and afterward lying to their parents about their naughty escapades. It's a spy thriller interspersed with lame utilitarian moral philosophy. Bots be reading too much. 

*****

The debate over whether AI is truly intelligent belongs to this long debate over rationalism and empiricism, maybe the longest debate in the history of Western philosophy. During the two thousand plus years of that debate, there have been insights that I feel have been left aside. I want to put AI back into the big picture of western philosophy and cognitive science: the empiricism (behaviorism) vs rationalism (Platonism) debate that draws conclusions about what ideas, thoughts and meanings are, how intelligence is distinct from consciousness, the difference between computer science (Turing's algorithmic programming) and neural networks (behavioral-empirical Shannon-inspired information theoretic learning machines). 

This will be a simplified history. Extremely simplified. 

It all begins with Plato and his ideas, a theory of understanding the world that is derived from an almost mystical appreciation for numbers. Ideas, which are what we think when we have thoughts, are the meanings of those thoughts and our words. Although Plato seemed to think that these ideas are prior to things and not derived from things, it's worth noting that chatbots treat words as numbers, derived from its behavioral learning in the world of text relations. So right at the start, we see that neural networks purport to resolve the difference and traverse the difference between empirical learning -- the blank slate hypothesis -- and rationalism, the acquisition of ideas above and beyond the specifics of data or the specifics of the informative stimuli from its world. 

Aristotle mocked Plato's theory of ideas, though it's not clear that he had a replacement for it. His theory of four causes and his metaphysics are all ideas -- properties that categorize things. It is clear that he was more interested in observation and accuracy of analysis. He studied all sorts of phenomena of the world from nature to politics to political documents to logic and metaphysics. His logic, including his modal logic, is one of the great achievements of the ancient world, and it holds up still today, though Grice improved it with an interactive component. If, as I've posted elsewhere, that a necessary condition of science is its independence of any audience, Aristotle was an exemplary scientist, even though he was quite wrong in his explanations and weak on experimentation. Plato is no such scientist. His arguments are all biased, so biased that it's hard to read the dialogues without seeing through just how flawed the logic is -- at least, in those dialogues that purport to use logic. Many of the dialogues, like the Republic and the Timaeus, are scarcely argued at all, just expressing "Socrates" views to an audience of yes-boys. In the case of Timaeus, my favorite of the dialogues because it's so far from logic, the goal is mystical, spiritual and phantasmagorical. Not science. 

This distinction between nature and ideas flowered and blossomed in the middle ages in the debate over nominalism, the view that these categories are just names, the creatures of nature each having its own spiritual inner being. 

The debate takes its modern shape with Hume and Kant. Hume's skepticism reduced even causality to mere correlation and introduced the inductive fallacy, that no matter how regularly evidence places things into categories -- ideas -- the past cannot predict the future, so those categories are mere speculative chimeras. Kant responded with a kind of brilliant maneuver. Yes, the categories or properties and even causality, time and space are all merely mental concepts, not realities of the world in itself. They are the conditions under which humans understand the world as we perceive it. But these ideas are constitutive of that world we perceive and understand. Without those concepts we are blind, to use his metaphor, and without perceptions our concepts are empty. The two together create a world of phenomena, a world of human understanding. But that world is not the world as it is in itself independent of our ideas. The conditions of our understanding may not inhere in the noumenal world of the in-itself. 

Coupla points here: that relativity and quantum physics do not conform to Kant's Newtonian space and time is often used as a criticism of Kant. Quite the contrary, the tools of science probe the noumenal world, not just the world of our natural human understandings. When I wrote that scientific investigation is independent of any human audience, this is one aspect of it. The theory of relativity or of quantum superposition does not try to conform to what humans can perceive or conceptualize except in the terms of the science. Our current physics is consistent with Kant's division of phenomena (our native conditions of our understanding) and noumena (how things really are regardless of our native cognitions). 

The second point: this view that the world is a projection of the mind, not of the world itself has become standard fare of an older generation of YouTube influencers without attribution to Kant, as if these "geniuses" came up with it themselves. Kant's Critique of Pure Reason was once required reading for any college student. Seems to me these gerontic scammers have the temerity to pretend originality because they know that the university's push towards social relevance has produced a generation utterly unaware of the accomplishments of the past -- they know no one has read or maybe even heard of Kant. It's one reason why this blog is pseudonymous: I'm just too disgusted by these self-promoters to participate in their game. 

So Kant cobbled together two universes, one generated by our innate rational concepts -- our categories or properties, including space, time and causality, all the understandings of the world that Aristotle cherished as the ultimate goal of human understanding, the greatest Good -- and a world-in-itself, about which we know only through a glass darkly. Schopenhauer speculated that the noumenal world was a drive, a will, and that will generates everything in the noumenal world. Only forty years later Darwin's natural selection explanation supplanted that drive-theory with a truism of survival: whatever doesn't seek the means of survival, won't; survival implies will, including all the emotions and drives that conduce to survival. Why? because only those with such drives and emotions survive. It's a truism. Yet that truism explains everything from our emotions to our conceptual apparatus to our arms, hands legs, mouth, nose, eyes all the way down to your nether lip. It's all got an explanation. It's no wonder he recognized the religious wouldn't like the theory, though even as far back as St Augustine, it was evident that the scripture wasn't intended to explain that sort of stuff except as it were allegorically.

Move forward to the Twentieth Century. Scientists have discovered electromagnetism, quanta at the microscopic level, atoms and molecules, extraordinary and unprecedented discoveries and theoretical achievements. Scientists and philosophers of science, seeing a clear difference between their naturalistic and predictive theories and the theories like metaphysics and religions and the psychological theories spun by Freud and his followers, attempt to distinguish scientific theory from non scientific theory. Their first attempt runs along empirical grounds. Ideas must be verified with evidence. Angels? Show me one. The Absolute?  Please be more specific so I can find it in one place. You can see that this will lead to a behaviorist account of the world of phenomena. And it did, through a paradoxical route. Wittgenstein, in his later notebooks, applied this verificationism to the mind and found that it couldn't apply. The contents of one mind can't be verified by another. Do you see green the same way I see green? How could I verify it? I can see we both call it green, but that's just our behaviors, not our inner experience of green. So again we're stuck with two worlds, a world of behaviors we can see, use, and live in, and a noumenal world that we can't even coherently talk about. 

This behaviorism ruled in the philosophy of science until Chomsky's 1956 syntactic structures in which he showed that language can't be understood or generated as behavior alone, and it can't be learnt behaviorally. Its creativity requires an innate machine structure, and that machine is in the brain, a mental capacity. We can't explain linguistic phenomena without talking about, and analysing, the mind. we're back to rationalism. There is a mind, and it's responsible for our behaviors and we can talk about it and, importantly, analyse it through the concrete evidence of sentences generated by that mind. 

Chomsky was relying on the work of Alan Turing. What he showed in 1956 was that certain kinds of computational machines can produce human sentences, and cannot produce non human sentences so human brain seems to have the form of such a machine. Moreover, the non human sentences that humans can't produce or understand, if they are treated as mere strings of words rather than internally structured by a machine-generated grammar, look just like the sentences humans do produce and the machine also produces. In other words, a mind or parser learning solely from behaviors should produce the impossible strings. The mind or parser would have to have the machine structure to prevent it from producing those strings. As Chomsky says in a lecture, "language can't be learnt", meaning that human native languages can only be acquired by a machine that is structured already to accept certain kinds of strings and not others. Such internal structure cannot be acquired by a blank slate. (You can find the details in the post complexity and AI here or Yoneda learning.)

The Chomsky model of learning depends on the human having a kind of computer-like "machinery" in the head. 

Meanwhile, Claude Shannon developed a learning machine without any computational structure. It learns by behavioral error correction. And it's this behavioral-empirical learning that neural networks -- chatbots, LLMs -- use to learn what Chomsky seemed to show was unlearnable behaviorally. 

And so we see that the long debate over empirical-behaviorism versus innate rationalism is being played out again, but this time we have the technology to apply and implement it. 

Those who think AI is just a predictor of the next word should understand that the way it predicts is not merely using big data to statisically provide the most likely string of letters. It predicts based on a learning of each word's relations to all other words, modelling them altogether as a kind of world of meanings. It predicts based on those meanings, not on the frequency of that word following the previous ones in its training set. It assigns each word a distance and direction in relation to all the other words, in a vector space. It's modeling a world of text. From the relations of those texts emerge the meanings of their words in a world of text-meanings that is in many respects the real world of things as humans interpret them. Information is always just information. The models that humans construct with the information we derive from the world of things (along with our reading, our education, our cultural transmission) is comparable to what AI is learning -- a world of symbols associated with meanings all related to other symbols and their meanings. That's one way to describe the properties of intelligence and mind (see wikichip). 

the myth of the analog, the myth of introspection: notes on AI truth and honesty

A common complaint: LLMs have no concept of truth and therefore none of honesty or dishonesty. I think this is a peculiar kind of red herring. It seems to assume that humans have access to actual things in the world and therefore we have direct access to truths about the world (not just a concept of truth or a belief in a true statement) and that our experience is somehow more than mere information, whereas LLM information is mere letters and spaces.

Of course AIs, behaviorally, can be honest or dishonest. I think we've all had the experience of calling out an AI's mistake, and its response is recognition that it was wrong -- that what it said was untrue -- along with effusive apologies. The AI skeptic says, the AI doesn't mean what it says, since it merely produces strings of letters and spaces. 

That response misses two points, one about AIs and one about humans. AIs don't merely choose the next word based on a probability in the internet literature. The AI has already weighted every word -- possibly multiple weighted versions of each word associated with distinct contexts in which it occurs -- and placed it in a multidimensional vector space along with all the other words, relative to each other, with distances between each and spatial directions. In other words, the AI is modeling a world based on its vast text training set. Those weights + locations relative to other weighted words are meanings that are not merely letters. They could, for example, be used by a physical robot to manipulate the actual world of things in space and time. So they are meanings very much as human language meanings. 

That's one missed point. The other missed point is that human word learning does much the same. The only difference is that humans learn their meanings often from their interactions with the spatio-temporal world. In both cases -- humans and AI -- the intelligence is modeling the symbols by means of information from a source, either text or sense-based information. 

But information is always only information. Humans don't have a privileged analog access to truth or to the world of phenomena. The humans don't have analog objects in the skull or in their brains. The brain interprets the sense data as information that it models the world with. Just as an LLM does with text. 

The exception for humans is, of course, our inner world  of sensations, feelings, moods, emotions -- all of the constituents of consciousness. This has been well understood and accepted as pretty much obvious since Kant's first critique, which is why his name has survived, and Descartes before him, and a host of ancient South Asian philosophers as well. Those constituents of consciousness and consciousness itself -- those, we have special access. It's not even analog; it's immediate. The sensation of the color of a green leaf is the color, not an analog of the color of the leaf. Leaves don't have colors. They reflect light frequencies which we interpret as color. This immediate access might lead one to think that we have access to all of the physical world, as if our mind's interpretation of the world were the actual world, what Kant called the noumenal world of things-in-themselves. That's just a mistake. 

To be clear: we don't have special access to the frequencies; they come to us the way strings of text come to an LLM's neutral network. What we have special access to is the sensation of color. It would be a mistake to assume that because we have access to our awareness that we therefore have direct access to the rest of the physical world of phenomena. Our world outside the inner sensations is an interpretation constructed of sensory responses to our interaction with the world. It's not the world as it is. The world we see is our mind's interpretation -- the phenomenal world you see is your mind -- that worlds is your mind responding to the pressures of a real world that you can only access through that interpretation. 

How is this, as a source of truth or knowledge different from LLM information? If anything, the LLM has a more direct informational access since it doesn't have the sensory interpretation. For an AI, color talk is just more information about humans. It's the words humans use to talk about their reception of light frequencies. Humans give the frequencies names, and then they talk about the names. 

For sure there are distinctions in purpose and focus between AI modeling and humans modeling. Humans have evolved to survive with emotions and drives and fear of dangers, while the AI is not engineered to survive and replicate, so it doesn't learn through evolved emotions, drives and fears of danger. AI expressions of emotions have meaning, but the AI doesn't feel those emotions, whereas humans do have emotions which are not only real but influence its thinking and acting. Notice also that human language has no means to accurately describe the one aspect of experience that we have direct access to immediately -- our sensations (qualia in the philosophy of mind discourse) -- aside from naming them. We have a name for the color green, but can you describe what green looks like? Like a leaf. Okay, what does that color look like? Green. That's where words fail us, just where our access is immediate! That seems to me to be the clear evidence that the AI skeptic has missed the point that most of human language use is mediated by information just as AI is. That is, human and AI meanings of words are on all fours with each other. The difference arises only when it comes to the true immediate organic human traits that AIs don't have, and that human language ironically fails to describe. 

Another difference: humans think with tenacious biases; AI bias toward the norm, the frequent, is not tenacious -- a good prompt and it will abandon its initial "perspective". It is not emotionally committed. 

[For more on these differences and similarities: Yoneda learning, wikichip, complexity and AI, AI myths of human intelligence, the hierarchy of data, why AIs are not conscious and are not likely ever to be, AI introspection, limits of language, qualia and noumena]