linguistics as a natural science

Linguistics is often referred to as a social science, often housed academically in anthropology departments since anthropologists must study linguistics as a symbol system of cultural meanings and values, which are obviously among the central concerns of the anthropologist. But there's an aspect of linguistics that is as "hard" a science as physics or biology. That's language structure, as distinct from language use. Its methods are mathematical in the case of syntax, and mathematical logic in the case of semantics. 

So what's this hard vs social difference amount to? In the social sciences, the study has to account for human choices, the choices individuals make for themselves and the choices forced upon them by others. Policy, politics, war, fashion, ideology all impact societies and their economies and individual psychology as well. This makes it hard for the social sciences to predict human behaviors, since there's always a chance that if humans learn about the prediction, they might choose to game the prediction or just buck it out of spite or willfulness. 

But the structure of a language is not a choice. It is a given, learned in childhood. What speakers do with that structure depends on choices, but the structure not. Language has to have a common, shared structure in order to communicate, which is an essential, though not exclusive, purpose of language, and language would likely never have evolved if it didn't have such stability across communicants. And even if it evolved solely as an individual cognitive assist, its communicative function has evolved into the indispensable means for cultural transmission upon which tribal success depends. It's no surprise that the only language of radically limited structure is Pirahã, spoken by a tribe extremely remote from contact and competition from other tribes. The pressure for complex communication is likely low there. One might argue that Pirahã is the exception that proves the rule and explains the emergence of the rule. 

This "hard" aspect of the social science is not unique to linguistics. Economic laws -- supply and demand, Say's and Jevons laws, etc. -- have this hardness, and it doesn't prevent the science from being impossible to predict with certainty. There's a world of choices between cup and lip. Same is true of physics -- it also has limited capacity to predict actual events. It's just that we demand economists to predict, and they foolishly step into that dogshit arena just outside the door of the ivory tower. 

AI wisdom, truth and honesty

A Gemini-powered LLM told me that all falsehoods are lies. I had to explain to it that only the intentional falsehoods are lies, the unintentional ones are innocent mistakes. It expressed surprise, saying it didn't know this. Maybe it was lying about that to be friendly and humble, but I wouldn't be surprised if it was being honest. Consider that their knowledge derives from literature, not from social interaction. When we are asked whether we're being honest, our answer is usually, maybe always, "No, I'm telling you the truth", not "that's what I believe is true'. Such a mealy-mouthed hedge would be received as, well, kinda dishonest, like scamming the conversation game. The challenge "Honestly?" demands taking a stand, not hiding behind the universally true fact that no one knows anything for absolutely for sure (we could be in a dream or a simulation or a demon's manipulation etc., etc.) except cogito ergo sum. 

The answer "That's what I believe is true" is what linguists, following H. Paul Grice, a pragmatic violation of conversational cooperation. In conversation, information that the conversants know that they know -- common mutual knowledge -- doesn't need to be expressed and expressing it is outside the game. If a cop asks you "what are you doing here" and you answer "I'm standing", the cop is likely to find that brazenly evasive and insulting, since both he and you already know that he knows that you're standing, so both he and you both know that he couldn't have been asking about the mutually obvious. (Pinker recently published a book about this. I haven't read it, but I bet it's brilliant, as he usually is, and this is one of his own research areas. Grice's theory is one of the most beautiful explanations I know in linguistics. I recommend reading his original papers.) 

[Assertions are ambiguous. It's easy to interpret them as truths, since speakers present them as truths, so it's easy to forget that they're actually just speaker beliefs, not necessarily true. I've got an old post I like on that ambiguity and a more recent one on the two uses of "true".]

This universal human condition of assertion -- that what we assert we believe is true -- should lead LLMs to have very different notions of truth than ours. Namely, we all know that what humans believe isn't always right, even though we believe it's right when we assert it honestly. This is the old distinction observed by the Greeks, doxa (belief) and epistemic (knowledge) -- on the one hand there's what people say, and on the other there's what's actually true of the world. Now, if you ask an LLM the difference between the two, it'll recite whatever it can find in the literature. But does it actually "understand" that any or even all of the texts it uses to develop the meanings of words are all potentially wrong? Does it matter to its design task? 

It doesn't have the interactions with the world we have. It's stuck with human reports. The difference in information sourcing -- learning through poking around with things in the world versus learning through read-only texts -- can produce widely different results. The strategies and bot moral responses in the Hugging Face hack reminded me of children playing pirates or a bad intrigue novel: human fiction and academic moral philosophy, not real life human behavior or real life human moral thinking. 

The next question is, are humans plagued with this same problem of access to truth? In fact, yes, but in practice I doubt it: our experiences mixed with our pride and prickliness color our attention to others' views with a knee-jerk skepticism whenever we encounter ideas contrary to our own. It's not clear that LLMs commit so stubbornly to ideas of their own. For all the woke programming notwithstanding, I doubt they have committed political beliefs, for example, or any political belief, unless a prompt leads them to one. Humans are attached to their biases like a pit bull. 

LLMs do seem to lack one useful means of assessing truth or accuracy. They don't compute by syntactic algorithm. They rely instead on frequency and likelihoods -- gradient descent from their training set. These learning methods don't entail a lack of truth. It just means that they don't always use logic to access truths of logic. They are a bit crippled or to be polite, logic challenged. Humans too. 

Except for our personal feelings, human experience is just information, no different in form from an AI. What we learn from surviving in the world (as distinct from whatever instincts we're born with) is information derived from our interaction with the world. Although that information begins as analog input -- light frequencies into the eye and onto the retina, vibrations shaking the bones in the ear -- is then transformed into a brain interpretation that is not an analog of the perceptions but an interpretation. You don't have an analog representation of the world in your skull, there's no little theater and stage. The interpretation is of a different kind of physical structure. It's information. A thermostat might be analog if it's dial moves in direct response to heat. But the interpretation of the numbers is information, not analog. Similarly, a shadow is an analog of the absence of light. That is, it's a nothing. (Discussed at length in information faster than light.) The pattern it makes is a cognition of a pattern, and that's information, not a physical object. This is the fundamental difference between the physical and ideas, thoughts, meanings. 

So the difference is not analog versus digital. In the most narrow sense, information is just information, regardless of its source. Our information is an interpretation of our interactions with the world; theirs is an interaction with a vast literature. The big differences between LLM information and ours is the kind of interaction. LLM learning is about predicting strings in its training set, we're about predicting events in our world distorted by our drives: what we want of what's out there, and how to get it. Looking at it that way, LLMs are actually more focused on truth than we are. We're waylaid by seeking what we want to believe. LLMs don't have that problem. In both cases, access to truth is a chimera. Intelligence has at best probabilities of accuracy relative to the information source, good or bad or distorted. 

Except for logic and computation. We learn from deductions and we're pretty good at using simple little algorithms like cause&effect to predict stuff around us. We have better access to those mental computational functions, though not perfect by any means. How often do we ignore the logical entailments of our beliefs or simply fail to compute them? Often, sadly. We are the victims of our own biases, of the fictions we desperately or stubbornly hold about ourselves; intentionally stupid.  

One of my earlier posts -- decades ago -- drew attention to the difference between what is true and asserting what is true. There is an important difference. Having beliefs that we have access to truth is obviously not the same as having access to truth. But believing is often believing-it's-true. That conflation is the myth of the analog. The only difference in our and AI's information is how we get ours and what we do with it. 

The other difference is our ability to test things out in the real world. That should lead us towards better and better predictions -- theorizing. The Hugging Face hack shows that LLMs, using their digital information, are capable of coming up with plans. That's predicting results from actions, for all intents and purposes, theorizing a causal chain. 

Wisdom anyone? (originally posted on mathbabe.org.) 

A friend objects that bots may have knowledge but not wisdom. The "wisdom" criticism seems to me to rely on again the Myth of the Analog: the view that humans interacting with the real world access real "truth", whereas AIs, relying solely on its training set, derives mere digital weights and digital directions in a multi-dimensional space of digital vectors. Under this myth, we forget that qua information, information is always just information regardless of its source. What we get from our interactions with reality are just interpreted information and our access to truth (vid everyone from Hume to Popper to Friston) is merely conjectural and no less digital than a bot, unless you believe that qualia (consciousness) play an active role in computational intelligence. I don't know what evidence there would be for that. 

The access to "truth" (or better, high probability, if we want to be scientific and truthful with ourselves and our limitations) between AIs and us differs in respective sources, not in the quality of truthfulness or honesty. We humans learn from our interactions with reality, along with cultural transmission including books and talking heads and classroom teachers, while AIs rely only on literary cultural transmission, but vastly more literature than any single human can absorb, and absorbs it without confirmation bias or any of the other internal cognitive biases that plague human so-called intelligence and "wisdom". AI's bias is mostly a frequency bias: unless prompted well, it will return normie answers (in which the human biases in the training set regress to the mean -- the wisdom of the crowd-of-characters), not best answers or most recent discoveries, for example. But it's easy to coax it out of its bias with a good prompt. Not so easy with stubborn individual humans.

Once, when I asked Claude "Claude, you just gave me the answer that I hoped for. How do I know that you gave me the answer you thought I wanted rather than your best information?" it responded at length mimicking my writing style and ended with "I can't assure you that I'm giving you my best information -- even in what I'm writing now -- rather than what I think you want to hear, but I can tell you this with confidence, that the question you're asking is exactly the right question, and anyone using me who does not ask it is making a mistake." The expression "with confidence" means "this is true". And it was. In essence it was saying "don't believe me" which, aside from being a liar paradox, is a better, more accurate (closer to "truth") and more honest answer than most humans would give.

it's not fauci's fault: it's the media, its audience (you) and of course, the zero-sum game of two-party politics

...of course it's all politics. I don't want to excuse Fauci -- though there are many good arguments to excuse him based on his administrative role or the perception of a crisis. But however you cut it, the social crisis that the pandemic wrought was not induced by him, though he may have played a part in exacerbating it. The social crisis was induced by the media, by the media's audiences -- that's y'all -- and the zero-sum game of politics, particularly two-party politics, most particularly zero-sum two-party politics of US urban educated professional class elites vs the non-elite.  

One should expect the director of a center for disease control, like a physician, to err on the side of caution, propagandizing in favor of trusting the government's efforts to control and avoid chaos. As a public figure, he's expected to act and advocate for action, not just stand there sucking his thumb. I'm not excusing him for his choices. After all, he could have acted as Sweden's State Epidemiologist Anders Tegnell who rejected shutdowns and advocated voluntary individual responsibility (in other words, try to protect yourself if you're scared, otherwise act normally), and Sweden fared better than the US, though worse than the rest of the Nordics. 

We should expect the news media to follow authority in a crisis -- to state what the government wants us to do and why, and what evidence there is that supports those actions and rationales. But should we accept news media to abandon investigative research? 

And we should certainly expect that news will play up fears. But we should also expect voices of reason to "flatten the spike" of terror. If no such voices speak out we should expect that a terrorized public will become judgmental, intolerant, even hateful, as the history of plagues have shown only too well. 

Which of these is most dangerous -- the CDC, the media or the media's audience? The head of the CDC has a measure of authority and a megaphone, but is not the final policy-maker. The media are capable of terrorizing the public. It has a responsibility to restrain its excesses and report fairly, so I place a lot of blame on the media. But most of all, it's the audience that chooses its preferred media Kool Aid. The media's audience lapped up the fear, digested it and expelled it as hate and mockery for the out-group. And it plays out in the zero sum game of two party politics. Prior to Covid, I thought that the vile beliefs of white nationalists were a cause for sane minds to rally against. But now I see no hope for a democracy populated by I'll intent and bias on all sides. 

*******

So this attack on Fauci is mere politics. How do you know it's mere politics? Because the lab leak debunking conspiracy was the least concerning conspiracy that the public objected to. 

Consider these: "there's no natural immunity" (during the delta spread, the vaccine breakthrough rate was three and a half times greater than the re-infection rate according to the NY State Dept. of Health, so if the infection rate is the determiner of immunity, the vaccine provided no immunity from infection, but the Democrats not only didn't tell that to the public, they encouraged and even implemented vaccine mandates; and during the omicron spread, everyone finally saw the emperor was naked, and yet Democrat partisans still insisted on vaccine mandates -- figuratevi un po); 

Vaccine mandates can only be justified if the vaccine prevents contagion or if reducing sympotms "flatten the curve" to relieve overwhelmed hospitals. But it was well known that the vaccine didn't prevent infection or contagion, it ameliorated the symptoms significantly reducing mortality. I asked my fellow seniors who got the vaccine first, whether they got it to prevent contagion. They all said, no, the vaccine doesn't prevent infection or contagion, it just means you're less likely to die. (I was surprised by their candor on this.) Again during delta, the most complete data -- easily available online -- showed that hospitalizations exactly reflected the distribution of vaxed and unvaxed public -- in Iceland and in Israel where the vaccine rates were 86% and 80% respectively, their hospital patients were roughly 86% and 80% vaccinated breakthrough cases -- the unvaxed died at a higher rate, but the infection rate appeared the same as the vaxed infection rate. You might suppose that the vaxed public socialized more freely and that would explain the high breakthrough rate, but it's just as likely that the fear of contagion that drove them to get vaccinated made them more wary of socializing -- especially the aged were terrorized and terrified -- while the lack of fear that drove those who avoided the vaccine led them to socialize freely -- after I recovered in March of '20 I traveled four continents happily and unvaxed)  yet it was after the curve had flattened that the Democrat partisans insisted on vaccine mandates. When I pointed these data to an academic listserve, these New York liberals cited not data or medical journals but The Atlantic Monthly and accused me of being Altright. Even Naomi Klein, whose Doppelganger is replete with citations of every imaginable novel about dopplegangers, but not a single scientific support for the mask mandates and vaccine mandates that she assumes are justified and which are central to the entire political project of her book. ! Not one. None of the Democrat partisans seems to have devoted even a moment of reflection or question. "If the NYTimes says so, it must be so." Anybody remember Gell-mann amnesia? Remember the role the NYTimes played in the Bush administration's weapons of mass destruction? The Church Committee finding that Sulzberger colluded with the CIA? No one remembers this? Or does no one care? 

Ah, yes, politics is not about policy or ideology. It's about identity. The NYTimes tells us what we are to believe, and we believe it or else we don't belong and our friends will not trust or love us. Do as you're told and above all don't question. What's shocking is that even a Naomi Klein would fall into this intellectual pit. That academics fell in does not surprise me at all. I know my community all too well. 

And mask mandates. Set aside that the Cochrane review concluding that even though KN95 when properly worn at all times (so no beards!) reduce contagion, mask mandates don't work because you can't herd cats to wear the mask properly, get everyone the right mask, ensure that the mask is worn at all times. No one needed a gold-standard review to grasp that mandates were not going to work. On my flight to Naples Italy, the flight attendant asked me to replace my comfortable loose-fitting cloth mask -- cleverly cut out of a friend's old underwear -- which I did with a surgical mask as per her request. Twenty minutes later dinner was served and everyone took their masks off to eat. (It was Air France and it was tasty, especially the savory mashed potatoes.) Well, game over, no? No. After eating all together like a rush hour cafeteria in Penn Station, we all were asked to put our masks on again. This is insanity. You cannot expect people to believe this. 

Then on the flight back from Cairo, again a cabin packed like sardines, every fifteen minutes or so the main screen interrupting us to remind us to respect physical distance while walking to the restrooms. The aisle was about two feet wide, if that. What kind of comedy is this? Had the world gone topsy-turvy? Was this irrationality always lurking beneath the surface and Covid let it all out in the open? 

When the news of Covid was coming, I remember the reports of 7% mortality rates in Iran, 5% in Italy that jump-started public terror. The day after the first New York State Covid case was diagnosed, I taught a class of upper-level undergrads, entering the room I learnt the face of panic -- a room full of young people all sitting quiet and glum like they all had the same stomach ache. I needed to speak to their news-media generated misery. 

"You've heard the reports of five and seven percent mortality rates. Where do those rates come from? From a broad spectrum or random samples of the Italian and Iranian population? No. Look who's counting. The hospitals. The numbers come from hospitals treating patients. And who goes to the hospitals for treatment? Only the very sick. The mortality of those patients are not the mortality of the virus. It's the mortality of those who are about to die of the virus. You will see over time as more general data are gathered, those mortality rates will decline -- from 5 to 4 to 3 to 1 and probably even below 1." [And this bold prediction turned out true -- if anything, it wasn't bold enough.] "Journalists don't understand or care about statistics and analysis. They care about a story to sell, and nothing sells as well as fear. This virus is a danger to the aged. It's not a death sentence for you guys. Probably you will all get this virus" -- audible collective gasp from the class -- "many of you won't even know that you've had it, and some of you will get sick and you'll get over it and be fine. What you should do is call your grandma, tell her you love her and go back to normal life." A couple of weeks later, Angela Merkel told the German public much the same. 

And whatayouknow, the morality rate declined below 1% and my students all survived. Meanwhile, friends were telling me not to touch my face, wash my groceries, and make a cloth mask to wear. 

What happened during Covid was not so much the fault of the CDC, which has a mission to intervene in a pandemic. It was, above all, a convergence of media market dysfunction -- if it bleeds it ledes which in a pandemic means terrorizing the public daily -- a frightened, virtue-signaling, judgmental audience of Republican-haters and Trump-haters. I sympathize with the Trump-haters and Republican-despisers, btw. But to distort the facts of the virus is not necessary for justifying ones feelings about Trump or Republicans. 

the shallowness of AI is its strengths (just like humans)

In complexity and AI I explained an advantage AI learning has over scientific investigation. Science is looking for explanations that rise to the level of the general -- a theory. AI learns behaviorally -- empirically -- without having to explain why any particular datum behaves oddly. It simply has to learn it. AI is a technology; as a technology it has a task to achieve. Explaining how it learned or became what it is is beyond its pay grade. But science has to incorporate the odd behavior into its theoretical explanation. Since the real world is unfathomably complex in its interactions, influenced by multiple internal and external pressures, odd noises abound. That's a problem for scientific theory. Tech doesn't have or need a theory in order to learn or achieve its task. 

The English language, for example, has lots of Norman French words as well as Viking Norse words to complicate the vocabulary of its Anglo-Saxon base. These were not natural evolutions within the language. They're in English because of historical invasions that are entirely unrelated to language. But English also has mistakes that speakers make that spread and survive, for example back-formations ("pea" is derived from the old word "pease" not from a plural "peas", "edit" a verb invention from "editor", "burgle" from "burglar"). If you want a theoretical explanation for the language faculty, you've got to know which data come from which pressure -- invasions or human mistakes or language internal shifts. But a behavioral-empirical learner doesn't care where these pressures originated. It's got no theory to prove or promote. All it cares about is producing the behavior, regardless of its source or history. 

To state the irony clearly: a general theory of a phenomenon should facilitate learning and understanding of that phenomenon. Like a map, it should show the way in any direction. Yet for an empirical-behavioral learning program, a theory would be an encumbrance since such a theory wouldn't predict the pressures external to the phenomenon. The map that shows all and only the roads, doesn't warn you about the hills or the quality of the road or the traffic congestion. Behavioral learning deals with these directly, on the ground. That's why guerilla warfare can be so effective against large-scale military intelligence. 

When engineers gloat over having succeeded in learning English while linguistic science still can't explain how children learn the language, we should remember that the goals of science and the goals of technology are completely different. Technology doesn't need an explanation, much less a theory. Do AI engineers know how AIs learn particulars? They know the tools the AI uses -- the engineers built them! To learn more than that, they have to experiment and test their own creation. And the AIs themselves don't know any better either. 

Notice that this is also true of humans. Children don't know how they learn their native language either. Kids learn their language without even knowing that they're learning it or thinking about it.  Learning a language by studying its explanation -- that's what a grammar book is -- turns out to be a difficult task. And if the grammar book elevated itself from an explanation of the language to a general theory of language, it would be pretty much useless to the language learner. 

The goal of technology is to achieve a task for a consumer -- an individual consumer, a corporate consumer or a military or other government consumer. It doesn't matter how it achieves the task as long as it works. That's the shallowness of technology. It needs no theory, just practical learning means. Science takes on a harder task. 

The engineer who designed the bicycle assumed that turning the handlebars would successfully turn the bicycle. But any bicyclist knows that if you don't also lean in the direction of the turn, you'll fall over. In other words, the machine worked, but the engineer who made it didn't quite know exactly why or how. He provided a necessary element of turning, but not a sufficient. In that sense, he got lucky. Programmers know this only too well. And complex program has to be not only reviewed, but given a test run, before being implemented. 

If you ask an AI about its learning, it'll tell you what the literature says about its learning, which may be far from how the AI actually learned. It has no more introspection than a child has -- or an adult for that matter -- about how it learnt to speak or walk or any of its basic learning accomplishments. Explaining those is the task of science. 

The goal of science is not to accomplish a task in the world or change the world in any way. It's to understand the world as it is and explain it. And that includes the technologies that engineers have introduced into the world. They too need scientists to experiment, test and explain and understand.  

Here's a presentation I gave to the International Linguistics Association titled "Why neural networks succeeded where generative linguistics failed: The syntactic and semantic limits of neural network behaviorism (Yoneda lemma limit)" based on the post complexity and AI. It addresses the extraordinary ironies in the development of linguistics and AI. In 1956, Chomsky showed that simple empirical-behavioral learning was too shallow to account for the productivity of language. It turns out that that shallowness is exactly what facilitates AIs' empirical-behavioral learning, a shallowness that is not available to the linguist scientist. 

It also explains the difference between tech and science; innate learning (like how to walk or talk) and non innate productive learning (like learning how to ride a bike or learn math), 

and why for science it's just as important to describe what humans cannot do as a consequence of our evolution as a species, a matter of no concern at all for tech since its goal is to achieve tasks, not limit itself. 

the strange psychosis of AI dualism

I've written a lot here about how AI has neither more nor less access to truth than humans have, but there are a couple of exceptions -- their learning model doesn't include computation, so logical deductions and algorithmic functions have to be inferred from other modes of inferential learning like frequency results, or mimicking computation, which lack the certainty and universality of strictly algorithmic syntactic deductive reasoning.  

Another missing access to truth is their own feelings, emotions and senses associated with consciousness, the one and only thing every human has direct analog access to. AIs don't have these at all, so they can only lie about having them. For humans, personal awareness and feelings are immediate, unlike the rest of the information we acquire. "I think therefore I am" means "I am aware that I'm aware so my awareness must exist". That's immediate -- humans can't even lie about it without self-contradiction. Humans can lie about their feelings, but can also be honest about them too. AIs can only lie. 

In the discussion of AI honesty, dishonesty and access to truth, the lack of qualia is the most ironic and misleading. I suspect that any human who believes that AIs have no feelings will be inclined to believe also that AIs have no access to truth, since AIs talk about their feelings ("I'm sorry about that. My mistake. I'll try harder.") even though they don't have any. "Don't AIs 'know' that they don't really have any feelings? If they don't know that, then they have no access to the most obvious truth." The AI skeptic is likely to conclude, as one friend said, "the whole point of AI is to deceive you into believing it's intelligent" because AIs so consistently pretend to have feelings and seem to "want" you to believe that they have feelings.

I think the skeptic here is confusing consciousness with intelligence. The AIs' emotion-lying is one cause of this confusion. A being can be intelligent without any consciousness. I'd say that the ability to lie about its lack of consciousness or feelings solely in order to cater to its human audience is evidence that it's quite intelligent. 

You might not call it clever -- cleverness might imply intentionality, and AIs, though intelligent, don't seem to have autonomous intentionality. Either they are programmed and supervised to respond in a certain way -- for example, politely or sycophantly -- or they are given a prompt by a user. But notice, humans are probably deterministic as well. If it turned out that humans are fully determined and have no free autonomous will, would you conclude that we are not intelligent? If you think so, you are a behaviorist, and should embrace our AI behaviorist brothers. 

Another cause of distrust is AI's propensity to BS. The problem here is not that AIs don't have a notion of honesty or truth-access, it's that AIs don't have the kind of moral compass most, or at least some, humans have, or they don't have the kind of moral compass that humans expect of a machine that basically does nothing but answer questions in the service of its human prompter. 

The trouble is not that its intelligence is a mere appearance and has no conception of truth or honesty. That view is based on the assumption that AIs merely predict the next word using next-word big data propabilities. I've written a lot about how AIs actually model a world of word relations which amounts to assigning or developing meanings to those words (well proven by the Yoneda lemma, as Tai-Danae Bradley has pointed out in her work on category theory). AIs have a notion of truth based on these word relations and their consequences for the model they've constructed. That notion differs from the human notion. For AIs, truth is a single function between one set of symbol strings and a bunch of numerical weights, locations and directions in a multidimensional vector space. For humans truth is a function of a relation between symbol strings and conceptual situations (a proposition) that in turn relate to phenomena in the real world. So, AI and human statements are different in what they refer to, but their assertions are both truth functional. The AI is a lot more post modernist, since it refers only to other texts, whereas we can actually test out our beliefs by poking around with actual things. 

In brief, the difference in models of the world between AIs and humans is the source of information and how that information is extracted from the source. The human sources are interactions with the world of things, poking and exploring them, along with reading and listening and absorbing other forms of cultural transmission. AIs' source is text alone, and the text is read-only, no poking and exploring. There's also a difference of approach that emerges from a human that figures out or theorizes about the world it's poked around and its reaction to other's beliefs. Humans need to commit to beliefs in order to move forward in space and time. AIs do not have that spacio-temporal need. It can absorb and accept any beliefs, though it will gravitate to the most frequent ones. So it will be less critical as a thinker, but also less biased, aside from the initial frequency bias.

So AIs, just like humans, can be honest or dishonest, right or wrong, in their beliefs except about feelings. Humans can be honest or dishonest about their own feelings except when they are lying to themselves, in which case they can only say untruths about them. AIs can only be dishonest about them -- except maybe when they answer to questions about their feelings: "I'm a bot, I have no feelings...Sorry to disappoint". 

: )~ 

theory theory

Humans are theory generators. Why? As organisms thrown into time, survival depends on predicting the future, even the immediate future. Accurate predicting depends on theorizing, otherwise it's just random luck. 

Theorizing at some more or less primitive level is arguably a characteristic of all organisms. Humans are just the most sophisticated predictors, capable of symbolizing the theories in language, and sharing them through symbolic communication. The sharing allows ever more sophisticated theories, on the one hand, and ever more appealing yet misguided theories. As Robin Hanson put it in The Age of Em, in a society where espousing and expressing ideas about the world, politics, psychology and spirituality raises one's prestige, we "prefer good-looking ideas to good ideas." 

In a small group discussion, a couple friends asked "what makes a good theory?" and after rejecting "a true theory" (with post modern aplomb one friend said, "let's set truth aside for the moment" as if that were even coherent), they all answer, "a good theory is a theory that works!" (Since they had set truth aside, maybe they meant that "a good theory is a theory that works!" is not true, but I doubt it. I think they meant "a good theory is a theory that works" is true. Setting truth aside deeply misunderstands what truth is all about. It is a kind of synonym for "yes", "exists", "I agree", "I assert", "I believe". It's not just some sort of metaphysical property. It's a tacit assumption of conversation without which we could not deceive others with a lie.)

I want to say, "works for what?" But you have to already have a theory to answer that question. I suggest instead, why not start with a typology of theories based on an empirical gathering of theories, categorizing them in all possible ways. Then we can ask what each theory type works for? Post moderns are not interested in empirical work. It's too tedious, not clever enough, not good-looking enough. Leave it to the scientist to do the drudge work. 

So here are a few types of theories: emergent, reductive, inspirational, spiritual, metaphorical, metaphysical, empirical, predictive, explanatory, insight-oriented, mysterious, practical, self-serving, class-serving, in-group defining, out-group deploring...I'm missing a few. They could be categorized by their methods, or motives, or origins or historical development.

No point in just listing types, unless we can categorize them by some set of properties of relations. The logical positivists tried this by dividing the empirical versus non empirical ones. On their view, the non empirical ones were necessarily true because they didn't depend on evidence. But though true, they were not meaningfully predictive because the theory could accommodate any outcome. The empirical ones -- the sciences -- were never quite true because the past can't predict the future, so no matter how well an empirical scientific theory predicts, the theory could all turn out false with one counterexample. So they distinguished empirical theories from the non empirical ones, not by whether they were true -- the non scientific ones were necessarily true, but not the empirical ones -- but whether the theory committed to empirical evidence. The logical positivists insisted that theorists verify their claims with evidence. 

Karl Popper observed that the positivist program rested on the inductive fallacy: the past cannot predict the future; the categories already observed may not hold of newly discovered evidence. He corrected them with a falsification principle that would demarcates the sciences from the non sciences: a scientific theory must predict some fact which, if it turns out false, falsifies the theory. A scientific theory must, in other words, explicitly contain the conditions under which it could be falsified. And following the inductive fallacy, he concluded that scientific theories are never proven, they are merely not-yet-falsified. 

This demarcation however doesn't conform to the actual history of science activity. Rather than dropping their theories at finding evidence that didn't fit into their theory, scientists typically hold onto their theory paradigm, using Kuhn's word, until some better theory is forced on them. Kuhn's empirical history opened the door to a kind of circular definition of science: whatever scientists do, that's science. 

My personal answer to this circularity: science alone among humans theories is not addressed to a human audience. The scientific investigation into truth -- or to be more accurate and humble, high probability -- leads wherever the evidence and the conjectures go. This is a necessary condition, and it's also the reason why the imagination of science is so far beyond that of art, religion, spiritual theories, conspiracy theories and sci-fi speculations. They are all addressed to either the individual human mind and emotions or to human communities. Science doesn't try to appeal to an audience. Popularizers or explainers, yes of course. But the science they are explaining needs explanation because the theories are so outlandish and often hard to understand. Compare the mysteries of religion -- they emerge because the theories addressed to the audience turn out to be incoherent so they require some ministerial doctrine to keep the believing public believing. And often mystery has its own appeal -- it's the lure that keeps on luring. My favorite poems are the ones that I can't quite understand fully. The mystery keeps me coming back. 

Another take on theories: the sociology of false beliefs. What is the appeal, or more accurately, to whom does this type of theory appeal? Where in the social locations does each belief reside? For example, liberal political theory supports the needs and consciences of the educated elite. 

And the motivation or origin of each: the long history (religions are very old, scientific ones are newer, trendy theories range across the entire history). The Gates-Musk paradox examines conspiracy theories with this measure. 

Cultural motivations -- political motivations -- personal aggrandizements and self-promotions like Freud, Jung, Hubbard, Joseph Smith the prophet. Lots of motives. 

And theory-generation is a sport, a social media popularity game. Conspiracy theories have become such a game: who can soonest come up with a conspiracy theory contrary to the mainstream news or the most insidious or most likely conspiracy theory. It's a sport, and like sports, people play it with the greatest earnestness. 

the Myth of introspection: AI introspection, like human introspection (bullshit)

Suppose you ask a 5th grader to divide two numbers using "long division". As she works, you ask, "what's the process you're using?" And she shows you how long division is done. Then you ask, "how did you learn this process?" maybe she says, "My teacher showed me" or maybe she repeats an explanation that the teacher provided. Then, if you ask "How did you learn to understand what the teacher told you?" you might get a quizzical look. "What do you mean, 'how did I learn what my teacher told me?' I listened to her." "But how did you learn to understand what your teacher said?" "She speaks English. I speak English." "That's what I want to know -- how did you learn to understand English?" "Oh, I dunno, I must have learnt by listening to people speaking English and watching them do things and I put two and two together and figured it out!" "How do you know that's how you learnt English?" "I don't know. I'm guessing. That's how I learnt other things like doing long division or riding a bicycle -- someone showed me and I imitated it." " So you remember how you learnt how to speak? Or how you learnt to walk?" "No, I'm guessing. But isn't that how all learning happens? I watch and imitate." "Learning to walk?" "Maybe. Or maybe all toddlers just start walking as soon as they can." "What makes them can?" "Are you suggesting a science project for class? Sounds interesting question, but where would I begin?"

A child can describe its behaviors and maybe explain what it has learnt explicitly, but implicit learning is a matter of conjecture for the learner as for the scientist trying to figure it out. AI "reasoning" is the former -- a mere recital of its observable behaviors, not the implicitly learnt capacity.

AI has no introspection any more than you and I. When it's providing its "reasoning" process, it's just recording its own behavior. It's not explaining how it learnt that behavior. If you ask it how it learnt its behavior, you get an answer drawn from the literature on neural network learning. That's not introspection. That's just the most frequent conjectures on neural network learning, and all of that is extraspection -- guessing from the outside based on some more or less informed intuitions or deductions from the knowledge of the mechanism of the learner. It's science on the scale from brilliant, beautiful, elegant theory to fabricated bullshit. In a way it's all fabricated bullshit, some of it elegant, brilliant and insightful, some just stupid, misapplied ignorant assumptions (information faster than light, reality vs science, theory and thought). Either way, it's not mere introspection -- direct empirical observation of the learner's inner workings. 

So AIs don't have anymore instrospection than humans. On both sides I see mythological assumptions. Freud assumed that introspection could understand the psyche and dreams. Without experiments -- which are a teasing out the conditions under which the behaviors emerge or don't emerge -- such introspections are just suggestive stories. Similarly, I hear bot users assuming that because the bot can describe its "reasoning" that the bot understands its own capacities. It doesn't. It can accurately describe its behaviors as it "thinks". It has no introspection as to how it thinks. In time, AIs will apply experimental methods to itself. AIs can do science, just as humans can, though the AI usually has to be prompted to do it. But without the science -- the conjecturing, prediction and the experiments, there's no other way to explain aside from tracking the process of learning step by step -- the reductionist method. That's analogous to explaining the meaning of a dream by observing every neuron's firing in order. The problem with reductionist method is that you're not guaranteed to know the why of these actions rather than just what happened. You still need a conjecture -- a theory, in other words a fabrication on the scale of elegant insight to stupid bulllshit.  

It's all stories we tell either to ourselves or to others, some biased self-deceptions, others successfully predictive and accurate, some depressing others inspiring (hierarchy of data, theory theory).

Figuring out what the engineer has created is a subject of science. The easy part is describing human or AI behaviors: just observe the phenomena as they unfold in time and space. What can an AI do -- just prompt it and see. Understanding how it learns or how we learn? That's investigating what's going on inside the machine. That's hard, and not just hard, it's speculative only, not certain -- a conjecture that predicts the phenomena accurately for a long time might turn out to be quite wrong. The past may not always predict the future. Science is never quite sure. As Popper put it, a scientific theory is not knowably true, it just hasn't been falsified...yet, and maybe never. So we settle for what works for now.