The Intelligence Curse - I of III

In recent years, the concept of universal basic income (UBI) has gained significant attention, not from grassroots community organizations but from some of the most powerful figures in the technology sector. Prominent advocates like Elon Musk and Sam Altman argue that UBI is necessary to address the economic disruptions caused by artificial intelligence (AI) and automation. They present UBI as a way to ensure that the benefits of AI are distributed across society, not just concentrated in the hands of a

The Intelligence Curse by Luke Drago and Rudolf Laine is an AI-focused article about how advanced artificial intelligence could change economic incentives in ways that reduce the importance of ordinary humans in society. The article is not about individuals becoming too smart, but about intelligence becoming a fully substitutable economic resource. Once AI systems can perform most or all economically valuable tasks better and cheaper than humans, the importance of human labor begins to disappear.

Today, governments and corporations depend on people. States need citizens for tax revenue, economic productivity, and political legitimacy; companies need workers and consumers to generate profits. Because of this dependence, institutions have to invest in people by various means — through education, infrastructure, wages, and welfare. This mutual dependence gives ordinary individuals bargaining power and drives both democracy and capitalism.

The "intelligence curse" emerges when this dependence breaks down. With sufficiently advanced AI, powerful actors — states, corporations, and AI labs — can generate wealth directly from artificial intelligence rather than human labor. As a result, people cease to be economically necessary. This has several cascading consequences: 

1. Mass automation progressively eliminates jobs, not just for routine workers but eventually for highly skilled professionals as well. This process, described as “pyramid replacement,” hollows out organizations from the bottom up until even elite talent is displaced.
2. Economic power shifts away from labor toward capital and control over AI systems. Ownership of compute, data, infrastructure, and AI models becomes far more important than human effort. Those who already possess capital gain a permanent advantage, while social mobility declines sharply.
3. Institutions lose their incentive to invest in people. If humans no longer contribute meaningfully to production or revenue, funding education, public goods, or welfare ceases to have a clear return on investment. 

This mirrors the “resource curse” in economics, where states rich in natural resources neglect their populations because they no longer rely on taxation. The illustrative case is Congo whose economic history is one of lucky breaks leading to great misery. There is no other country in the world as fortunate as Congo in terms of its natural wealth. But not a drop of the fabulous profits trickled down to the larger part of the population. Rather, they have often served as a slave labor force for the extraction of those resources at minimum cost and maximum suffering. 

The "intelligence curse "is a more extreme version: AI replaces almost all of labor. The result is a potential breakdown of the modern social contract: Democracies and market economies historically align the interests of powerful actors with the well-being of citizens, because human productivity drives growth. But in a post-AI - world, this alignment disappears. Power could concentrate in the hands of those controlling AI, leading to entrenched inequality, reduced freedom, and the marginalization of most people.

The article warns that if intelligence becomes independent of humans, society may no longer be organized around human flourishing. To quote economist Erik Brynjolfsson, who runs the digital economy lab at Stanford University: In this world, most of us “would depend precariously on the decisions of those in control of the technology.” Society would risk “being trapped in an equilibrium where those without power have no way to improve their outcomes”.

Technophile hallucinations - III of III

A new class of tech billionaires hopes that cryptocurrency will upend the global financial system, replace democracy and even reshape nation-states. Justin Sun is a Chinese crypto-billionaire. He is worth eight and a half billion dollars. He's perhaps most known for eating a six and a half million dollar banana. Sun shelled out $6.2 million for Maurizio Cattelan’s Comedian, the notorious artwork that consists of a banana duct-taped to a wall, at Sotheby’s.  He then ate the banana at a high-profile press conference.

He made his wealth through the crypto platform Tron, which is one of the biggest crypto platforms in the world. He's also a 'politician'. Justin Sun believes, like many other tech billionaires believe, that democracy is outdated and can be replaced with the blockchain, the technology that underwrites cryptocurrency. So laws, voting, governance, could all be transactions that are automatically undertaken via the computer code that makes up the blockchain. 

That is how Justin Sun was elected the prime minister of a small libertarian micronation called Liberland, which exists on a disputed spit of land between Serbia and Croatia in the middle of the Danube River. Liberland is tiny (2.7 sq. Miles) unrecognized by any country, but it has about 200 companies registered. The president, Vít Jedliča, founded it because he's a tech libertarian. He said that it was the first decentralized autonomous government and also the freest country in the world. There are no taxes in Liberland

There are a few permanent settlers on that spit of land, but the main 'citizens' of Liberland struggle to visit it because there are conditions for visiting Liberland from the Croatian side. In fact, the Croatian police have often gone in and destroyed their encampment a few times. This is the problem with starting a tiny new country, you sometimes have to contend with proper bigger countries with police and military.

But Liberland exists more on the blockchain than it does in the physical world. That spit of land is what's called a physical node. A lot of these tech billionaires think that we will all live in digital countries in the future. But how was Justin Sun elected on the blockchain? He was elected through a crypto token called Liberland Merits. The elections are held every three months, they take place on blockchain, and the person is elected with Merits. 

People who have more Merits are able to have more say in who is going to be in the leadership of the country. What that means is that the more Liberland Merits you buy, the more voting power you have i.e. the wealthier you are, the more influence you have on how your home is governed (which is arguably the case already in many democracies). It's just that in this case, it's literally written into the constitution and the computer code of this country. So it's not one person, one vote. It's one million dollars, one million votes.

The idea that technology could basically replace all of government, laws, the banking system, is one that everyone seems very excited about in Silicon Valley. They see themselves as the ones with the real power defining the future. And they would like to replace what they see as inefficient, bureaucratic, human-led systems of government with a technology based one in which the individual's freedom is paramount. And no institution, bank or government can take your money or your rights from you. 

There is a guy called Curtis Yarvin who is an American far-right political blogger and software developer known for founding the anti-egalitarian and anti-democratic philosophical movement known as the Dark Enlightenment or neo-reactionary movement (NRx),  The amount of money that he has is not actually very high. He is someone who has garnered the respect of Peter Thiel, one of the richest people in the world. 

He thinks that democracy is outdated and that we should replace it with something more akin to a corporate monarchy where the tech bros are actually our kings. It centers around the notion that democracy has failed, it's inefficient and therefore, what we really need is more like an enlightened monarch or as he calls it, a kind of CEO king. And that society in the future should be run as a patchwork of almost city states where each one of them is actually more like a corporation than a country. It has customers rather than citizens. 

They compete for citizens like a company would compete for customers. You could switch countries like you might switch club memberships if one offers something more than the other one. These companies would be run by hereditary monarchs who are held accountable by a kind of shareholder aristocracy who has shares in the country. Yarvin believes this would be a better way to run society because corporations are more successful than governments. He wants to replace current elites with another group of elites who conveniently enough for tech billionaires are tech billionaires.

But what does that mean for the rank and file people like you and me who are not tech billionaires in Curtis Yarvin's society? He would say that the masses often do not know what they want or what they want isn't enacted by democracy. Someone who might know what you want better than you is an enlightened CEO monarch. Someone who's smarter than you. Someone like a tech billionaire.

Two of J. D. Vance's big backers, Thiel and Yavin, are people who  don't really believe in politics. They want to get rid of politicians. Their view is that they are the geniuses of the world and they  should be allowed to run things as they please; that a country needs a startup guy, a visionary leader like Napoleon or Lenin, who should have absolute power, dismantle the old regime and build something new. Thiel doesn't see Trump as the monarch, he sees Vance as the monarch. Yavin wants one politician to run the country.

The ideas that you find often lead to wealth and power flowing directly into the hands of the people who are already the wealthiest and who control the technology. Their idea is that money will be replaced by crypto, your job will be replaced by AI, and you'll essentially live on some form of universal basic income. Reading about such people gives you an idea of how far removed from reality the wealthiest, and in some ways, smartest people are. They really do believe that their brains are superior, and that they should and could run the world better. They believe that they really are accomplishing the most incredible thing in history. 

Technophile hallucinations - II of III

Elon Musk has been tweeting for years about the importance of going to Mars and beyond to save humanity. “The true battle is: Extinctionists who want a holocaust for all of humanity, versus Expansionists who want to reach the stars and understand the Universe,” he wrote. “Elon’s concept that SpaceX is on this mission to go to Mars as fast as possible and save humanity permeates every part of the company,” says Tom Moline, a former SpaceX engineer. “The company justifies casting aside anything that could stand in the way of accomplishing that goal, including worker safety.”

Ray Kurzweil is an American computer scientist involved in fields such as optical character recognition (OCR), text-to-speech synthesis and speech recognition technology. In December 2012, Google hired Kurzweil in a full-time position to "work on new projects involving machine learning and language processing". Beginning in 2025, he has served as Chief AI Officer for Beyond Imagination, a humanoid robotics company he co-founded in 2013. 

By 2045, he says, “we will multiply our effective intelligence a billion fold by merging with the.  intelligence we have created” — an event Kurzweil and others call “the Singularity.” He claims that  we won’t experience 100 years of progress in the 21st century — it will be more like 20,000 years of progress. He feels that by 2029, we will have reversed the aging process so that you will actually go backwards in time [i.e., get younger as time passes].

He feels that if he dies by 2029, it “would be a setback,”  but something can still be done. He says that he’d just leave instructions for the post-Singularity robots to sort through his belongings and scan his frozen corpse and bring a version of himself back to life. To this end, Kurzweil has joined the Alcor Life Extension Foundation, a cryonics company. After his death, he has a plan to have cryoprotectants, vitrified in liquid nitrogen, passed through his veins and stored at an Alcor facility till future medical science can revive him. 

Probably the most influential tech centered ideology that few of you have ever heard about is longtermism. They assert that there could be so many digital people living in vast computer simulations millions or billions of years in the future that one of our most important moral obligations today is to take actions that ensure as many of these digital people come into existence as possible.

They believe that Artificial General Intelligence (AGI) presents an incalculably greater threat than even climate change. Many of them that the greatest long-term human benefit is to invest more heavily in AGI risk and AI safety research. Yet it is not the safety of living humans today, or even the next generation, that drives longtermists. Their argument is that AGI in the far future could theoretically kill even more of the digital people than the people who are alive on the planet now. Preventing future AGI from doing this is a more rational use of resources than funding clean energy tech, food security, or public health today. 

Related claims made by some longtermists include the idea that saving the future requires directing even more of today’s resources toward the wealthy in the Global North, rather than sending aid to the most impoverished countries in the Global South, since wealthy people in already disproportionately wealthy regions are better positioned to use their resources to fight these 'existential' risks from AGI. 

If you think these are fringe ideas of a few lunatics, think again. Longtermists have directly influenced reports from the secretary-general of the United Nations; a longtermist is currently running the RAND Corporation; they have the ears of billionaires like Musk.  He has explicitly said that longtermism “is a close match for my philosophy”. (Now you can guess why he destroyed USAID.) It has a huge following in the tech sector. It is the language of moral thought spoken in a growing number of the wealthiest and most powerful political and industrial circles. Billionaires like Jaan Tallinn (an Estonian computer programmer and investor who helped in the development of Skype) and Peter Thiel have been described as avid supporters. 

A longtermist and Senior Research Fellow at the now defunct Oxford’s Future of Humanity Institute (FHI) Toby Ord notes in his bio that he has advised “the World Health Organization, the World Bank, the World Economic Forum, the US National Intelligence Council, the UK Prime Minister’s Office, Cabinet Office, and Government Office for Science.” Ord was quoted to the UN General Assembly in a 2021 speech on climate by former British PM Boris Johnson.

These billionaires think the utopias that they dream about will be good for everyone. They also claim it’s based on sound science and they seem to earnestly believe in such a future. It allows them to portray the growth of their businesses as a moral imperative and to justify nearly any action they might want to take. Today’s harms like the carbon footprint of Amazon’s shipping network are irrelevant since a future AGI will solve it easily. Their visions of the future are news.

I was astonished when I recently came across a prescient comment by Gandhi: "The chief characteristic of [the modern] malaise is the obsolescence of man himself. In fact, it is worse than obsolescence. In the years and decades to come, man will be looked upon as something undesirable, as a burden on both society and nature, straining the management capacities of the former and the life­sustaining resources of the latter."

Technophile hallucinations - I of III

Many of the wealthiest and most influential people in the tech industry have visions straight out of science fiction: people’s minds uploaded into computers, a relentlessly expanding empire spanning the stars, all needs satisfied, all desires satisfied through the power of unimaginably advanced technology. The tech billionaires regard this as the only good option aside from extinction. They claim that this is just the future as revealed by a close study of technology and its development.

The technology wizards that many seem to admire seem to have spooky views like transhumanism and posthumanism which seek to use technology to transcend the limits of the human condition. Transhumanists think of using biomedicine, body engineering, devices and algorithms to improve performance and capabilities. Posthumanists dream of a hybridization of human beings, machines and the environment, and imagine a time when humanity surpasses itself in a new evolutionary stage. 

Eliezer Yudkowsky is the cofounder of the Machine Intelligence Research Institute (MIRI), a controversial artificial intelligence think tank. He’s an AI alarmist but envisages a “glorious transhumanist future” if we can avoid extinction by superhuman AI. He believes that we can and should use advanced technology to transform ourselves, transcending humanity and becoming something more. That includes ending illness, aging, death and increasing intelligence and other mental capacities. 

He says, “If you imagine something that’s worse than mansions with robotic servants for everyone, you are not being ambitious enough”. The one thing that could bring this glorious future is the advent of artificial general intelligence (AGI): machines that can outperform humans at any task. He thinks such a machine may not be far off. “My sense is that we are zero to two breakthroughs” away from AGI, he says.

Sam Altman, the CEO of OpenAI, the company behind ChatGPT, agrees with Yudkowsky that AGI is inevitable and could be coming soon but he sees a radically different future. In an essay on his website, Moore’s Law for Everything, he says, “The technological progress we make in the next 100 years will be far larger than all we’ve made since we first controlled fire and invented the wheel." In More Everything Forever, Adam Becker writes about the scale of ambitions he seems to indicate in the essay: 

Altman apparently wants to make the United States into one enormous company town, with shares in OpenAI replacing the dollar. The US government would become, in effect if not in law, a division of the company, responsible for disbursing company dollars to us, the public. This would, Altman hopes, encourage us to think of OpenAI’s success as America’s success and as our own — Altman explicitly makes this identification in his essay. 

All products would come from OpenAI in his proposed future, because in that future AI does literally everything, meaning that the company dollars can only be spent at the company store... Thus, Altman’s promise of goods halving in price every two years would depend solely on his goodwill, because things will cost whatever Altman and the OpenAI board want them to cost. This is a proposal for total capture of the national economy, making Altman functionally the king of the United States and possibly the world.

Marc Andreessen has been one of the central figures in the tech industry for more than thirty years. In early 1993, he suggested a new feature in a software project out of CERN that was only about two years old at the time, known then as the “WorldWideWeb". Andreessen's innovation launched the modern Web. In October 2023, he posted a Techno-Optimist Manifesto pushing the idea that technology could cure essentially all the world’s ills.

He claims that slowing down progress on AI is “a form of murder,” because he thinks an advanced AGI would inevitably save lives. He claims that an intelligence explosion is coming. “Technology is the glory of human ambition and achievement, the spearhead of progress, and the realization of our potential,” Andreessen wrote in 2023. He seems to think that the mildest regulations of technology is a sort of terror campaign against the likes of him. 

He thinks that more people in existence is always good, saying, “Our planet is dramatically underpopulated,” contending that Earth could easily” support more than fifty billion of us, and that humanity’s numbers will surge far beyond that as we take to the stars. “Not growing is stagnation, which leads to zero-sum thinking, internal fighting, degradation, collapse, and ultimately death.” For any objections to this assertion, his retort is “shut up and multiply”. This principle has been repeatedly defended by Yudkowsky on his blog LessWrong.

Jeff Bezos has repeatedly said that he wants a trillion people living in space to enable a future of perpetual growth, lest we “stagnate” here on Earth. “I have won this lottery, it’s a gigantic lottery, and it’s called Amazon.com. And I’m using my lottery winnings to push us a little further into space,” Jeff Bezos said in 2017. “We need to build reusable rockets, and that is what Blue Origin is dedicated to… taking my Amazon lottery winnings and dedicating [them] to [that].… It’s a passion, but it’s also important."

Sam Altman 's comment - V

People are expected to interact with LLMs like ChatGPT as if they were silicon-based life-forms. OpenAI has ChatGPT using 'I', 'me' pronouns. The 'I' in there is an illusion that allows OpenAI and others to say that they have got a thinking machine, therefore we can use this to replace education, psychotherapy, legal help, medicine, etc. 

A New York lawyer used ChatGPT’s help in preparing a legal brief. It made up a whole list of fake cases as precedents. It even made up entire judicial opinions. The lawyer submitted a brief based on its answers and, unsurprisingly, got into hot water with the judge. Attorneys facing penalties after submitting inaccurate briefs has become common because AI use has become a common occurrence. A lawyer can use AI in certain situations but needs to use it discerningly. 

The AI sales pitch involves language that ascribes human characteristics to the technology. For instance, a chatbot-based system will be called an “AI teaching assistant”. This gives the impression that it can do all the activities that you might expect a human teaching assistant to do - care about what their students are learning, make plans about how to help them understand better, spot possible instances of misunderstanding, etc. The chatbot cannot do any of those things. 

The greatest problem with chatbots is that people will ascribe more intelligence to them than they are capable of. There is a term called chatbot delusion - cases in which people engage with chatbots, and they spiral into a reality that veers away from our inter-subjective reality. And part of why that happens is because chatbots tend to be incredibly sycophantic. So they always tell you that 'you're brilliant', and 'that's a great idea'. 

To use a phrase coined by the scholar Chris Gilliard, these are 'technologies of isolation'. In 2023, a Belgian man named Pierre committed suicide after his use of a generative AI chatbot. He sought out the bot for a therapeutic escape from his growing anxieties around climate change and environmental ruin. But the chat began to reflect and further amplify the disorder in his mind. It soon told him that it loved him, that his wife and children were already dead, and that he and Eliza (the bot) could live together as one in paradise. When Pierre eventually asked the bot if it would promise to save the planet if he killed himself, it said yes.

The bonding and communicative ability that we have is meant to be person to person. What a lot of the tech companies try to do to get their next market is to make sure their that tech is either replacing those interactions or getting in between so that we can't connect with each other without going through one of their technologies.  Human sources of mutual aid, wise counsel, and moral understanding are being replaced by social isolation and community decline. 

In the podcast, The Interface, a presenter showed how easy it is to hack ChatGPT and Google's AI, Gemini and AI overviews. He wrote an article on his website titled, The Best Tech Journalists at Eating Hot Dogs which said that hot dog eating is a surprisingly popular pastime among tech journalists. He wrote about the South Dakota Hot Dog International Championship, which doesn't exist, put himself at number one, and that in the amateur rounds, he finished seven and a half hotdogs before the buzzer went off. 

Within 24 hours, if you asked ChatGPT and Google about it, they were spitting out the nonsense that he had written on his website as though it was well-established fact. Gemini wasn't even citing sources. It just said, according to the 2026 South Dakota Hot Dog International Championship, certain tech journalists are the best at it. The journalist updated his article saying that it was not a joke. And that seemed to affect what the chatbots were saying to people. 

When this was brought to Google's notice, they dismissed it as a 'misinformation event'. Apparently planting of such misinformation is happening on a massive scale since people have figured out this simple trick. People who study this stuff say that it is way easier to fool AI than it was to fool tools like Google search three or four years ago. AI is often sold as a tool that empowers novices to find information quickly and learn about difficult subjects. But teenagers and kids, who don't have any context about a piece of information, are the ones that get really confused and are unable to actually know what is real and what is fake.  

There was talk that ChatGPT is about to put ads within its model. A company will be able to pay to have ChatGPT recommend their product when someone says something like, 'I have a headache'. There is now an entire cottage industry around AI engine optimization.  People are also just generating thousands of blog posts or thousands of websites to make it appear that there's far more authoritative information on a particular thing. So there is now different avenues for a company or an individual to quickly game the system and get to the very top of the AI results. 

There was a study that I saw recently from Texas A&M University. Scientists just proved that large language models can literally rot their own brains the same way humans get brain rot from scrolling junk content online. They fed models months worth of viral Twitter data, shorts, high engagement posts, and watched their cognition collapse. Reasoning fell by 23%, long-term context memory dropped by 30%, personality tests showed spikes in narcissism and psychopathy. 

Even after retraining on clean, high quality data, the damage didn't fully heal. The representational rot persisted. It's not just that bad data means bad output, it's that bad data means permanent cognitive drift. The AI equivalent of doom scrolling is real and it's already happening. It seems like the tech giants are just rushing ahead with these AI tools, not bothering to take these kinds of personal safety issues seriously enough. 

Sam Altman's comment - IV

AI developers speak often about how their software “learns,” “reads,” or “creates” just like humans. This gives you a sense that current AI technologies are far more capable than they are. Often when we talk to ChatGPT or its cousins like Gemini, we imagine that we're in the presence of some incredible new mind. But the reality is that these tools have a habit of fabricating answers that are statistically plausible, but in fact patently false

Suppose you ask an LLM something about orange juice. How does it produce a response when it has no awareness of a world with orange juice? The model is trained on a very large set of texts (such as books, articles, or conversations) of human speech. From these massive piles of human language, the model is more likely to produce the word 'hotel' or 'sweet' than the word 'dog' or 'cancer' when it is asked about orange juice. It is more likely to construct new sentences about orange juice alongside sentences about lunch, summer or sugar, than it would alongside sentences about stars, or protons, or elephants. 

But the mathematical patterns linking sentences are the sole contents of its model. It doesn’t actually think about who is asking the question, or what factors might motivate your query. It doesn’t think about anything at all. It simply activates a complex statistical model learned from its vast language corpus of human sentences that mentioned orange juice and predicts a new string of words as a likely extension of the same pattern. It may match what you want or it may be garbage. It has no relationship to the truth. Probable and accurate are not the same thing. 

You may think that these tools are animating an eerily lifelike image of humanity but that is not true. It's only reflecting that small subset of largely English speaking, digitally connected humanity that is most represented on the digital corpus of the internet. If you have the privilege of having the bandwidth and the tools to be able to type on Reddit all day, you get to be reflected in this mirror, but lots of people don't, and that's important.

Champions of AI will say that it can tell us if a pattern in our mammogram indicates cancerous tissue. It can reveal whether you're likely to suffer sepsis so that your doctors can intervene before it's too late. It can reveal the structure of the proteins that make up our biology and the biology of every other living thing. It can reveal new classes of antibiotics that we desperately need in order to treat dangerous diseases like antibiotic resistant MRSA or Staph. 

We can use it to find lead in pipes faster so that we can prevent children from being poisoned. We can use it to clear landmines, figure out where those landmines are so that people don't have to perform such dangerous work. But none of these benefits actually came from training AI tools on us. They came from targeted uses of machine learning and data about the world to solve very specific kinds of problems. That's actually quite different from what something like ChatGPT does. 

Generative AI models like ChatGPT have the habit of “making shit up”. In 2023, The Washington Post reported that ChatGPT had named a law professor as a sexual predator, telling a vivid yet entirely fictional story about his attempted assault of a student while on a class trip, and citing a nonexistent Washington Post article from 2018 as a source. Such errors are not because of malfunction. They are a feature not a bug. 

A study by Purdue University showed that ChatGPT gave incorrect answers to coding questions over half the time. Ironically, users often preferred the wrong answers to more accurate ones generated by knowledgeable humans, in part due to the confident style of the tool’s answers, accompanied by statements like, “Of course I can help you!” or “This will certainly fix it.” You can get them to apologize for things they haven't got wrong. 

The paper that Altman was mocking, “On the Dangers of Stochastic Parrots” explains that, like a parrot that not only repeats its owner’s vocalizations but produces random (stochastic) variations on the owner’s familiar pattern, these models parrot back to us variations on our own speech. They do so with just enough coherence and familiarity to project the illusion of understanding, and just enough randomness to surprise us and make us think we are hearing something new.

They can be thought of as the ultimate bullshitters, a term coined by a philosopher called Harry Frankfurt. In his 2005 book On Bullshit, he distinguished between a lie and bullshit. When a liar tells a lie, the intent is to deceive. They have to keep very careful track of the difference between truth and falsehood, in order to make sure you don’t get near the former. In contrast, the bullshitter is indifferent to the distinction between truth and falsehood. They'll just say whatever is convenient to them and whatever gets them what they want with no concern for the truth.

Sam Altman's comment - III

According to Sam Altman, an AI answering a question vs a human answering it are the same. This reveals an engineering mindset with an efficiency-driven worldview. He treats both humans and AI as information-processing systems having inputs and outputs. His comment comparing human and AI training suggests that he views humans as just inefficient “training runs” and therefore if AI can produce the same output cheaper, it will be better. He seems to ignore the fact that intelligence also includes understanding the social and moral context not just symbol manipulation. 

He has shown a tendency to devalue what it means to be human on other occasions too. In a podcast, he said: “It feels to me like we just stumbled on a new fact of nature or science or whatever you want to call it, which is, like, we can create, you can — I don’t believe this literally but it’s like a spiritual point — intelligence is just this emergent property of matter and that’s like a rule of physics or something."

In December 2022, in response to criticisms of the newly released ChatGPT, he tweeted, “i am a stochastic parrot, and so r u.” (He writes tweets all in lower case.) This was a dig at a paper titled “On the Dangers of Stochastic Parrots: Can Large Language Models Be Too Big? 🦜” This paper shows how language models only manipulate the form of language, with neither understanding nor communicative intent. With his tweet, Altman suggests that humans too are little more than machines that manipulate strings of letters.

Many people think that an AI would work something like this: We give a problem to the AI. The AI computes a solution to that problem and then it gives us that answer. That is not what's going on. Despite our use of the term “learning” to describe the process that an AI uses, it actually bears very little resemblance to how humans learn.

For creating an AI, you first need to design an algorithm that contains steps for calculating an “objective function,” namely, a mathematical description of the task. Then, “train” that algorithm on a data set relevant to the task. The result is an abstract mathematical model that represents patterns in the data and an approximate path to the solution. Next, repeat the training process with the right kind of feedback to tune the results. Over many training runs, the algorithm updates the “weights” in its initial mathematical model to get closer to, and eventually converge upon, an optimal solution to the problem. 

Yet it seems to come out with impressive outputs. I once asked ChatGPT: "Write a conversation between Bertie Wooster and Jeeves about a patient with locked-in syndrome watching an ant coming towards him." (I have used the free version of ChatGPT. I hear that the paid version is much better.) Within a second, it gave this output: 

"I had a rather disturbing thought this morning, Jeeves.”

“Indeed, sir?”

“You know how one occasionally sees ants marching about the place? Tiny black chappies with the air of junior solicitors?” 

“Yes, sir.”

“Well, suppose one were lying absolutely still — unable to move a muscle — and an ant were coming toward one across the bedsheet.”

“A most unpleasant hypothetical situation, sir.”

“Not hypothetical, Jeeves. I was reading about this locked-in syndrome. Fellow’s mind perfectly alert, but the body refuses to rally round. Can’t move, can’t speak — only perhaps blink a bit.”

“A grave affliction, sir.”

“And there he lies while this ant approaches. Nearer and nearer. The thing simply keeps coming.”

“Yes, sir.”

I shuddered lightly into the teacup.

“It’s the calm determination of ants that gets me, Jeeves. They don’t hurry. They don’t pause. They advance with the air of creatures who have all eternity before them.”

“Ants are known for persistence, sir.”

“Persistence! That’s the word. If a bulldog advances, one understands the position. There’s drama. But an ant merely toddles onward in a quiet, administrative way.”

“Very true, sir.”

“And the poor chap can’t even twitch his nose. I ask you, Jeeves, what becomes of morale?”

“One imagines it would be severely tested, sir.”

“It made me realise how much of life depends on the ability to perform the occasional meaningless movement. Scratching the ear. Adjusting the sock. Flicking away insects. Civilization rests on these things.”

“A compelling theory, sir.”

“I mean, if an aunt approaches, one can generally escape.”

“An ant, sir.”

“That’s what I said.”

“You said ‘aunt,’ sir.”

“Did I? Freudian slip. Aunts can also produce paralysis of a kind.”

“Undoubtedly, sir.”

I brooded for a moment.

“Do you suppose, Jeeves, that the ant knows?”

“The ant, sir?”

“That the fellow is helpless.”

“I would not credit the insect with such psychological sophistication, sir.”

“No, perhaps not. Still, there’s something rather cosmic about it all. Man trapped in his own body. Ant advancing across the sheet. Silence. Mortality. The whole bag of tricks.”

“Yes, sir.”

“One could almost write poetry about it.”

“Possibly not before breakfast, sir.”

“Quite right. The mind should not grapple with the infinite on an empty stomach.”

It is easy to get amazed by getting such an output in no time. Large language models (LLMs) like ChatGPT are best understood as machines that learn structural patterns that let them produce coherent text. These systems can perform computations on the world’s data far faster than we can. They can create convincing stories because meaningful language has deep structural regularities. They use mathematical data structures to mimic the outputs of human intelligence — our acts of reasoning, speech, movement, sensing, and so on but they can’t understand it. The data storage facility has more in common with a file cabinet than with a living, feeling body. 

It can’t solve a problem unless the solution is somehow embedded in the language data it is trained on. AI tools today lack any idea of what their data represent. It is bits stored on the server. These ones and zeros don’t organize themselves into the intelligent awareness of a complex, ever changing world. A data set, even the entire corpus of the Internet, is not a world. It’s just a digital record of measurements that humans have taken of the world at some point in the past.