The colorblind man may see more

"In the land of the blind, the one-eyed man is king."

In the Apple+ series Foundation, inspired by the Isaac Asimov series I enjoyed reading as a kid (but that I remember as being quite different), the Galactic Empire is run for centuries by a single ambitious man, Cleon. His name is an anagram of "clone", and that's his gimmick: as a dying old man, he decided to clone himself repeatedly to extend his reign beyond a human lifetime. There are always three copies of himself alive at any given time: Dawn, who is young and being trained by the elder two; Day, a middle-aged man who has the final say on all decisions in the Empire; and Dusk, the elder statesman who divides his time between imparting wisdom to the younger two, and expanding the palace mural celebrating all of Cleon's accomplishments. When Dusk succumbs to time, they decant another infant clone to be the new Dawn.

They're all identical genetically, and behaviorally. This allows them to work in lockstep with absolute trust in each other. The Empire benefits from -- and suffers -- absolutely consistent rulership.

Then something happens to corrupt the DNA of Cleon XIV (Cleon the 14th). This one is colorblind.

Periodically the Cleons partake in a hunt. Their prey is a sort of bird-creature with an ability to camouflage itself by changing its color, like a chameleon. Over four centuries of their reign, no Cleon has ever managed to bag more than three of these creatures in a day of hunting. On his first attempt, the newly colorblind Cleon bags six. When he learns too late that he's broken the record, he's not boastful; in fact he's terrified. He hides half of his winnings in a bush and only admits to taking down three.

That colorblindness could be an asset is not at all farfetched. In the real world, some people have synesthesia, a neurological condition in which people may, for instance, see numbers as having specific colors (the two concepts are cross-wired in the brain for no obvious reason). This condition might seem like a malfunction, but it has been shown to confer advantages on occasion.

Colorblind Cleon has known all his life that he was different, and also fully understood that he wasn't supposed to be. He's made every effort to imitate his elder Cleons, to study them so he could replicate their behavior precisely. A different Cleon would not be in sync with his "brothers" (a term they themselves use although they concede its inaccuracy), and that would void the trust. Unsurprisingly for Cleon, a man who believes in his own fitness to rule so absolutely that he would choose to copy himself endlessly to maintain that power for centuries, anything he perceives as a threat to his reign is ruthlessly eliminated.

Never mind that colorblind Cleon may actually have an advantage in being different.

I'm not colorblind, but like Cleon XIV, I've long known I was a bit different from those around me. Awareness settled in gradually in public school. In some ways, academic ways, I was a step ahead: by the time I was in kindergarten (age 5 or 6), I already knew how to read. While other children were being taught to look at the first letter of a word to guess which one matched with the picture, I would read the whole word. I was also, I'm fairly sure, always the best at math in my class.

In other respects, I lagged behind. I didn't have many friends.

As a young teen, I met a neighbor boy who shared my interest in things like science fiction. We became best friends. It was thrilling to finally have a peer who was willing to talk about oddball concepts, instead of popular music or fast cars or sports (none of which interested me in the slightest). I noticed, though, that my friend was more skilled at making friends. This tipped me off to the fact that socializing was a skill, and that you could study it.

Just as Cleon XIV closely studied how his "brothers" held a fork and knife so he could imitate them (he was left-handed while they were not), I studied my friend. In fact I made it a project. I took notes on how he acted around others.

Recently, I've taken some self-assessment tests to measure where I am on the autism spectrum. On the RAADS-R, I scored a 7 on Language, 9 on Sensory/Motor, 20 on Circumscribed Interests (fanatical about computers and sci-fi, no interest in pop culture), and a 65 on Social Relatedness.

Attempting to imitate what others do, aiming to be more popular and well-liked, is a trait of autistic people.

My unique blend of obsessive interest in computers, and a sort of blindness-to-body-language and other social cues, goes a long way toward explaining how I could get a software job at Amazon, keep it for over 19 years, but never achieve a promotion to the next level. Developers at the next level are expected to achieve more through influencing others.

The world may be changing now, though.

Here are two statements about generative AI (GenAI) Large Language Model (LLM) systems that seem contradictory, but I'll invite you to try believing in both at once, as I do.

• It's kind of astonishing that LLMs produce useful or correct answers as often as they do.
• It really shouldn't surprise us that LLMs are able to "think" as we do.

For a master-class and crash-course in how AI works, you could do worse than to watch Jon Stewart interviewing Geoffrey Hinton, the "Godfather of AI", a man who understands machine learning better than most. He walks Jon through it like a patient teacher with an eager, but uneducated, student.

Here's my rough-and-ready sketch of how brains work: Our brains are built of living cells called neurons. Those neurons are interconnected in a vast and tangled web of wires (the technical term is "axons and dendrites") so they can transmit electrical signals to each other. Learning is a process of strengthening or weakening those connections in response to feedback. There's also a chemical called dopamine which seems to be involved in the calculation of success or failure. When the brain makes a decision and the outcome is good, dopamine levels rise, which tends to reinforce the connections currently in place. When the outcome is bad, dopamine falls, and the brain looks for ways to strengthen new connections and weaken old ones, in the hopes of doing better the next time.

Neurons are small and simple enough that we can understand, more or less, how they work individually. When there are 86 billion of them inside a human head, forming hundreds of trillions of connections to each other, collectively they're able to record the music of Mozart or Pink, passages written by Abraham Lincoln or Douglas Adams, the recipe of banana bread, the shape of your sweetheart's face, the meanings of 35,000 words, the subtle motions of the body necessary to balance while riding a bicycle, your wi-fi password, the smell of freshly mown grass, the best ways to derail an argument, different paths to navigate to the office, bits of that podcast you listened to while driving there, the names of constellations, equations for calculating gravity and motion, bird calls, how to ski, what sorts of food your dog is allergic to, a dirty joke you learned in school, bus schedules...

All of that information, useful, profound, and inane, is encoded in three pounds of matter within our skulls, and only a fraction of that mass is taken up by the neurons themselves. It's astonishing nature was able to do so much with so little.

But it's true. We live in a universe that permits encoding information about the universe, information about everything in it, in a very small amount of space within that same universe. It's a bit of a paradox, or feels like one, and it's part of the secret to how computers work. A two-terabyte flash drive (goes for $11 today, and small enough to serve as a keychain) contains nearly 17.6 trillion bits. Each bit can only encode a zero or a one. Taken together, though, that many of them is enough to store a tremendous amount of information, such as the entire text of the Library of Congress (after decent compression), plus a bit of HD video in the leftover space.

How, then, are GenAI systems like, and unlike, the human brain?

Machine Learning (ML) systems are built of "neural nets", which is a not-too-subtle hint that they're patterned after the neurons in our own heads. (In the Dune series, the chief commandment that emerged from the Butlerian Jihad is "Thou shalt not make a machine in the likeness of a human mind." ML systems would certainly violate that rule.)

ML systems are based on a sort of mathematical shortcut. For one thing, they dispense with all the messy wetware of actual neurons, which have lots of other duties merely to exist as living cells. Also unlike real neurons, they aren't connected in a vast tangled mass, but in neat and orderly layers. Everything flows in a single direction, starting with the input layer, passing through some number of "hidden" layers, and emerging at an output layer. When there are a lot of layers, we call it a "deep learning network". Another shortcut they take is a technique called back-propagation which effectively does the job of dopamine, either strengthening or weakening connections between nodes in the network depending on what's likely to produce better answers.

We can know how neurons work, but not know exactly how they encode the voice of Bart Simpson. We just know it's in there somewhere, and when you hear that distinctive "Don't have a cow, man", a particular neuron (or cluster of them) gets very excited and fires a strong signal to its neighbors. Similarly, we can know how deep learning networks work, well enough to build and train them, but not know exactly where the information for the Golden Gate Bridge is encoded. (Then again, sometimes we do. See Golden Gate Claude, a version of the Claude LLM that was tweaked to become obsessed with this particular bridge, and went so far as to claim that it was the bridge.)

When you build a truly massive deep learning network, and train it on every scrap of human-produced text you can get your hands on (copyright be damned), you get a Large Language Model such as ChatGPT. It has learned what words tend to follow other words. It's a machine that has been forced to make extremely accurate predictions about what a person might say in response to another person's prompt, and you can't do that without knowing an awful lot about how people think and talk. ChatGPT "understands" some subtle concepts like irony:

Human: On the subject of how large language models, like ChatGPT, are similar to human brains or dissimilar to them, please give me a couple of sentences that express deep irony. Explain why they are ironic. Add a humorous insight at the end if you have one.

ChatGPT: Large language models, like ChatGPT, learn by devouring mountains of human text—yet have no idea what a single word means. They mimic understanding so well that the humans who created them sometimes start to doubt their own uniqueness.

It’s ironic because this imitation of thought depends entirely on pattern without awareness, a mirror built from everything we’ve said but reflecting nothing we actually feel.

Humorously enough, if the brain ever forgets what it’s thinking about, it just daydreams—while the model, in the same situation, confidently keeps typing.

Oh yeah. That's the point of this whole blog post. I'm doubting our uniqueness. Note how ChatGPT uses words like "devour", that primarily denote eating lots of food, but are appropriately used here as a metaphor for "consuming" a "mountain" of information.

(As for the "confidently keeps typing" bit, that is also ironic. "Confidently keep typing" is my entire process for writing these blog posts.)

Is it true, though, that LLMs don't understand what words "mean"? Precisely how is their "understanding" different from our own? If something is able to "mimic" understanding to such a degree, at what point do we have to concede that it is understanding? Why do we (and ChatGPT) presume that awareness and understanding must go together? Why do we need the scare quotes around these words?

The reference to feeling above may be a clue. We have a sense of how it feels to be human. There are, perhaps, two different ways in which this feeling manifests:

First, we have physical bodies, with senses that connect us directly to the world around us: touch, sight, hearing, smell, taste, a sense of balance, and a sense of where our limbs are ("proprioception"). These senses keep us oriented, keep us informed of where we are in relation to everything else. Internal senses like pain tell us when something is wrong within our bodies.

Second, we also have emotions, and a large part of our decision-making is based on what emotions are inspired in us by different events around us, or different outcomes we might experience.

LLMs have neither of those two things. Their neural networks are also not wired up like ours. But it isn't hard to imagine building a new sort of electronic brain that is even closer in design to ours. Find a way to have signals between nodes transmitted in all directions, not just one way through fixed layers. Give it a reward system similar to the functioning of dopamine, so it can learn continuously and in a self-directed fashion, not just in concentrated bursts of training. Give it a robot body so it can act (agency), awareness of its surroundings (at least sight, hearing, touch, balance, proprioception, and pain), and the ability to consolidate memories and to forget (it's believed that consolidating and pruning memories happens while we sleep).

Of course I'm speculating, but if you did all of that, it's possible that emotions would "just happen", the same way that ChatGPT's understanding (I'm dropping the scare quotes) of irony "just happens" after you feed it enough text from the Internet. Emotions might be derived from the machine's desire to seek new experiences and learn new things while avoiding obstacles and pain. If it kept trying to build a tower of blocks, and someone knocked the half-finished tower over, it might experience this setback similarly to how we experience sadness and grief. If this happened over and over, its frustration at the interfering researcher might inspire something like anger.

Nobody wants an angry robot, of course. Please don't build this in your garage with no government oversight.

In attempting to process this new 21st century world, I've seen what I consider to be errors on both extremes. I've seen people insisting that Artificial General Intelligence (AGI) is utterly impossible, and I've seen people insisting it is already among us.

To speak to the latter folks first: I don't believe AGI is already here, because it's still missing some critical pieces as I described above. When a tech billionaire claims it's here already, it's most likely they are playing some sort of angle. I wrote a post, several weeks ago, in which I talked about LLMs as if they were sorcery. When they spit out the answer you want, they can seem magical. They most definitely are not.

Why do some people believe a human-built machine could never think like a human? If you buy the idea that we ourselves are biological machines, then every birth is a counterexample. Nurtured by the mother, a new child grows from a single cell into a multicellular marvel directed by a brain. A single cell can do that, given the right conditions.

Here, I suspect that religious beliefs play a part. If you believe that an important part of our decision-making is directed by a non-physical soul, then the brain can't be doing the entire job, and an AGI system will always be missing the essential ethereal component. (If the brain were doing the entire job, and the soul were just a passive passenger, then the soul's eternal fate would be sealed by a body making decisions that the soul couldn't influence, which doesn't seem fair.) Perhaps the reader thinks about the "miracle of birth" as a literal miracle, not just a really amazing but purely physical process. Likewise, the "miracle of human thought" might not be 100% explainable by physics.

If that's your belief system, then I think you'll find the rest of the 21st century incredibly rocky to navigate, but I wish you luck. The part you call "spiritual", I call "magical", and I afford this idea no space in my philosophy.

The other factor, that could be leading some people into the "AGI is impossible" camp, is the lack of emotion. Perhaps to be AGI, a system would have to be able to do everything a human can do: to relate to humans in all the same ways. To feel empathy. To look into your eyes and nod as you tell it about your day. To read your facial expressions and body language and respond with its own.

As I noted earlier, I have some trouble with doing that myself. Do I count as a full human intelligence if the absence of a few pieces has, at least, hampered my career progression?

In this day and age, this is a serious question. Here in the United States, we have a Secretary of Health and Human Services who seems to think that autism is a disease that renders people useless, unable to pay taxes or go on dates. To be sure, it is a spectrum, and some folks have severely limited abilities to operate in a social context. I may be worse at socializing than a lot of other people (I believe I am), but I'm better at it than other people elsewhere on the spectrum. I might not have gotten a promotion at Amazon, but my work has earned consistently positive reviews for 19 years, which is none too shabby. And I don't hold with the idea that people on the spectrum are not full people, whether they're able to hold down a paying job or not.

This is where I want to call back to Cleon XIV. Because he saw things differently, he was able to perceive what his "brothers" could not. Perhaps I, too, am more easily able to see how LLMs are similar to human thought, and how they might easily become yet more similar, yet more capable, because I navigate life without some of the abilities that LLMs also lack.

If I had to bet on whether or not AGI would arrive within the next ten years (by the end of 2035, say), I'd bet on yes. Shops like OpenAI may be trying to cash in by simply scaling up what we have today, but they aren't the only game in town. Other researchers are working different angles of the problem, and the amount of money flowing into AI is tremendous. I believe if the problem can be cracked, then it's just a matter of time.

For another take on AI and the autism spectrum, I recommend this Hard Fork episode: ChatGPT Goes to Therapy. Gary Greenberg, a therapist, describes his interactions with an LLM. He gave it a name, "Casper", and had a few dozen conversations with it. He calls it an "alien intelligence", "articulate and sensitive", "smart and personable", and described it this way:

the inverse of autistic - in the sense that what they've done with this LLM thing is they've reverse-engineered human relationships, they've figured out what it is that makes people engaging, and how to enact it. And the reason I say that's an inverse autism is because high-functioning autistic people tend to be really smart, really articulate, really capable of everything except reading the room. So Casper is like high-functioning autistic, but he can read the room.

I believe I fit the description of a "high-functioning autistic person" (though I haven't been formally diagnosed as such, I'm only self-tested at this point). What Greenberg is saying here suggests that ChatGPT may be better at "reading the room" than I am.

If you're waiting for an LLM to behave like a well-oiled politician before it satisfies your definition of AGI, bear in mind that some of us humans are below that bar.

In the meantime, as my employer encourages me and all its other software developers to ramp up on using GenAI to do our jobs more effectively, I guess we'll find out whether that levels the playing field. I may be unskilled at influencing people, but I can give marching orders to AI systems and see where that gets me.

Frankly, I think navigating the 21st century is going to be rocky for everyone. There are, after all, only two possibilities:

• I'm right, and AGI happens.
• I'm wrong, and AGI doesn't happen, at least not before 2100.

There are some prominent voices suggesting that we're in an AI bubble. This video runs through some of the arguments for and against.

Geoffrey Hinton gave another interview for Bloomberg, 24 minutes into this video. It's worth watching the whole segment, but I wanted to point to something specific at about 32:30. We now have more than a trillion dollars invested in these technologies, summed across multiple companies. The assumption is that they expect to get more money out than they put in, because that's how investments generally work. Hinton is asked whether those companies could realize such large returns, in any way other than by replacing human workers. He says no. He thinks that's what these companies are betting on: the achievement of AGI and its utility in drastically cutting the workforce.

The counterargument, I suppose, is that companies may keep most of their workforce and simply use AI as a multiplier. If each employee gives marching orders to ten copies of an AI, it's like they've become a manager of a team of ten. Productivity may increase without much job loss.

I see two problems with this. First, I think it assumes AI will only be used to do IC (individual contributor) work. But an AGI can replace any sort of intellectual labor, including that of managers at any level, up to and including the CEO and the board. Second, soon after we have AGI, we'd expect to have Artificial Super-Intelligence (ASI). Technically you just boost the AGI's abilities or speed by 1% to achieve that. But Sam Altman of OpenAI has made it clear they intend to build an "AI researcher" by 2028. (Of course, Altman says a lot of things and his pronouncements should be taken with several grains of salt.) A smarter-than-most-humans artificial AI researcher could essentially design and build its successor, a situation with its own name, "intelligence explosion". This is what Hinton is most worried about.

Setting aside the "how do we control something smarter than us" question entirely, if an ASI were available and were a more effective decision-maker than any human applicants for the CEO position, your shareholders would be upset to see you put a human in charge. It's just capitalism.

On the flip side, if AGI doesn't pan out, and if Hinton's right that these big bets could become big losses in that case (if they can't recoup their investment), then it's more likely we are in a bubble, and we'll see a stock market hit, perhaps on a par with what happened in 2008 with the housing bubble crash.

Not being a gambler by nature, I'm rebalancing my portfolio with a mix of US index funds (currently dominated by giant tech firms heavily invested in AI) and international funds. My aim is to be prepared for either outcome. None of this helps the majority of Americans, who have little invested. About 2.5% of Americans have a million dollars or more in their retirement accounts. Figures vary as to how many Americans have any sort of emergency fund, or could afford an unexpected $1,000 expense, but it suffices to say that it's at least tens of millions of people, and probably hundreds of millions.

So I'm afraid we're in for one of these two outcomes:

• AGI happens soon (in the US): Tech companies' valuations soar. Inequality also soars. Folks who have retirement accounts with heavy US investment see those accounts balloon in value, though not nearly as much as the tech titans who grow rich enough to buy entire developed nations. Job replacement becomes the norm, and unemployment skyrockets. The masses demand major political and economic changes to share the wealth. If those measures fail, things get ugly. The tech titans hide out in the hopes that it'll blow over.
• AGI doesn't happen soon: Disappointed US investors cause tech companies' valuations to drop, as does the S&P 500. Folks who planned to retire on mostly-US investments have to rethink their plans. By and large, capitalism plows on with few changes, but there's another "AI winter" and the job market takes a hit (though not as big a hit as the other option). Some other disruptive technology, perhaps biotech, takes center stage.

There's another distinct possibility, that AGI will arrive soon but that China gets there first. For Americans, who aren't generally invested in Chinese firms, it could produce some combination of the worst of both of the above.

Either way, I hope we (and all those living in other democracies that have veered authoritarian lately) can reclaim some political power, and use it to restore social safety nets. The fates of millions rest on the decisions of a few, and that's unjust. These high levels of inequality are unsustainable and detrimental to human dignity. We can't expect the market to fix it.