From chapter one

Five Hundred Years of Getting It Wrong

Get launch updates

In 1492, a German monk named Johannes Trithemius sat down to write a passionate defense of an ancient practice he believed was dying. The subject of this particular defense was the hand-copying of manuscripts. At the time, for Benedictine monks, it was a painstaking, meditative labor by which they had preserved and transmitted human knowledge for centuries. But a new technology was arriving that could reproduce a text far faster than a person copying it by hand. It could get books into the hands of far more people, and maybe more importantly to Trithemius, it threatened to make the monk’s most sacred work unnecessary. Trithemius believed that abandoning the work would be a serious loss, and he wrote out the details of why this would be so devastating.

He called his treatise In Praise of Scribes. And when he was finally finished, he had it printed on a printing press.

It’s easy to laugh at the irony, but that doesn’t make his concern any less valid. Some of his objections were about the physical nature of the books themselves. He had practical concerns, including how long paper would last compared with parchment. But his primary concern was centered around what this new process would take away from the monks who were doing the transcribing.

The transcribing of these texts often took months to hand copy, and during that time, Trithemius believed that the monk gained something from spending so much time with the words. Something beyond just the ideas or the message in the text itself, and more importantly, something that wouldn’t be gained simply by reading the text if it was produced and delivered by a printing press.

His theory was that the true knowledge that the monk would gain came from the hours of slow, meditative repetition, and without that arduous labor, the monk might have the general ideas, but they wouldn’t possess the true knowledge of those sacred texts. What he was describing was a process we can now call internalization.

We have all experienced some form of internalization in our lives. There is a level of understanding when we read an explanation for something, but it is something else entirely when we can explain it to someone else days or weeks later. And that second level of understanding is what Trithemius believed would be taken from the monks. And the loss of that deeper understanding would not only be a loss for the monks in the short term, but also a loss that would extend to everyone the monks would go on to teach in the future.

The fact that Trithemius decided to have the treatise printed is a perfect illustration of what we would see moving forward with every new technological milestone. The benefits were obvious. In this case, the new technology would be able to get the word out to hundreds of readers in a relatively short time. Something that would have been impossible before its invention. But the cost of that advancement would be a reduction in a practice that produced a deeper level of knowledge, and perhaps a deeper level of human meaning.

And that is why I think the story of Trithemius is a great place to begin. It perfectly illustrates a tension that we will see throughout this book, both in the history of past technologies, and in what we are likely to lose in a world in which AI continues to progress.

When I say that we have spent five hundred years getting it wrong, I am not implying that nobody was able to identify risks and dangers of these technologies when they were in their early phase. Instead, I see a pattern in which we knew what the cost would be but we were either too enthralled with the gains, or we were too seduced by the ideas of more direct dangers that captured our imaginations in a way that produced urgency to solve them, while the quiet costs accumulated underneath.

These quiet costs include things like the loss of a deeper form of understanding, which is real but is harder to see, in the sense that it is not something being physically taken away, but rather something that doesn’t occur, which otherwise might have. We tend to have more urgency when the loss is something we have currently and stand to lose. With AI, we fear the jobs, which we currently have, being taken from us. It is harder to wrap one’s brain around the loss of understanding Trithemius was worried about than it is to imagine losing a job we already have.

Time and time again, technology has made a massive leap forward, and it does exactly what we hoped it would do, and its benefits are real. We do not have to downplay its positives in order to understand and accept its costs. The printing press was a perfect example of this duality. The railroads are another.

From the prologue

Prologue

There are times when, despite every desire to be present, I know I am not. My body is there and my mind is out drifting into some other thought. My wife has known me for over fifteen years now, and she calls it ‘the nod.’ She says that I have two forms of listening and when I am really paying attention, my mouth stays slightly open, just waiting for a chance to interject. In the other form, I just nod, as some signal to the other person that I am just paying attention despite her knowing that my mind is somewhere else.

She’s right. I learned that a long time ago in my high school psychology class. Give constant feedback to the person who is talking, so they not only perceive you as listening, but they also like you more because the nod is a signal that you are agreeing with them. But I am not listening, and I am not agreeing. I am flushing out some half baked idea or thinking through some conversation I had weeks ago, and what I should have said.

For years, I have been working on it and I have tried to be more present in the moment, but it’s a habit that I fall back into more often than I would like. That said, I have no regrets on doing it at our home a little more than two years ago, because the kernal of a thought that inspired this book came out of one of those nods.

To paint the picture, it was early fall of 2024 and my wife invited over a couple of clients of hers for dinner. She is a real estate agent, and because of that, we spend several nights a year gathered around a table with happy couples who just bought their first home. Knowing how my mind works, she tends to like me to play it safe and stick to easier topics, and just kind of listen. And on this night, I started off doing just that.

When they arrived, they said the house was easy to find. They also told us that our dogs were cute. I told them that they were right on both counts. After some more pleasantries, we all kind of huddled around the table, kind of standing at the edge of the kitchen, with nobody quite ready to sit down. Sensing a somewhat awkward pause in the conversation, I stopped nodding and just blurted out a question before I really took time to think how it would sound out loud.

“Okay, imagine your partner created an AI version of you, and an AI version of themselves. Then, while you were at work, they ran thousands of simulations where they practiced a conversation in which their goal was to convince you to agree with them on something you currently disagree on. Would that be worse than cheating?”

If I thought it was awkward before I spoke, I wouldn’t even know what to call that next minute after I asked the question. The couple seemed genuinely confused and my wife gave me a look that signaled that I had done the exact opposite of what she had wanted me to do on this night. After a minute or so, I walked the question back a bit.

I said I was curious about kinds of betrayal beyond just the conventional ones, and all this talk of AI lately made me think of this scenario. I asked them to go along with me, and I pulled out my phone and proceeded to ask them to help me build a scale from one to ten. First, we started with white lies, then got into things like emotional and physical infidelity. I asked them to tell me what they would rate a co-worker throwing them under the bus, and I even got a number from them on streaming cheating. For those unaware, streaming cheating is when you watch an episode ahead, then act surprised later on when you watch it together. And the absolute worst version of streaming cheating is when the person “guesses” the plot twist just before it happens.

After we built out the scale, we returned to the AI scenario, and I asked them to think of a decision that matters to both of them that they don’t currently agree on. They landed on education for their children. Whether they would send their future children to public school or private school, specifically. He had grown up going to private schools and swore by them, while she found them a bit pretentious.

I then returned to the scenario. “Okay, let’s imagine that your partner builds out the AI and runs the simulation over and over again. Finally, the AI determines a sequence that, if followed precisely, will end up with you changing your mind, and agreeing with them on this topic. They use it on you, and it works. Then, years later, you find that exact script on their laptop and realize what happened. Where would that betrayal land on this scale?”

The husband was first to respond and said four. The wife, after thinking for a moment, said three. For both of them, this was below where they had put streaming cheating. And it was well below the standard forms of betrayal. As they explained why, I started nodding again. I was there, but I wasn’t listening. There was just too much going on in my head.

* * *

Over the next several months, I became obsessed, and I started asking versions of that same question to hundreds of people. I found ways to sneak it into conversations at dinner and parties, to bring it up at work events, and even broke it out during a soccer game, which saw me shouting across my wife just to talk to her client’s husband three seats down. Thankfully, I had gotten better at laying out the scenario, as I got more and more practice. I built the betrayal scale first, before I even mentioned anything about AI.

The betrayal scale often looked the same, but the answers about my AI scenario never got any higher than six. I actually got a few zeroes, too, even though the scale I laid out started at one. Over and over again, the AI scenario landed below other offenses that the couples themselves listed as minor. Meanwhile, my answer was at the top of the scale, and it seemingly got higher the more I really thought about the scenario.

Because of this, I kept finding myself nodding during conversations at home and at work, because all that was running through my mind daily was how I couldn’t believe how low of a betrayal score these couples continued to give this scenario. It was hard to understand why I had felt so passionately about this being a betrayal as bad, or worse, than any other while they saw it more akin to a white lie.

Beyond that, once I had opened up the door to talking AI, the conversations started to bleed into the actual concerns they had with AI. For most it was job loss, and an uncertainty with what would happen with their industry. Understandably, most of the people I was having these conversations with had just signed their names on a thirty-year loan, and they were terrified that they wouldn’t have a job in two years. Beyond that, I heard The Terminator scenarios. The end of the world, the robots taking over, the singularity. And they weren’t crazy for thinking about that, especially by the time I got to about my 30th couple or so. By then, the leaders of OpenAI, Google, and Anthropic had signed a public statement calling for the risk of human extinction from AI to be treated as an actual global priority.

So people were concerned about AI, but it was on these large, sweeping scales that saw us going into a recession at best, or becoming extinct at worst. Meanwhile, I never heard anyone talk about what we might lose in just an ordinary day where people still had their jobs and we were all very much alive in this AI-dominated future. I didn’t hear people concerned about the conversations we might stop having with each other because AI had better answers. Or the work that we would get done but not truly understand, because we handed it over to an AI agent, who did it in one-tenth the time.

End of this selection from the prologue.

Coming 2026

Get launch updates