My notes for this essay, written on a plane with no internet and no AI :D
I have been observing, in myself and in those around me, a tendency to increasingly offload our thinking to AI. From trivial decisions to complex thinking, it is easy, convenient, and in some cases, encouraged, to use AI for researching, reasoning, and answering our every query.
I recently read “The Perfect Match” by Ken Liu, a 2012 short story which describes this phenomenon with unexpected accuracy. In the story, a universal AI assistant named Tilly serves users by offering useful and enjoyable recommendations. The main character asks Tilly questions like “What do you recommend I do for breakfast this morning?” and defers to Tilly to find him a suitable person to go on a date with. The main character does not know what he wants to eat for breakfast, what music he would like to listen to, nor what to say on his date. “Who knows your tastes and moods better than I?” quips Tilly in an affectionate voice.
My friend recently went to a San Francisco startup event, where he encountered a man with a small device pinned to his shirt. The device was a sleek little capsule of polished metal, no more than two fingers wide. My friend asked about the device, and the man said it was a microphone which he used to record all of his conversations. At the end of the day, Microphone Man would kick off a workflow to summarize and analyze all of the conversations. He said, with the enthusiasm of a tech bro unveiling his latest setup, “I think Claude Fable is smarter than me. It’s better at critical thinking than I am, so I let Fable do all of my thinking these days.” (Side note: his startup is replacing human engineers by capturing their every input and operation, but without their explicit consent. He has offloaded his own thinking to AI, and made a business of offloading everyone else’s.)
An AI-generated image of Microphone Man based on the description from my friend + my imagination.
Before Claude, ChatGPT, and Gemini became household names, we were already offloading parts of our thinking to search engines. But search still required us to break down a question, evaluate sources, and synthesize an answer. AI increasingly performs those intermediate steps for us, producing a finished response to even complex or esoteric questions in minutes.
Tools like Google Deep Research and OpenAI Deep Research can now do work that might once have taken a single human being, minutes, hours, or days (see METR’s Task-Completion Time Horizons of Frontier AI Models). It saves you time, and it saves you thinking.
But it is a fine line between having an assistant that helps with your tasks, and losing all of your autonomy. Perhaps the question to ask is: who is making all of the final decisions for the things that really matter to you in your life?
In Ken Liu’s story, the main character believes that the algorithm knows him better than himself: “Everything Tilly suggests to me has been scientifically proven to fit my taste profile, to be something I’d like … What’s wrong with listening to Tilly so that the perfect product finds the perfect consumer, the perfect girl finds the perfect boy?” He defers all decisions, as trivial as what to wear and as important as how to find love, to his assistant. The Microphone Man, similarly, defers all higher-level thinking to Claude, which he believes is smarter than he is in all respects.
The offloading of thinking to AI creeps into my life, too.
There will always be some tradeoff between slow thinking and quick answers. Many questions merit quick answers (What is the weather now? Who was the president of XYZ country 10 years ago? What are the reviews for XYZ brand of skincare or sports equipment?). Many others, I think, would merit longer thinking.
Sometimes, I go on walks around my neighborhood without my phone. Invariably, questions pop into my head, questions I am so used to looking up immediately on my phone (Do cherries grow on trees or bushes? When and where was the first World Cup game?), but I find that I forget most of them by the time I get home — I remember the important few, and I assume the rest were insignificant enough to forget. Maybe there is some value in our lives to forgetting the trivial, to not having an immediate answer to every query that appears in our minds.
A few months ago, I was traveling in Portugal with my sister. After walking around the Monument to the Discoveries, which celebrates Portugal’s “Age of Exploration”, we got the feeling that Portugal seemed to idolize these “discoverers” and “explorers” whereas in the US, we would call them “conquerors” and “colonizers”. I asked our tour guide if Henry the Navigator or any of these men were cancelled, in the way that Christopher Columbus is very cancelled in the US. She responded that they were not, and in fact, men like Henry the Navigator were generally regarded as admired historical figures.
Monument to the Discoveries, which is a lot taller but didn’t fit into my camera. Totally unrelated to this article, but the guy on the farthest left looks a lot like Lord Farquaad.
My sister wondered why Portugal seemed so proud of their colonial history and why their response to colonialism seemed so different from how the US currently talked about and treated its own history of colonialism. “Let’s ask ChatGPT,” she said, pulling out her phone.
I suggested (with only a little bit of initial resistance) that we pause and think about why this might be. I suggested a few theories. Perhaps it was Portugal’s relative homogeneity and religiousness, compared to the US’s diversity of immigrants. Perhaps Portugal clung on to so-called “Age of Exploration” as one of the most prominent chapters in its national story. We wondered, postulated, made wild guesses, backtracked, connected our ideas, disagreed, and remembered historical details we learned in high school many years ago. We drew on our collective memories, knowledge, understanding of the world, and critical thinking skills. We knew we were speculating, and some of our theories were probably wrong; that was part of the exercise.
Eventually, we asked the same question to AI. Its response corroborated many of our theories and supplied several explanations we had missed. It also omitted a few possibilities we still found plausible. We had begun with a question, generated hypotheses, and only then used AI to test and extend our thinking. I relished the exercise.
I work in AI. I work on measuring Gemini’s capabilities in solving hard tasks, including those involving thinking and using tools. I also see many people in my life enthusiastically describe how AI has helped them in their working lives. For example, my cousin, who works at a Korean firm, uses Gemini to translate long official English reports into Korean, which helps speed up her work considerably. My colleagues at work develop research ideas and have coding agents implement the details, so that they can spend more time on the analyses. My friend prepared for the MCAT in just a few months with the help of ChatGPT as a personalized tutor, a process which included learning biochemistry from scratch.
One could argue that if you offload mundane thinking to AI so that you can do other, more important thinking, perhaps it is something that increases life satisfaction and productivity. Especially if AI is used to automate routine, repetitive, and tedious tasks (see the OECD’s report on the impact of AI on the workplace), tasks which previously human workers were paid a pittance to execute (see the International Labour Organization’s report, Digital Labour Platforms and the Future of Work), isn’t that a net positive to free up people to do other, more interesting, more fulfilling types of thinking? If we let the AI do the many menial tasks that encompass our jobs, to cheerfully execute hours of drudgery, don’t our lives become slightly more enjoyable?
The ease of using AI to answer our every query can also lead to lazy thinking. My mother teaches physics at an online university. She suspects that most, if not all, of the students complete their assignments using AI. She has noticed that some responses to assignments are nearly identical across students, as if they had just copied and pasted the question directly into the same AI tool, without a single original thought or opinion to differentiate their answers from the generic AI answer. She has no way of proving that AI was or was not used, and the answers are fairly comprehensive, so most of the students get an A.
AI can support learning, but it can also produce an answer without teaching you how to arrive at it. The process of solving a physics problem or writing an essay may be considered by many students to be tedious (Which equations? Which sources? Which arguments?). But then, what is the point of being in school or of learning?
There is no clear way to separate full autonomy of thinking from automating parts of menial work. It is often some blend. Like the Microphone Man, I collect data on myself and analyze it. In previous years, I even had AI analyze the data for me.
Am I any different from the Microphone Man? Perhaps what differentiates me is that I still collected and curated the data, formulated the questions I wanted answered, and evaluated the end results? Or that the data was my own, instead of recording other people’s conversations? There will always have to be some balance between automating menial tasks to free up time for rewarding endeavors, and doing the work yourself as a learning experience.
Jenny, another character in Ken Liu’s story, aims to counterpoint the main character’s over-reliance on his AI assistant. She exclaims, “Tilly doesn’t just tell you what you want! She tells you what to think. Do you even know what you really want anymore?” Our autonomy depends, at least in part, on continuing to participate in forming our own desires. But when we offload thinking about what we want (What music should I listen to? What movies should I watch? What food should I eat? What shoes should I wear?), who do we become?
What are we automating? Human work or human agency? Human tasks or human thinking?
Thanks for reading Art Fish Intelligence! This post is public so feel free to share it.