Prompto, Ergo Sum
How AI changes the relationship between making, understanding, and the self
Human beings have always extended thought through tools. Writing, diagrams, calculators, and software do not simply make intellectual work easier – they redistribute effort, removing some forms of friction while creating new possibilities for reflection, abstraction, and discovery. Generative AI may be another such expansion. But because it can produce the argument, image, or solution itself, it also risks separating the artifact from the understanding that would make it genuinely ours. This essay asks what happens when production continues while formation stalls, and what kind of thinker is formed when we begin to think in prompts.
The story goes that in Egypt dwelt one of the old gods, Thoth – he who had invented numbers and calculation, geometry and astronomy, and lest we forget music! But some would argue that the greatest of all Thoth’s creations was writing. The king of all Egypt at that time was Thamus. Thoth came to him, revealing the arts he had created, claiming that they should be passed on to the Egyptians for their benefit. Thamus questioned Thoth on the use of each, condemning the bad and praising the good. When it came to writing, Thoth said, “Now here, O king, is a branch of learning that will help the people of Egypt improve their memories, providing a recipe for one to attain wisdom.”
But Thamus answered saying, “O man full of arts, it is one thing to create these arts, and another to judge what measure of harm they have for those that shall use them. And it is for that reason that your tender regard for the writing that you have created, can only see the very opposite of its true effect. If men learn this, it will implant forgetfulness in their souls – they will cease to exercise memory because they will rely on that which is written. It is a method not for memory, but for reminder. And it is not true wisdom that you offer your disciples, but only its semblance. By telling them of many things without teaching them you will make them seem to know much, while for the most part they know nothing. As men who will be filled, not with wisdom, but with the conceit of wisdom, they will be a burden to their fellows.”
Plato, “Phaedrus” (c. 360 BCE)
Adapted from Reginald Hackforth’s 1952 translation
I first encountered this passage, recounted by Socrates, in a class my freshman year of college which traced the archaeology of writing systems around the world. “Lost Languages” spoke to my newly minted liberal arts soul. “This… this will make me sound interesting at parties” I thought to myself as I happily registered.
Perhaps this was Socrates’ worst nightmare manifest. Me, excitedly sharing stories of oracle bones in ancient China and quipu from the Andes, filled with the “conceit of wisdom”. I would like to think that eventually I did earn those stories, but the jury might still be out (I got a B). And as my professor might suggest, the problem wasn’t necessarily that the stories I told were false – it was that repeating them didn’t necessarily demonstrate that I had acquired any understanding.
Socrates worried that the written word would hollow out memory and produce only the semblance of wisdom. Critics make a very similar argument against generative AI.
We have all encountered this person. The person that delegates any questions they get to ChatGPT, the student who can no longer write an essay, the speech that always smacks of an LLM. We have apps now openly marketed as tools for manufacturing the appearance of work. This is perhaps Thamus’s worst nightmare – Thoth meeting venture capital.
Writing, by comparison, is one of those inventions that receives little criticism (except from my 7 year old self when confronted with a textbook). The written word allows ideas to be transmitted across time and space, reaching across distances and generations, allowing knowledge to accumulate and imagined worlds to gather communities far beyond their authors. It gives us history, law, literature, and science, and the possibility that a sentence written by someone long gone might change the mind of someone yet to be born.
Did Socrates get it wrong then? If writing is really all it’s cracked up to be, are we wrong about AI too?
Cognitive offloading describes how external tools help with mental work. A calculator allows me to expand the bounds of my cognition beyond my remedial math. TurboTax lets me stay on good terms with the IRS without having the tax code memorized – sorry Socrates! Writing is itself a similar tool. It externalizes our thoughts and gives us a surface to tangibly organize, inspect, and mutate them. These technologies do not simply make difficulty disappear. They instead redistribute effort, changing where one encounters difficulty.
Some friction is merely obstructive. The arithmetic can distract from the proof, the laborious tax code stands between me and finding loopholes (if the IRS is reading this, that was a joke). Removing this friction lets us spend our limited attention elsewhere, and at their best, these cognitive tools can compress difficulty to the extent that new forms of thought now become possible.
But some of this friction is formative. I would like to think the process of writing this essay might demonstrate this.
Before I started writing it, I carried these ideas around in my mind, iterating on them in my head on a commute or as my mind wandered throughout the day. Each time, it sounded a little different, as details fell into the crevices of my mind, lost to my memory. Writing the first draft allowed me to put the thoughts down so I could return to them later, to something that now existed partly outside me.
And as I returned to it, day after day, the ideas slowly began to change as a rough skeleton filled in. The difficulty, however, had not disappeared. Ambiguity became visible, contradictions emerged, awkward paragraphs were eliminated and re-written. Writing was not just packaging the thoughts that had already been produced somewhere in my mind. It was the struggle of finding a sentence, realizing it failed, and revising it until it pleased me. This was how the work I had set out to do became complete, and importantly mine.
Professor Judith Fan studies cognitive tools. Her focus is on “physical representations of thought [that people use] to learn, communicate, and solve problems”. Her work on drawing centers on the idea that a drawing makes otherwise invisible mental content visible. These marks, however, do not merely represent the finished idea. The process of producing them requires perception, memory, action, and abstraction. It requires imagining an audience, and feeds back into the mind that made it. These diagrams, sketches, and sentences are thus both a record of thought and part of the machinery through which thought develops.
And this was how writing worked for me. The page I wrote on relieved me of holding the essay as a whole, but also exposed the essay to my own inspection. It relieved the burden of memory, but created the formative friction of confrontation as I had to grapple with what I was actually trying to say.
This is the history of cognitive tools. They do not just let us do the same thing faster or more easily, they also create new representations through which we can preserve intermediate steps, discover abstractions, combine ideas, and ask new questions. Professor Fan highlights the Cartesian coordinate system as one such tool. By allowing algebraic relationships to be expressed visually, it exposed a bridge between algebra and geometry, allowing insights from one field to inform the other, opening up new mathematical possibilities.
And the same could be said of these other cognitive technologies. Calculators outsource rote computation from the mind, so the mathematician may grapple with the proof. An Excel spreadsheet can hold representations and relationships more easily than paper or lists, making patterns visible that would otherwise remain submerged in sheets of numbers. These tools do not impoverish cognition, they instead redirect effort toward harder problems.
And in this way, perhaps generative AI can be deployed as a similar tool. In writing, it can remove the blank page so that one can begin. It can produce alternatives against which judgment is sharpened. It can expose weaknesses, help retrieve adjacent concepts, and do all of this at a speed that permits more iteration than time would allow.
In the case of this essay, once it took shape, I put it into a chatbot. “Where has my argument been inconsistent, are the sources I quote accurately represented, what errors in expression have I made?”. (And after that, the essay went to my friend CJ, who was less obsequious than the chatbot.) The chatbot gave me another surface against which to argue.
It helped. The chatbot noticed the looseness in the essay, paragraphs which were contorted around themselves, places where “thus” was doing far too much work, and conclusions that appeared before their premises were earned. And so, AI did not replace the thinking I did, but it did give me a new representation to grapple with – an outside perspective.
Still, this changed the nature of the work. Before the chatbot, an awkward paragraph sat there staring at me, waiting for me to man up until I could figure out what was wrong with it. After the chatbot, the accusation arrived, named, sorted, and softened enough for my ego. What did this give me? Speed and iteration. But something also closed to me – the space in me where confusion percolated in my mind, slowly becoming articulate, an articulation that had to happen before anyone else could help me.
Do I still struggle then? Do I struggle enough? If I had been spared the difficulty of articulating the confusion, what exactly made the eventual thought mine?
The amount of struggle is not necessarily the right measure. An afternoon spent fighting my code to compile, or a week spent fighting Figma to get my designs to line up perfectly is not proof that meaningful understanding has been formed. The question should instead be what that difficulty was in service of.
Was this difficulty necessary to produce the artifact? Was it necessary to understand it, or to learn the domain? Was it necessary for the artifact to count as self-expression?
Which difficulty matters depends on the thing we are making, and what making it is supposed to accomplish.
If I use AI to summarize my meeting, perhaps little difficulty has been lost – a transcript and minutes exist simply to preserve information that would otherwise be lost. If AI organizes a spreadsheet, perhaps it might expose a pattern I could not have seen on my own. In both cases, the tool removes the burden of execution without necessarily displacing the judgment that gives the artifact its purpose.
We might treat a song differently. If AI helps write a song for me, the anxiety is less so that every word belongs to me, but that the feelings that the song was supposed to express are now outsourced. The song continues to change and move, while I remain still.
What of code? One might argue that code is instrumental, it is simply a tool that is supposed to work. And I would argue that very few programmers believe that authorship of a piece of code is determined by whether every line was written by them unaided. A plethora of tools already exist to help with these tasks, like compilers, libraries, documentation, and IDEs. Cognition in software engineering has always been distributed among an engineer’s toolkit.
Working code, however, is not the only product of programming. Building the system the code operates in is how one acquired the mental model needed to alter it, repair it, and importantly, answer for it when it breaks. AI may produce the working code faster than the programmer develops the tacit understanding required to maintain it. A person can possess the code without necessarily possessing the system.
Authorship, in this thicker sense, thus also requires answerability. To be able to call work mine has never required that every piece originated with me – most work any person has produced has never been this pure. It only means that I possess enough of the understanding embodied within it to answer for it. Can I defend the argument I make, or debug the system I produced? Did producing this artifact leave me with the understanding that the artifact’s existence implies?
The separation between artifact and maker becomes most stark in education, where producing the artifact is not the ultimate objective. The essay or problem set is a visible remnant of an invisible process. The purpose of this is not to merely generate the answer, but to form the person who can arrive at one on their own.
In the spring of 2026, Brown economics professor Roberto Serrano gave students in his welfare economics course a take-home midterm. The median score was 98% with 40 of 86 students receiving perfect scores. Having taken a class with Professor Serrano, I can attest to my own surprise at the news. Professor Serrano found that many answers were telling of ChatGPT – convoluted proofs which had otherwise very simple alternatives. In the subsequent final, which was in-person, the average cratered to a much more familiar 48.6%.
This scandal can be described as cheating, and for all intents and purposes, it was. But “cheating” doesn’t sufficiently capture the extent of the loss this displays. The students produced answers that advertised an understanding that only a handful could reproduce when the tool was removed. The artifacts advanced while the makers did not.
A number of other economics professors at Brown reported the same pattern. Perfect homework alongside poor tests, and answers to questions which used terminology and methods unlike those taught in class. Brown’s review of generative AI later found that both faculty and students expressed concern about the impact of generative AI on long-term critical thinking, with students even asking for clearer guidance and AI literacy instead of prohibition.
This is perhaps Thamus’s warning taking new shape, the appearance of knowledge is now detached from the process by which knowledge becomes available to the knower. The student becomes a bystander in the formation of their own knowledge.
A preliminary study from MIT offers a more controlled view into this anxiety. Researchers conducted an experiment in which participants had to write an essay with one group writing the essay with no help, another with the assistance of a search engine, and the last with a chatbot. When writing the essay, the chatbot group exhibited the weakest and least distributed patterns of neural connectivity and reported the least ownership of their essays – many struggled to quote their own work. The researchers termed these effects “cognitive debt”. The study has been criticized by other researchers, and remains far from being authoritative about generative AI’s impact on learning, but its findings give a more empirical shape to a recognizable possibility – that someone might produce an artifact while participating less fully in the cognitive cycle through which make it would ordinarily generate understanding.
None of this means that AI must impair learning. Professor Fan’s work suggests a framework in which these cognitive tools can become useful. The best tools do not merely deliver an end product, they sustain the cycle between making, inspecting, questioning, and revising. They allow the learner to manipulate the material, notice what the representation reveals, and use that revelation to ask the next question. These tools can make ideas easier to see, without making their sight intellectually hollow.
AI can certainly be an active participant in such a cycle. If used well, it focuses effort away from execution into intention, judgment, synthesis, and discovery. If used poorly, it collapses that cycle – instruction goes in, an acceptable artifact is produced, and the user accepts it.
The distinction we must draw then is not simply between human and machine production, or even between arriving at a thought and accepting one. The danger is accepting a thought without reconstructing the understanding that would make it answerable – concluding inquiry instead of beginning it.
One way that cycle is kept open is through resistance. This resistance is difficult to ignore when your interlocutor is another human. A person in conversation can work you for an answer. They can contradict you and stubbornly refuse to let your contradiction pass, forcing you to revise an argument. Socrates would make you defend a claim until the claim (or you) broke. My friend CJ told me that this essay was superficially engaging with its ideas when it needed tension and collision. After I had wiped my tears away, I revisited this essay and grappled with it until I felt I had achieved that tension.
AI can be instructed to challenge you, but its challenge is optional. You can regenerate its response, dismiss its objection, or simply instruct it to be more agreeable. It can contradict you, but the onus is still on you to accept that contradiction as binding.
When AI participates in making an artifact, production becomes distributed across contributions that are not always easy to apportion. A user provides some instruction, intention, revision, and approval. The model provides associations, patterns, formulations, and structure. And the line between those differs from person to person and prompt to prompt. The finished text of this essay expresses something I recognize, but did I arrive at it all on my own? Or did I simply accept it? The risk is that production continues while formation stalls.
Cogito, ergo sum. Je pense, donc je suis. I think, therefore I am.
Descartes’ proof rests on a thought experiment. He imagines an evil demon so powerful that it can produce illusions that make him doubt every facet of his existence, including sight, sound, memory, and even logic. But deception requires someone to be deceived. In a world where everything you perceive can be doubted, the act of doubt is all that remains.
The person who enters a prompt still satisfies the cogito. They read, choose, and accept, reject, and doubt. Someone is still there.
But the cogito Descartes gives us is the minimal self. The occurrence of thought can prove that I exist, but it does not tell us what kind of person I become through thinking. This requires a thicker account of the subject. One in which thought does not merely occur in me, but changes what I can perceive, understand, make, and answer for.
The process of making something does not only produce the thing. It also produces the maker. Writing the essay forms the person who can defend it. Working through the problem set forms the student who can recognize the concepts it tests. Building the system forms the engineer who can repair it. Such difficulty is valuable because experiencing it transforms the capacities of the person.
Generative AI introduces a new possibility that Thoth could not account for. The possibility that the artifact can advance while the maker remains still. The essay might become clearer, the proof complete, the code functional but the person whose name appears above it may not acquire the understanding its existence implies. Production and formation begin to separate.
The metaphysical question, then, is not simply where the thought is located. Thought has always extended itself into language, tools, diagrams, books, and other people. The question is what kind of self is formed when the things produced in my name no longer require me to become capable of producing, understanding, or answering for them. Did using AI help me become more capable, or did it merely let me produce the artifact before I had become capable?
Where does one go when they begin to think in prompts?
There is some irony that I only know Socrates (admittedly allegedly) recounted this story only because Plato wrote it all down. Perhaps the irony then will be that this essay will eventually be repeated in the trickle of tokens from an LLM, with nobody there to answer for them.