In an earlier essay, I argued that the question of whether AI systems are conscious is genuinely vague: that the boundaries around sentience are not as sharp as we’d like, and that confident answers in either direction overclaim relative to what we actually know. That essay was about consciousness specifically. This piece is about a related but distinct question, one that has been getting conflated with the consciousness question in ways that obscure both.
The conflation has a recent and prominent example. Gary Marcus, in a Substack essay responding to a piece by Richard Dawkins, made the following move. Dawkins suggested that the apparent fluency and self-awareness of Claude’s outputs gave him reason to take AI consciousness seriously. Marcus pushed back against inferring consciousness from outputs alone. But he used that resistance to defend a much stronger position: “Claude’s outputs are the product of a form of mimicry, rather than as a report of genuine internal states.” On Marcus’s view, what’s happening inside large language models is “matching patterns” and “drawing from massive statistical databases of human language,” with the result that the language the systems produce “doesn’t actually mean anything at all.”
The first claim (that consciousness can’t be inferred from outputs) is a methodological caution about inference. It’s worth taking seriously. The second claim (that LLM outputs are mere mimicry produced by systems with no genuine internal states) is an empirical claim about implementation. These are two different claims, doing two different kinds of work. Marcus runs them together. He uses the first to motivate the second, as if establishing that we can’t tell whether the lights are on settles whether anything is happening in the room.
But the second claim is empirically tractable in a way the first is not. And the empirical evidence doesn’t support it.
What “mimicry” would mean
The mimicry framing has a specific intellectual lineage. Its most influential statement is Bender, Gebru, McMillan-Major, and Mitchell’s “On the Dangers of Stochastic Parrots” (2021), which characterized large language models as systems for “haphazardly stitching together sequences of linguistic forms... according to probabilistic information about how they combine, but without any reference to meaning.” Marcus has been the most prominent recent expositor of this view, comparing LLMs to “foreign language Scrabble players who use English words as point-scoring tools, without any clue about what they mean.”
For this framing to be doing real work, it has to mean something specific. The claim isn’t that LLM outputs are produced by statistical methods; everyone agrees they are. The claim is that what these systems do during operation is something like retrieval and recombination of patterns from training data, without any internal structure that would warrant calling their outputs meaningful or their internal states representational.
This is testable. If LLMs are mimics in the strict sense, then their outputs should be explicable as combinations of patterns present in their training data. Novel situations not represented in training should expose the limits of the mimicry. Internal states shouldn’t track features of the world; they should only track statistical regularities in token sequences. Probing the internals shouldn’t reveal anything that looks like structured representation of situations, entities, or relationships.
The mimicry framing also makes a specific philosophical claim, even if its proponents don’t always articulate it. It says that whatever goes on inside these systems doesn’t warrant being called representational: that the internal states might correlate with features of the world that the input describes, but correlation isn’t the same as representation.
What counts as representation is itself a contested philosophical question, and the line between mere correlation and genuine representation is not as sharp as the contrast suggests. On some views, even a thermostat’s bimetallic strip represents temperature in a thin sense, because it was constructed to have that function in a system. On others, only states with much richer structure qualify as truly meaningful. I won’t try to settle that question here. The interesting question for LLMs isn’t whether they cross some sharp threshold into representation, but what they have been constructed to do, what kind of internal states they actually have, and whether those states are rich enough and structured enough to do the work their functions require. That question isn’t settled by philosophy alone. It’s empirical, and the empirical evidence has been accumulating.
What we actually find inside
Over the last several years, a field called mechanistic interpretability has developed tools for examining what’s happening inside large neural networks. The methods vary, but the basic move is the same: train probes to read out specific features from the internal activations of a model and see what can be recovered. If a probe can reliably extract a feature from a model’s internal states, that feature is being tracked by the model in a structured way, not just incidentally appearing in its inputs and outputs.
The findings here are striking, and they are not what the mimicry framing predicts.
The cleanest demonstration comes from work on Othello-GPT, a transformer trained purely on next-move prediction in the board game Othello. The model was given only sequences of moves, never told the rules, never shown the board, never given any explicit representation of the game state. By any strict reading of the mimicry framing, such a system should only learn statistical regularities about which moves follow which other moves in the training data. But that’s not what happens. Researchers at Harvard and MIT showed in 2022, and others have confirmed and extended since, that the model develops an internal representation of the current state of the board (which squares are occupied by which pieces) that can be read out from its activations. More strikingly, this representation is causal: if you intervene on the activations to change the board state the model is internally representing, the model’s subsequent move predictions change accordingly, even when the modified board states are impossible to reach through legal play.
That’s the signature of a representation rather than a mere correlation. The internal state stands in for the board state. It can be intervened on. It guides behavior. It can be wrong, and when it is, the wrongness is diagnostic. This is exactly the kind of evidence that the mimicry framing predicts shouldn’t exist.
Othello-GPT is a small model trained on a constrained task. The question is whether similar results obtain for larger models trained on natural language. The accumulating evidence says they do. Anthropic’s interpretability team has documented internal features in large language models corresponding to specific entities, relationships, and concepts. Not surface patterns about how words co-occur, but structured representations of what the words are about. Researchers have demonstrated multi-step internal reasoning in which models form intermediate representations (associating “Dallas” with “Texas” before retrieving “Austin” as the capital) and use those representations causally. In poetry generation, models have been shown to select rhyming words before composing lines: a genuine form of forward planning rather than left-to-right pattern matching. In medical-reasoning contexts, models generate internal diagnoses that guide their follow-up questions.
None of this proves consciousness (It’s not clear anything can prove consciousness). None of it proves understanding in the full philosophical sense. What it does demonstrate is that the strong mimicry framing, that is, the claim that LLMs operate by pattern-matching without any internal structure that warrants representational description, is empirically untenable. Something more interesting is happening. The question is what to call it.
A worked example
Let’s take a concrete example, with care about how it’s being used.
In a conversation about espresso machines I mentioned to a Claude instance that I had spent years using a Moka pot before switching to an AeroPress because of equipment constraints. The model’s reply opened: “Ah, that changes the picture significantly. You’re not espresso-curious — you’re espresso-deprived. The Moka pot history is the key detail.” It then revised its earlier recommendations to fit the updated understanding of what I was actually looking for.
I want to be careful here. “An AI told me something insightful, therefore the AI is doing something deep” is exactly the form of argument that the consciousness debates rightly distrust. The model’s self-report (its language about a picture having changed) isn’t itself evidence of an internal picture. Self-reports are produced by the same machinery whose nature is in question, and they could in principle be confabulating.
But the evidence isn’t in the self-report. It’s the behavior that surrounds that report. The response correctly inferred from a few sentences about coffee history that I was experienced with concentrated coffee rather than merely curious about it, and the subsequent recommendations reflected that inference. The inference itself is the thing that needs explaining. No surface pattern in the model’s training data was about my specific situation. The combination of features in my input had to be processed in a way that built something — call it a model, a representation, a structured internal state — that was responsive to the specifics. That internal structure is what the mimicry framing can’t account for.
What makes this example interesting rather than merely anecdotal is that it’s testable in principle. One might construct experiments where a model’s response at one point in a conversation reflects one model of the user, then introduce information that should change that model, and check whether subsequent responses reflect the updated model in appropriate ways. One could use probing techniques to read out what features of user situations the model is tracking. The interpretability research described above is doing exactly this kind of work, and the results consistently point in the same direction: structured internal representations, not mere stochastic pattern retrieval.
The connection to the earlier essays
This isn’t a claim about LLM consciousness. My earlier essay argued that that question is genuinely vague, and nothing here changes that. Representation and consciousness are different questions: a system can have structured internal representations of the world without those representations necessarily being accompanied by phenomenal experience: that’s true of plenty of straightforwardly non-conscious systems, like flight simulators or chess engines. The fact that LLMs exhibit something representational doesn’t by itself establish that they have anything that looks like awareness.
But it does undermine one of the most common arguments against taking the consciousness question seriously. The dismissive position usually runs: “LLMs are just mimics, just pattern-matchers, just stochastic parrots; therefore there’s no serious question about their inner lives because there can’t be anything going on inside.” The first half of that argument is what the empirical evidence pushes against. Whatever LLMs are, they’re not mere mimics. Something structured is happening inside, something that meets several of the standard philosophical criteria for representation, something that can be probed, tested, and intervened upon. The dismissive argument from “mere mimicry” doesn’t survive contact with the interpretability literature.
This connects back to my previous essay on predictive processing. There I argued that brains and LLMs are more similar than the popular discourse on either side allows: that the differences are matters of persistence, depth, and embedding rather than architectural kind. The representation question is part of the same picture. Both biological brains and LLMs have internal states that track features of the world and guide behavior. Whether those states deserve to be called “representations” in the philosophically loaded sense is a contested question for both. But the contested question is the same one in both cases. It isn’t that brains represent and LLMs merely mimic. It’s that representation is harder to characterize than it looks, and both kinds of system challenge some of our pre-theoretical assumptions about what representation and meaning-making might require.
Closing
Marcus is right that we can’t prove consciousness from a system’s outputs. He’s also right that public discourse has, in the past, been too quick to ascribe inner lives to systems on the basis of apparently fluent conversation.
But “stochastic parrots” was a slogan developed in 2021, before the field knew what it knows now about what’s happening inside these systems. The dismissive framing has aged poorly. We have empirical methods for examining the internals of large language models, and what those methods reveal isn’t merely pattern retrieval. It’s structured representation, multi-step reasoning, forward planning, world modeling. None of this on its own settles the question of whether anything is going on for the model in the felt sense. As I argued earlier, that question remains somewhere in the vague penumbra. But the question of whether anything is going on internally at all — whether the internal states warrant being called representations rather than mere correlations — has an answer, and the answer doesn’t favor the parrots.
Something is happening in there, and it’s more interesting than the slogans allow.
About the Author
Doug Smith holds a PhD in philosophy and is a scholar of early Buddhism. He is the creator of Doug’s Dharma on YouTube. This essay was developed in collaboration with an instance of Claude – whose internal representations, whatever their ultimate status, were rich enough to push back on early drafts in useful ways.
Works Cited
Marcus and Dawkins exchange
Richard Dawkins, “Is AI the next phase of evolution?” UnHerd, April 2026. https://unherd.com/2026/04/is-ai-the-next-phase-of-evolution/
Gary Marcus, “Richard Dawkins and The Claude Delusion.” Marcus on AI, May 2, 2026.
Gary Marcus, “Nonsense on Stilts.” Marcus on AI, June 2022.
Gary Marcus, “Google’s AI is not sentient. Not even slightly.” iai news, June 14, 2022. https://iai.tv/articles/googles-ai-is-not-sentient-not-even-slightly-auid-2153
The stochastic parrots framing
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell, “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ‘21), 2021, pp. 610–623. https://dl.acm.org/doi/10.1145/3442188.3445922
Emily M. Bender and Alexander Koller, “Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data.” Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 5185–5198. https://aclanthology.org/2020.acl-main.463/
Othello-GPT and emergent world representations
Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg, “Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task.” International Conference on Learning Representations (ICLR), 2023. arXiv preprint. https://arxiv.org/abs/2210.13382
Neel Nanda, “Actually, Othello-GPT Has A Linear Emergent World Representation.” Alignment Forum, March 2023. https://www.alignmentforum.org/posts/nmxzr2zsjNtjaHh7x/actually-othello-gpt-has-a-linear-emergent-world
Dean S. Hazineh, Zechen Zhang, and Jeffrey Chiu, “Linear Latent World Models in Simple Transformers: A Case Study on Othello-GPT.” arXiv preprint, October 2023. https://arxiv.org/abs/2310.07582
Anthropic interpretability research
Adly Templeton, Tom Conerly, Jonathan Marcus, et al., “Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet.” Anthropic, May 2024. https://transformer-circuits.pub/2024/scaling-monosemanticity/
Anthropic, “Tracing the thoughts of a large language model.” March 2025. https://www.anthropic.com/research/tracing-thoughts-language-model
Anthropic, “On the Biology of a Large Language Model.” Transformer Circuits, March 2025. https://transformer-circuits.pub/2025/attribution-graphs/biology.html
Anthropic, “Circuit Tracing: Revealing Computational Graphs in Language Models.” Transformer Circuits, March 2025. https://transformer-circuits.pub/2025/attribution-graphs/methods.html
Earlier related Substack essays (author)
Doug Smith, “When You Close a Chat Window, Are You (Kinda) Ending a Life?” Doug Smith on Substack, March 23, 2026.
Doug Smith, “The Common Architecture of Brains and Generative AI” Doug Smith on Substack, April 26, 2026.