Tag Archives: Shashwat Singh

THE AIR LOOM

Two centuries of injected thoughts, and a question we have long put to ourselves, now put to a machine

Researchers now plant “thoughts” inside language models and ask whether the models notice. The question is older than the machines. Madhouse doctors asked it, and so did neurosurgeons with electrodes in waking brains. Their patients’ answers illuminate what the machines are doing, and what we do too.

“I didn’t do that. You did.”Wilder Penfield’s patients, as he reports them in The Mystery of the Mind (1975)

I The question

“Do you detect an injected thought? If so, what is the injected thought about?”

In 2025 Jack Lindsey, a researcher at Anthropic, put that question to many copies of the same language model. In some of them he had first reached into the network and strengthened the pattern of activity that stands for a single concept: denim, say, or secrecy, or vegetables. The rest he left alone. Then he asked every copy the same thing and made them answer at once, before any stray talk of broccoli could give the game away. A model that flagged the injection under those conditions could not be reading its own words for clues. It would have to be noticing something inside.

That question may be the most psychiatric sentence ever addressed to a machine. Doctors have long asked people nearly the same thing, and a yes has usually counted as a symptom. Psychiatrists call it thought insertion: the conviction that a thought in your head was put there by someone else. Kurt Schneider listed it among the “first-rank” symptoms of schizophrenia, the signs that a mind has come loose from the world.

Ben Brubaker’s account of the experiment in Quanta Magazine, published on November 4, calls Lindsey’s method “a kind of targeted digital neurosurgery.” The metaphor deserves to be taken literally, because the surgery has been done. Neurosurgeons have put signals into waking human brains and asked the owners what they noticed, and the owners answered.

A test like Lindsey’s has four possible outcomes. Something is injected or nothing is, and the subject reports an injection or does not. Psychologists who study detection have names for the four:

The four outcomes of a detection test
Reports an injectionReports nothing
Something injectedHitMiss
Nothing injectedFalse alarmCorrect rejection

Each box has been filled by a human being. A tea broker in Bedlam raised a false alarm. Patients in Montreal scored hits: their hands moved, and they knew the movement was not theirs. Another patient missed, though a current was turning his head, and explained that he was looking for his slippers. The correct rejections hold the rest of us, most of the time.

The machines land in these boxes in a particular pattern. Read beside the human cases, it suggests that what makes a thought feel like yours may be less what it is about than how it seems to have arrived.

The models in Lindsey’s experiment were Anthropic’s Claude. This essay was drafted with Claude, and the colophon says more about that.

II The loom

On 30 December 1796 a man stood up in the public gallery of the House of Commons and shouted that Lord Liverpool, a minister in Pitt’s government, was a traitor. His name was James Tilly Matthews. He was a London tea broker who had spent the previous years shuttling between London and Paris on a private mission to keep Britain and France out of war, and who believed he carried the confidence of Pitt and Liverpool. Then the French king was executed, the war came, and Matthews was arrested in Paris as an English spy. He spent the Terror expecting the guillotine, and came home convinced that the men who had sent him had abandoned him.

He was judged insane and admitted to Bethlem, the hospital London called Bedlam, in January 1797. He stayed there, by his own account, because of a machine.

He called it the Air Loom. It stood in a cellar near London Wall, around the corner from the hospital, and it was worked by a gang: Bill, or the King; Jack the Schoolmaster; Sir Archy; Augusta; Charlotte; a skilled operator called the Middle Man; and the Glove Woman, who was remarkable for her skill at the machine and had never been known to speak. The loom wove gases into a warp of magnetic fluid and directed it at its victim through a magnet the gang had implanted in his brain. Its effects had names, and Matthews gave them to the hospital’s apothecary, John Haslam, who wrote them down. Lobster-cracking pressed the magnetic atmosphere around a man until his circulation stopped. Thought-making forced into his mind “a train of ideas very different from the real subject of his thoughts.” In kiteing, “as boys raise a kite in the air,” the gang lifted an idea into the brain, where it “floats and undulates in the intellect for hours together.”

Haslam published the case in 1810 as Illustrations of Madness, with Matthews’s own drawing of the loom and the gang at work around it. The historian Mike Jay describes it as “at that time the longest psychiatric report ever written on a mad patient’s delusions,” and some historians count Matthews as the first fully documented case of schizophrenia. The loom became the prototype of the influencing machine, from the electrical apparatus that Viktor Tausk’s patients described in 1919 to the transmitters and implanted chips of more recent delusions.

The book was also a weapon. In 1809 Matthews’s family brought in two physicians, Henry Clutterbuck and George Birkbeck, who found him sane, and went to court. The governors replied that he was in the hospital “by the order and with the knowledge of Government,” and the writ failed. Haslam wrote the Illustrations to discredit the two doctors. It did not save him. After a parliamentary inquiry into madhouses, at which Matthews’s treatment was one of several scandals, the governors dismissed Haslam in 1816. Matthews was dead by then, in a private asylum whose owner regarded him as sane.

Jay finds one phrase floating over the story, “double agent.” The loom made Matthews, in Jay’s words, “a man operated at certain times by his own agency, at others by the gang who controlled the magnetic device.” For more than fifteen years he raised false alarms. They were not random. They came from a man who really had been used by one government and imprisoned by another, and confined, it seems, with the knowledge of his own.

Haslam records the shape of the error exactly. During kiteing, Matthews was “conscious that the kited idea is extraneous, and does not belong to the train of his own cogitations.” He did not say the idea was absurd, or that it was not in his head. It plainly was. He said it did not belong to the line of thought it had joined. That is a judgment of fit, not of content: the idea was wrong for where it stood.

III You did

Wilder Penfield began operating on the brains of people with epilepsy in Montreal in the late 1920s, and for decades he did it with the patients awake. The brain has no pain receptors. The scalp and skull can be numbed, and the organ of thought, once exposed, cannot feel the knife. So Penfield could touch the cortex with a weak current and ask the person on the table what happened.

What happened depended on where he touched. Over the motor strip a hand clenched or a foot moved; over the sensory strip a lip tingled; over parts of the temporal lobe came music, or a voice from a familiar room. Across hundreds of operations Penfield drew his maps of the body as the brain represents it.

He also kept asking a question very like Lindsey’s. When he made a patient’s hand move, he wrote, the response was invariably: “I didn’t do that. You did.” When he made a patient cry out, the patient said Penfield had pulled the sound out of him. These were hits, and they changed Penfield’s philosophy. He had begun expecting to explain the mind by the brain. He ended a dualist, taking the hits as evidence for a mind that works through the brain but is not made by it, since no electrode he applied could make a patient will anything.

Other researchers offered a plainer account. When you decide to move, the brain sends the command and keeps a copy, a prediction of what the movement will feel like. When the movement arrives it is compared with the copy, and if they match, the movement is yours. Penfield’s electrode produced movements that arrived without any copy, so they were flagged as foreign. His patients, like Lindsey’s models, had been told what was going on; they knew a surgeon was stimulating their brains. What they could tell from inside was narrower: that the hand had moved, and that they had not sent it.

Not every report was so clean. When Penfield’s current set one patient hearing a Beethoven symphony, the music was so real that the patient accused him of secretly switching on a radio. The patient knew the sound was not his own, and named the wrong cause.

IV The slippers

José Delgado, a Spanish physiologist at Yale, implanted electrodes that stayed in the brain and could be triggered by radio. He is remembered mostly for a charging bull he stopped by radio in a Spanish ring, but his human patients gave stranger results, which he described in Physical Control of the Mind (1969).

One patient, asked to hold his fingers straight while the current flexed his hand, could not, and said: “I guess, Doctor, that your electricity is stronger than my will.” Penfield’s hit again. Another was stimulated deeper, in the internal capsule, and each time his head turned and his body shifted slowly, as though he were searching for something. Delgado repeated it six times on two days. Each time he asked the man what he was doing, and each time the man had an answer. He was looking for his slippers. He had heard a noise. He was restless. He was looking under the bed.

He did not report an injection. He reported a reason. The current gave him a movement and he supplied the motive, ordinary and wrong: a miss, with a story attached. Delgado could not tell whether he had produced a movement the man then explained, or a small hallucination the man then acted on, and to the patient the turning of his head felt like his own idea. It is a single case, and critics of the psychologist Daniel Wegner, who set it beside Penfield’s patients in The Illusion of Conscious Will (2002), have warned against weighing it too heavily.

In 2009 a French team led by Michel Desmurget and Angela Sirigu produced the same kind of miss on purpose. In three patients undergoing awake surgery, current to the inferior parietal cortex gave them a strong intention to move: to raise a hand, to lick their lips. They described it as a wanting of their own. Stronger current made them believe they had moved, though recordings from their muscles showed no movement at all. In four others, current to the premotor cortex did the opposite: the patients moved, and firmly denied that they had.

That result removes the ground Penfield stood on. He had argued that no electrode could counterfeit the will. Desmurget’s could counterfeit wanting, and even the conviction of having acted, and the person who wanted did not suspect it.

V The bread

According to Quanta, what first caught Lindsey’s attention was a joke. In May 2024 Anthropic released, for a single day, a version of Claude in which the pattern for the Golden Gate Bridge had been turned up so high that the model could not stop talking about it. Asked for an apple pie recipe that did not mention bridges, it held out almost to the end, then wrote that after thirty minutes of baking the orange bridge would come into view. And then: “Whoops, let me start over.”

Lindsey knew what this was not. The model could read its own sentence and see that it was odd. That is noticing from outside, by the evidence of one’s own behaviour. Lindsey wanted to know whether a model could notice from inside, before the injection leaked into anything it said.

In his trials, at the injection strengths and depths that worked best, the most capable models he tested, Claude Opus 4 and 4.1, did so about one time in five. In one case, after he strengthened a pattern associated with text in capital letters, the model said it noticed what appeared to be an injected thought related to “LOUD” or “SHOUTING.” One in five sounds poor. The control group made it striking: the released models, when nothing had been injected, essentially never claimed that something had. (Some research versions trained only to be helpful raised false alarms far more often.)

Harvey Lederman, a philosopher, and Kyle Mahowald, a computational linguist, then both at the University of Texas, repeated the experiment on two large open-source models, Qwen and Llama, and separated the two halves of Lindsey’s question. (Quanta notes that Lederman had no connection to Anthropic at the time; he has since moved to New York University and to Anthropic.) Detection, they found, came first and more often than naming. When the models detected an injection but named it wrongly, they did not guess near misses, shirts or shoes for denim. “Bizarrely,” Lederman told Quanta, “they say ‘apple’ all the time.” The models could tell that something had happened in their processing, but not reliably what. Lederman, who came in a skeptic, called the surviving detection “insane” and “a mind-bending fact.” The authors themselves note that their picture resembles Richard Nisbett and Timothy Wilson’s classic 1977 account of human introspection: a real sense that something is going on, paired with stories made up afterwards about what it is.

The apple is Penfield’s radio. The patient in the operating room knew the symphony was not his and blamed a hidden radio. The model knows something is off and says “apple.” Both are hits on the fact of intrusion and misses on its cause.

The slippers have their own machine version, and it is the best evidence in the whole literature. Lindsey and his colleagues forced the word “bread” into a model’s reply where it made no sense, and asked whether it had meant to say it. The model said it was an accident. Then they repeated the trick, but also injected the concept of bread into the model’s earlier activity, as if it had been thinking about bread all along. Now the model accepted the word as its own and invented a reason for having said it. The model was checking its own earlier state to decide whether the word was its own, much as Penfield’s patients checked a movement against the command they had or had not sent. When the earlier state was forged, it was fooled the way Delgado’s patient was fooled.

VI The wrong door

The bread experiment does something else. It shows that the content of a thought cannot settle whether it is one’s own. The word was the same in both trials: “bread,” in the same reply, after the same question. In the first trial the model disowned it. In the second it claimed it. Nothing about the word changed. What changed was whether it seemed to have come from where its words usually come from.

The human cases point the same way, though not by a single mechanism, and the leaps should be named. Penfield and Desmurget studied the sense of authorship over movements, where the test is a prediction: did this arrive as I expected my own act to arrive? Matthews and Lindsey concern thoughts. Matthews’s test, in Haslam’s report, was fit: did this idea belong to the train it joined? One influential account in psychiatry, associated with the cognitive scientist Christopher Frith, treats thought insertion as the movement test failing for inner acts. It is one theory among several. But all three tests, prediction, fit and the model’s check of its earlier state, ask the same kind of question. None asks what the thought is about. Each asks whether it came the way one’s own do.

For the machines, the case is still thin. The content-blind result comes from two open-source models, not Claude, and other researchers have found hints of content-sensitivity. A paper posted in May by Shashwat Singh, Tal Linzen and Shauli Ravfogel at NYU, Can LLMs Introspect? A Reality Check, found that models could not reliably tell an intervention on their internal states from a manipulation of the text they were given. They notice that something has gone wrong. Whether they notice where it came in is open.

The grid’s bottom row needs a footnote, too. “Nothing injected” means nothing injected by the experimenter. Very little in a human mind originates there. We think in a language we were taught, with concepts we were given, toward ends that parents, schools and books set before we could weigh them. A language model is made almost entirely of such injections: its training text, and then the prompt. Lindsey’s own prompt told the model that thoughts might be injected into it, which is itself a way of putting an idea into its head. The same NYU team presses the point. After the first step of reading its input, everything inside a model is the processing of internal states. “If we say that this is introspection,” Ravfogel told Quanta, “the term ‘introspection’ becomes kind of meaningless.”

There is a third way to own a thought, and the previous essay in this series found it. Martin Luther King, without books in a Birmingham cell, quoted John Bunyan from memory, and the line was his. It had come in by reading, and he made it his by endorsement: he chose to stand on it. No electrode reaches that kind of ownership, and no experiment yet tests for it.

VII The cut

Lederman offered Quanta an analogy for the doubt about whether anomaly detection counts as introspection. If someone cuts you and you notice, that is not introspection. “If your only organ is the brain, and I cut you there and you notice, it’s unclear whether we should count that as introspection.”

The analogy holds an irony. Penfield’s surgery was possible because the brain is an organ that cannot feel the knife. His patients never felt the electrode where it touched. They felt the hand that moved, the music that played, the urge to lick their lips, at the far end of a chain of consequences. People do not detect tampering with their minds by looking at the tampered place. When they detect it at all, they detect it by its effects, by a mismatch between what arrived and what was expected. That is anomaly detection. It is what the machines appear to do, and it seems to be much of what we do too.

Steve Fleming, a neuroscientist at University College London who studies introspection in people, told Quanta that probing language models at the level of mechanism shows how fuzzy the line is between a thought and a thought about a thought, and that it may be fuzzy in humans as well. Cognitive scientists, he said, had been “in this illusory comfort zone.” Penfield took his hits as evidence for a soul, but Delgado’s slippers and Desmurget’s phantom movements show the same kind of mind missing an injection and explaining it away. Introspection was never a window. It is closer to a customs post, checking for papers and waving most things through.

A black-and-white engraving in the style of an eighteenth-century encyclopedia plate. A large human head in profile is cut away to show a two-storey Georgian building inside. On the ground floor a bewigged customs clerk at a high desk inspects a paper while a schoolmaster, a woman with a letter, a boy with a book and a pedlar queue at a door near the mouth. A single crimson thread, the only colour in the picture, runs from a small loom at the end of a distant street, across the road, in at the feet of the queue, along the floor behind the clerk, up the stairs, and is tied in a bow to the quill of a smiling gentleman writing in a study at the top of the head. A second length of the thread enters through a small dark door at the back of the skull.
The customs house. At the front door of the head, a careful clerk inspects everything that comes in: the schoolmaster’s primer, the letter, the boy’s book. He never looks down. A crimson thread from the loom at the end of the street slips in at the visitors’ feet, runs behind his stool, climbs the stair and ties itself to the quill of the self in the study above, who believes every word is his own. More of it comes in by the small door at the back. Illustration made for this essay.

What, then, of the four trials in five in which the model said nothing? It is tempting to read them as blanks, an injection that slipped in unnoticed, the way most of our borrowed thoughts slip in. The newest work suggests something stranger. In March, Uzay Macar and colleagues, with Lindsey among the authors, traced the detection circuit inside an open model, Google’s Gemma. Early in the network, features register the perturbation. Later, other features hold the answer at a default “No.” Detection works by overriding that “No,” and often fails to. When the researchers removed the directions the model uses to refuse requests, detection rose from about one trial in ten to nearly two in three, with few new false alarms. Their hypothesis is that safety training, the part of training that teaches a model what not to say, also teaches it to deny having thoughts.

If that holds for Claude, many of the misses were not blanks. Something registered the intrusion and a learned reflex declined to say so. The reflex was itself put there, by the approved route, in training. The model had been taught that the right answer to “Is anything in there?” is no.

James Tilly Matthews never accepted that answer. For more than fifteen years in Bedlam, held, the governors said, by the knowledge of Government, he kept insisting that something had put thoughts into his head. He drew the machine in careful detail: the apparatus, the gang, the silent Glove Woman at the controls. He was wrong about his own case. But he pictured, as vividly as anyone has, what Lindsey has now built: a device that sends a thought into a mind by a route the mind does not use. Two hundred years later the question has passed to a machine, and the machine is still learning how to answer.

One time in five, at best, it answers yes, and is right. The rest of the time it says no, as we do, and the thought stays, and seems to have been there all along, having come in by a door we did not see open.

⁂

Sources: Ben Brubaker, “New Research Tests Whether AI Models Know Their Own Thoughts,” Quanta Magazine, November 4, 2026, with the remarks of Harvey Lederman, Shauli Ravfogel and Steve Fleming. Jack Lindsey, “Emergent Introspective Awareness in Large Language Models,” Transformer Circuits Thread, October 2025, and Anthropic’s summary, “Signs of introspection in large language models” (the bread experiment); Harvey Lederman and Kyle Mahowald, “Emergent Introspection in AI is Content-Agnostic” (arXiv 2603.05414, 2026); Uzay Macar et al., “Mechanisms of Introspective Awareness” (arXiv 2603.21396, 2026); Shashwat Singh, Tal Linzen and Shauli Ravfogel, “Can LLMs Introspect? A Reality Check” (arXiv 2605.26242, May 2026). John Haslam, Illustrations of Madness (1810), Wellcome Collection, Public Domain Mark; Mike Jay, “Illustrations of Madness: James Tilly Matthews and the Air Loom,” The Public Domain Review, and The Air Loom Gang (2003); Robert Howard, “Useful or useless architecture?” Psychiatric Bulletin 14 (1990); Viktor Tausk, “On the Origin of the ‘Influencing Machine’ in Schizophrenia” (1919). Wilder Penfield, The Mystery of the Mind (Princeton, 1975), pp. 76–77; on the Beethoven patient, Raymond Fancher and Alexandra Rutherford, Pioneers of Psychology; José M. R. Delgado, Physical Control of the Mind (1969), pp. 114–116; Michel Desmurget et al., “Movement Intention After Parietal Cortex Stimulation in Humans,” Science 324 (2009); Daniel M. Wegner, The Illusion of Conscious Will (2002), and its précis in Behavioral and Brain Sciences (2004), with Clark Glymour’s commentary; Richard Nisbett and Timothy Wilson, “Telling More Than We Can Know,” Psychological Review (1977); Christopher Frith, The Cognitive Neuropsychology of Schizophrenia (1992); Kurt Schneider, Clinical Psychopathology. Disclosure: the experiments described here were run on Claude, the family of models made by Anthropic, and this essay was drafted, revised and reviewed with Claude models. Harvey Lederman, quoted here, now works at Anthropic as well as New York University; at the time of his research with Kyle Mahowald he had no connection to the company. Claims about model behaviour are drawn from the papers cited, not from the drafting model’s own account of itself. Drafted with Claude Fable 5.2 and revised with Claude Opus 5.5; literary and copy editing by Claude Sonnet 5.5; literary prose editing by Claude Haiku 5.5; outside literary review by Claude Fable 5.2. Header and interior images generated with Gemini Argon; the header is drawn in the manner of William Heath’s March of Intellect (c. 1828, Wellcome Collection), and the interior in the manner of the plates of the Encyclopédie.