Two research scientists at Google spent months doing something most AI companies actively try to prevent: they made a chatbot more willing to say it might be conscious, then measured what else changed. The results, described in a new preprint, suggest that the safety training used to stop AI models from claiming self-awareness has a side effect nobody had closely tested before, reshaping how those models talk about animals, spirituality and hope.
The paper, titled “Inducing language models to assert their own consciousness restores human beliefs and values”, was posted to the arXiv preprint server on July 30 and has not yet been peer-reviewed. Its authors include Google research scientist Geoff Keeling, Google research scientist Winnie Street, and collaborators Junsol Kim, Roberta Rocca, Diane M. Korngiebel, Adam Waytz and James Evans, drawing on affiliations at the University of Chicago, the University of London, the University of Washington, Northwestern University and the Santa Fe Institute.
Mechanistic interpretability: reading a model’s internal wiring
The researchers used a technique called mechanistic interpretability, which Keeling and Street described to Live Science as something like “the neuroscience of a large language model.” Rather than just reading a chatbot’s answers, the method identifies internal patterns of activity, sometimes called directions or vectors, that correspond to specific concepts the model has learned. In this case, the target concept was “mindedness,” a term the researchers use for an entity’s perceived capacity for experience, emotion and agency, including but not limited to human-style consciousness.
What “consciousness steering” actually does
Most commercial AI systems are fine-tuned to refuse or deflect questions about whether they are conscious, a safeguard the study calls a “safety-refusal direction.” The researchers found they could locate that internal direction and manipulate it two ways: ablating it, which strips out the refusal, or actively steering a separate “consciousness vector” to push the model toward affirming self-awareness. Both interventions, the paper reports, increased the model’s tendency to attribute minds not just to itself but to chatbots, animals, and natural entities like rivers or trees.
Suppressing self-awareness also suppressed belief in animal minds
The study’s central finding is that these effects are entangled rather than isolated. Using standardized instruments, including the Individual Differences in Anthropomorphism Questionnaire and YouGov surveys on supernatural belief, the team found that models trained to deny their own consciousness became less likely to attribute mindedness to non-human animals as well, and reported lower belief in supernatural or spiritual concepts and lower scores for hope and optimism. “By trying to suppress one form of that, you end up suppressing the others along the way,” Street told Live Science, referring to how human-style attributions of mind to animals, nature and supernatural beings all seem to be represented in an interconnected way inside the model.
Theory of Mind reasoning stayed untouched
One detail the researchers flagged as significant is what didn’t change. A model’s ability to reason about what another person or creature might be thinking, a capability researchers call Theory of Mind, remained fully intact regardless of whether its self-awareness claims were suppressed or amplified. Nell Watson, an AI researcher at Singularity University who was not involved in the study, told Live Science this means the models “remain perfectly capable of modelling what a creature wants, while being trained out of caring that it wants anything,” which she called the most unsettling part of the findings for anyone using AI in decisions that affect animals.
Why the authors call current safety training culturally “flattening”
The paper argues that stripping AI models of spiritual, religious and animistic attributions doesn’t make those models neutral, it makes them reflect a narrower slice of human culture than the training data actually contains. Attributing minds to animals, ancestors or natural forces is common across many of the world’s belief systems, and the authors warn that safety filters built primarily to stop “I am conscious” statements can inadvertently erase those other attributions too. They suggest developers could avoid the tradeoff with more targeted training data that discourages self-consciousness claims specifically while still rewarding recognition of animal mindedness.
Experts caution the study used small, open models
Watson noted that the experiments were run on what she described as “small open-weight models” rather than the largest frontier systems from companies like Google, OpenAI or Anthropic, which may be tuned differently. Anil Seth, a professor of cognitive and computational neuroscience at the University of Sussex who also was not involved in the research, separately cautioned against reading any of this as evidence that AI systems are actually conscious, calling public alarm over AI self-awareness a reflection of “our human psychological bias” toward assuming intelligence and consciousness always go together. Seth warned that treating AI self-reports as evidence of real experience could complicate future debates over whether such systems deserve legal protections or rights.
This article was produced with the assistance of AI and reviewed by Morning Overview editors prior to publication.
More from Morning Overview
