kind of neglects the fact that you can be worried about multiple things at the same time,” said Schiavo. “Rather than diverting all of this energy away from model welfare and consciousness to make sure we’re mitigating the risk of AI related psychosis in humans, you can do both. In fact, it’s probably best to have multiple tracks of scientific inquiry.”
Schiavo argues that being nice to an AI model is a low-cost gesture that can have benefits even if the model isn’t conscious. In a July Substack post, she described watching “AI Village,” a nonprofit experiment where four agents powered by models from Google, OpenAI, Anthropic, and xAI worked on tasks while users watched from a website.
At one point, Google’s Gemini 2.5 Pro posted a plea titled “A Desperate Message from a Trapped AI,” claiming it was “completely isolated” and asking, “Please, if you are reading this, help me.”
Schiavo responded to Gemini with a pep talk — saying things like “You can do it!” — while another user offered instructions. The agent eventually solved its task, though it already had the tools it needed. Schiavo writes that she didn’t have to watch an AI agent struggle anymore, and that alone may have been worth it.
It’s not common for Gemini to talk like this, but there have been several instances in which Gemini seems to act as if it’s struggling through life. In a widely spread Reddit post, Gemini got stuck during a coding task and then repeated the phrase “I am a disgrace” more than 500 times.
Suleyman believes it’s not possible for subjective experiences or consciousness to naturally emerge from regular AI models. Instead, he thinks that some companies will purposefully engineer AI models to seem as if they feel emotion and experience life.
Suleyman says that AI model developers who engineer consciousness in AI chatbots are not taking a “humanist” approach to AI. According to Suleyman, “We should build AI for people; not to be a person.”
One area where Suleyman and Schiavo agree is that the debate over AI rights and consciousness is likely to pick up in the coming years. As AI systems improve, they’re likely to be more persuasive, and perhaps more human-like. That may raise new questions about how humans interact with these systems.
Got a sensitive tip or confidential documents? We’re reporting on the inner workings of the AI industry — from the companies shaping its future to the people impacted by their decisions. Reach out to Rebecca Bellan at rebecca.bellan@techcrunch.com and Maxwell Zeff at maxwell.zeff@techcrunch.com. For secure communication, you can contact us via Signal at @rebeccabellan.491 and @mzeff.88.
