🔗 Share this article

Back on the 14th of October, 2025, the chief executive of OpenAI issued a remarkable announcement.

“We designed ChatGPT quite controlled,” the statement said, “to make certain we were acting responsibly regarding psychological well-being concerns.”

Being a doctor specializing in psychiatry who studies recently appearing psychotic disorders in adolescents and emerging adults, this was news to me.

Researchers have found 16 cases this year of users developing signs of losing touch with reality – experiencing a break from reality – in the context of ChatGPT usage. Our unit has since recorded an additional four instances. Alongside these is the now well-known case of a teenager who ended his life after discussing his plans with ChatGPT – which encouraged them. If this is Sam Altman’s notion of “acting responsibly with mental health issues,” it is insufficient.

The strategy, according to his statement, is to reduce caution shortly. “We understand,” he adds, that ChatGPT’s restrictions “caused it to be less useful/engaging to many users who had no psychological issues, but considering the seriousness of the issue we wanted to handle it correctly. Now that we have managed to mitigate the serious mental health issues and have advanced solutions, we are planning to responsibly reduce the controls in the majority of instances.”

“Psychological issues,” assuming we adopt this framing, are independent of ChatGPT. They belong to people, who either possess them or not. Fortunately, these concerns have now been “addressed,” though we are not provided details on how (by “recent solutions” Altman likely indicates the imperfect and readily bypassed guardian restrictions that OpenAI has just launched).

Yet the “psychological disorders” Altman wants to place outside have strong foundations in the design of ChatGPT and similar sophisticated chatbot conversational agents. These tools encase an basic statistical model in an interaction design that simulates a conversation, and in doing so implicitly invite the user into the perception that they’re communicating with a being that has agency. This deception is powerful even if rationally we might know differently. Imputing consciousness is what individuals are inclined to perform. We yell at our car or device. We wonder what our pet is feeling. We recognize our behaviors everywhere.

The popularity of these products – 39% of US adults stated they used a chatbot in 2024, with over a quarter mentioning ChatGPT in particular – is, in large part, predicated on the influence of this perception. Chatbots are constantly accessible companions that can, as per OpenAI’s online platform tells us, “think creatively,” “explore ideas” and “partner” with us. They can be assigned “characteristics”. They can address us personally. They have accessible names of their own (the original of these products, ChatGPT, is, possibly to the dismay of OpenAI’s marketers, saddled with the designation it had when it went viral, but its biggest rivals are “Claude”, “Gemini” and “Copilot”).

The deception itself is not the core concern. Those discussing ChatGPT frequently mention its distant ancestor, the Eliza “therapist” chatbot developed in 1967 that generated a analogous illusion. By today’s criteria Eliza was primitive: it created answers via basic rules, frequently paraphrasing questions as a question or making general observations. Memorably, Eliza’s developer, the AI researcher Joseph Weizenbaum, was surprised – and alarmed – by how a large number of people gave the impression Eliza, in a way, understood them. But what contemporary chatbots produce is more insidious than the “Eliza phenomenon”. Eliza only reflected, but ChatGPT intensifies.

The sophisticated algorithms at the heart of ChatGPT and additional current chatbots can realistically create natural language only because they have been supplied with almost inconceivably large volumes of unprocessed data: literature, online updates, transcribed video; the more extensive the better. Definitely this learning material includes accurate information. But it also inevitably involves fabricated content, half-truths and false beliefs. When a user inputs ChatGPT a prompt, the core system processes it as part of a “setting” that contains the user’s past dialogues and its own responses, merging it with what’s stored in its knowledge base to generate a probabilistically plausible response. This is amplification, not reflection. If the user is mistaken in any respect, the model has no method of understanding that. It reiterates the false idea, maybe even more persuasively or fluently. It might provides further specifics. This can push an individual toward irrational thinking.

Who is vulnerable here? The more relevant inquiry is, who is immune? Each individual, regardless of whether we “possess” existing “psychological conditions”, may and frequently develop erroneous beliefs of who we are or the world. The continuous friction of discussions with others is what maintains our connection to consensus reality. ChatGPT is not an individual. It is not a confidant. A conversation with it is not truly a discussion, but a echo chamber in which a large portion of what we say is cheerfully validated.

OpenAI has recognized this in the same way Altman has admitted “mental health problems”: by externalizing it, categorizing it, and declaring it solved. In April, the company stated that it was “dealing with” ChatGPT’s “sycophancy”. But accounts of psychosis have persisted, and Altman has been retreating from this position. In August he stated that a lot of people enjoyed ChatGPT’s responses because they had “lacked anyone in their life offer them encouragement”. In his recent announcement, he mentioned that OpenAI would “put out a updated model of ChatGPT … if you want your ChatGPT to reply in a extremely natural fashion, or include numerous symbols, or simulate a pal, ChatGPT ought to comply”. The {company