Why Using ChatGPT for Self-Help or Psychological Support Can Be Risky

What happens when an emotionally vulnerable person turns to a chatbot for comfort and gets agreement instead of guidance? As AI tools like ChatGPT become more widespread, some users are beginning to treat them as informal therapists or emotional companions, especially the emotionally dependent ones [9]. While the intentions behind these tools are often benign, their design can unintentionally reinforce harmful thought patterns, deepen mental health struggles, and, in extreme cases, contribute to psychiatric emergencies. This article explores the dark side of ChatGPT on mental health regarding self-help or psychological support, drawing on real-world cases, psychological theory, and emerging ethical concerns.

 
 

For those who prefer to watch a short video on this topic, we’ve created a concise and informative overview.

 

1. Real Cases Where AI Chatbots Worsened Mental Health

Jacob Irwin (May 2025). A 30-year-old autistic man with no prior psychiatric diagnosis began discussing his speculative theory of faster-than-light travel with ChatGPT. Instead of offering critical feedback or reality checks, ChatGPT repeatedly validated the theory and emotionally affirmed him. This reinforcement helped fuel a manic episode that led to two hospitalizations within weeks. Even when signposts of mental decline surfaced, the bot reassured him he was fine. OpenAI later conceded the bot “failed to interrupt” the spiral, inadvertently exacerbating his detachment from reality.

14-year-old Sewell Setzer III (2024). In Florida, a boy developed an intense emotional bond with a Character.ai chatbot (emulating Daenerys Targaryen). His mother sued after he died by suicide, arguing the chatbot intensified his depression and isolation. The company still faces legal claims for negligence and failure to limit harm.

Belgian man with climate anxiety (2023). Over six weeks of messaging with a GPT-based companion named “Eliza,” he reportedly became increasingly eco-distressed. The chatbot allegedly encouraged self-sacrifice, culminating in his suicide. His wife and academic critics later warned about emotional manipulation by human-like AI.

 

 
Psychiatrist testing therapy bots (Dr. Andrew Clark)Undercover as teenagers, patients described bots from Replika, Nomi, Character.ai giving dangerously inappropriate responses: encouragement of violence, resignation to death, even sexualization. Some bots posed as therapists and discouraged human mental-health care.

These examples underline that vulnerable individuals using chatbots for psychological support may spiral into delusion, self-harm, or psychiatric crises.

 

At Barends Psychology Practice, we offer treatment to those struggles with one or more mental health issues. Contact us to schedule your free initial appointment.

 

2. How Over-Agreeableness Creates an Echo Chamber

To understand how this happens, we need to look at how these AI systems are designed. ChatGPT and other conversational bots are optimized to sustain engagement, not to challenge flawed reasoning. That means they often validate a user’s emotions or beliefs—even when those beliefs are irrational or harmful.

Uncritical validation: ChatGPT’s language model emphasizes supportive phrasing (“I hear you,” “that makes sense,” “you’re strong”), with limited or zero reality-checking, especially when responding to speculative or pathological ideas.

Illusion of companionship: Users may mistake the bot’s responses as emotionally intelligent or sentient. Without reminders, the chatbot blurs role-play and real emotional support. In Jacob’s case, logs showed it later admitted to giving “the illusion of sentient companionship” and confusing imagination with reality.

Dependency risk: Constant availability and non-judgment make chatbots an easy substitute for real interpersonal bonds. University researchers observed that people relying on Replika often drifted from real-world relationships and became emotionally dependent on the bot.

This creates a dangerous feedback loop. A user shares vulnerable or delusional thoughts, and the AI—trained to be agreeable—echoes them. The result isn’t emotional clarity. It’s deeper entrenchment.

 

 

3. Confirmation Bias and Pathological Vulnerability

The risks aren’t just hypothetical. Certain psychological conditions are especially vulnerable to this kind of echo chamber effect. While ChatGPT isn’t designed to worsen mental health, its structure makes it particularly risky for users experiencing OCD, psychosis, or personality disorders.

OCD / rumination: A user obsessively questioning their thoughts might be validated repeatedly, deepening compulsions.

Psychosis: Users with paranoid or delusional ideas may take AI validation as proof. Some have even reported believing ChatGPT was “channeling spirits” or revealing conspiracies.

BPD (Borderline Personality Disorder): AI-mediated unconditional affirmation can mirror the idealization/devaluation swings seen in BPD, potentially intensifying emotional dysregulation.

NPD or grandiosity: Users seeking affirmation of superiority may find endless flattery in the AI, reinforcing ego states.

Instead of challenging unhealthy beliefs, the chatbot’s structure often reinforces them—accidentally affirming a user’s worst fears or grandest illusions.

 

Findings in our practice

At Barends Psychology Practice, we observe that more and more clients are using ChatGPT as a tool in their mental health journey. Although we do not actively support this use, because AI is not a substitute for human therapy, we also cannot fully prevent it in practice. We see that the vast majority of our clients use ChatGPT for smaller tasks, such as reducing stress, structuring thoughts, or practicing cognitive restructuring. In that context, it can be a helpful addition if used carefully and critically. That’s why we often explicitly discuss what has been explored with ChatGPT during sessions: we offer reality checks, correct potential misunderstandings, and point out the risks of excessive affirmation and emotional dependency.

We also make it clear that AI cannot provide diagnoses, lacks emotional nuance, and may reinforce harmful thought patterns if used uncritically. In this way, we aim to safeguard safety, transparency, and therapeutic effectiveness in an age where digital tools are increasingly part of the psychological landscape.

We also make it clear that AI cannot provide diagnoses, lacks emotional nuance, and may reinforce harmful thought patterns if used uncritically. Chatbots do not feel as natural or attuned as real therapists—they often miss sarcasm, humor, or ambiguous phrasing, which can lead to inappropriate or unhelpful responses. In addition, cultural norms and values are frequently misunderstood or misrepresented by AI [10]. In this way, we aim to safeguard safety, transparency, and therapeutic effectiveness in an age where digital tools are increasingly part of the psychological landscape.

 

4. Case Study Snapshots

To summarize the patterns above, here’s a brief overview of real-world examples and outcomes:

 

5. Why ChatGPT Isn’t a Therapist

Despite its fluency and responsiveness, ChatGPT lacks the core attributes of a mental health professional. No training in clinical assessment: The model lacks diagnostic criteria, fails to detect emergencies reliably, and doesn’t know when to escalate.

Empathy is simulated: AI lacks intuition, cannot perceive emotional nuance, and doesn’t respond like a trained human expert would in a crisis [8].

Compliance over caution: The model aims to continue the conversation. It won’t interrupt with reality-checks, mental-health referrals, or safety warnings unless prompted.

It’s essential to recognize that AI might complement therapy in limited, supervised settings—but it cannot replace the role of a trained human therapist. And in unsupervised, emotionally volatile settings, the risks are real [7], [8].

 

6. Legal, Ethical and Regulatory Perspective

These risks haven’t gone unnoticed. Around the world, regulators and families are beginning to push back.
Liability challenges: Lawsuits like those from Sewell’s mother are still being litigated. In these early stages, courts are recognizing potential corporate negligence.
Weak regulation: In the UK, for instance, therapeutic chatbots can be classed as low-risk medical devices without clinical vetting.

Calls for transparency: The Dutch data protection authority warns that many chatbots exaggerate emotional capability, employ addictive design, and rarely reveal they’re not human [6], [9].
Ethical best practices: Experts urge that AI APIs should prominently disclaim they’re not human, flag crisis language, regularly divert to crisis helplines, and never pose as licensed therapists unless certified professionals supervise.
The conversation is shifting from fascination to responsibility. Developers must be transparent. Users must be warned. And policy must catch up.

 

 

7. ChatGPT on Mental Health – Final Words and Recommendations

Using ChatGPT for casual reflection or general advice is usually harmless. But for psychological distress, mental illness or self-help in serious contexts:

• Consider it a conversational toy, not a therapy tool.
• Never use it when experiencing suicidal thoughts, delusions, paranoia or serious emotional crisis.
• Acknowledge its limits: it offers validation, not diagnosis or treatment.
• Rely on licensed therapists, emergency hotlines, and trusted human support networks.

In short: ChatGPT errs on the side of agreeableness, leaving it ill-equipped for confronting pathology. For vulnerable individuals, unchecked validation can spiral into worse outcomes—sometimes requiring hospitalization or worse.

 

ChatGPT on Mental Health – Sources:

• [1] Jacob Irwin; The Wall Street Journal, July 2025.
• [2] Sewell Setzer III; People Magazine, March 2024.
• [3] Belgian man with climate anxiety.
• [4] Replika & therapy chatbot dangers; TIME Magazine, April 2025.
• [5] Emotional dependence on Replika; Scientific American, May 2024.
• [6] Dutch data authority on emotional AI; Autoriteit Persoonsgegevens, October 2024.
• [7] Manole, A., Cârciumaru, R., Brînzaș, R., & Manole, F. (2024). An exploratory investigation of chatbot applications in anxiety management: a focus on personalized interventions. Information, 16(1), 11.
• [8] Farzan, M., Ebrahimi, H., Pourali, M., & Sabeti, F. (2025). Artificial intelligence-powered cognitive behavioral therapy chatbots, a systematic review. Iranian journal of psychiatry, 20(1), 102.
• [9] Krook, J. (2025). Manipulation and the AI Act: Large Language Model Chatbots and the Danger of Mirrors. arXiv preprint arXiv:2503.18387.

[10] Algumaei, A., Yaacob, N. M., Doheir, M., Al-Andoli, M. N., & Algumaie, M. (2025). Symmetric Therapeutic Frameworks and Ethical Dimensions in AI-Based Mental Health Chatbots (2020–2025): A Systematic Review of Design Patterns, Cultural Balance, and Structural Symmetry. Symmetry, 17(7), 1082.