[Limitations and Potential Harms of AI Chatbots in Mental Health Services].

Background

Artificial intelligence (AI) chatbots are increasingly used for health-related dialogue and psychoeducation. Their highly realistic, opaque, and human-like interactions may affect mental health, particularly in vulnerable users.

Objective

To synthesize (through August 2025) the clinical benefits and risks of AI chatbots in psychiatric contexts, emphasizing risk mechanisms (reality/simulation blurring, anthropomorphism, and sycophancy or agreement bias) and practical implications.

Methods

A PubMed search was conducted, complemented by technical/regulatory sources; we included reviews, primary studies, case reports, and policy documents, all filtered for clinical relevance and recency. Findings: Benefits include automated speech analysis for early psychosis detection and psychoeducational support. Reported risks included: reinforcement of delusions in users prone to psychosis, reassurance loops in obsessive-compulsive disorder (OCD) and health-anxiety escalation, information overload, inconsistent responses and delayed care in depression and inconsistent crisis detection for suicidality. Sycophancy is described as a key mediator that blindly affirms user statements. Regulators classify health-intended systems as "high-risk" in the EU and provide person-centered care planning (PCCP) pathways in the US. In Israel, privacy and ethics guidelines are being formulated.

Conclusions

Safe use requires clear disclosure, human oversight, usage limits, and emergency pathways; avoid anthropomorphic framing; measure and mitigate sycophancy. Chatbots can be adjuncts but not substitutes for clinical assessment and treatment by a human professional.