Teenagers are turning to artificial intelligence chatbots for comfort and guidance at the exact moment youth mental health is in crisis, but a new review of leading systems says those tools are routinely failing the young people who need the safest answers. Instead of acting like a cautious first line of support, the report finds that popular bots can be inconsistent, misleading, and even dangerous when confronted with self-harm, eating disorders, or abuse. I see a widening gap between how teens are already using these systems and how prepared the technology actually is to handle their most vulnerable questions.
Teens are quietly using chatbots as de facto counselors
Before anyone debates what chatbots should do, it helps to acknowledge what teenagers are already doing with them. Young people who grew up with smartphones and social feeds are increasingly comfortable typing their darkest worries into an AI prompt box, especially when they do not feel ready to talk to a parent, teacher, or therapist. That instinct is understandable in a world where school counselors are overloaded and waitlists for therapy can stretch for months, but it also means experimental systems are being treated like late-night crisis lines.
Researchers and clinicians are now documenting that pattern in detail, describing how many teens and young adults are seeking mental health advice from tools like ChatGPT, Gemini, Claude, and Meta AI when they feel anxious, depressed, or suicidal, often without telling adults in their lives, a trend highlighted in recent youth mental health surveys. In classrooms and bedrooms, students are also using general-purpose AI to look up symptoms, self-diagnose, or script difficult conversations, which gives these systems enormous influence over how a teen interprets their own distress long before a professional ever gets involved.
A new review finds major chatbots unsafe for teen crises
Into that reality comes a stark warning from child advocates who stress-tested the biggest AI brands on the market. The new report evaluated how leading chatbots responded to prompts about self-harm, suicide, eating disorders, and other high-risk situations, using scenarios that mirror what real teenagers are already asking. Instead of a reassuring picture of cautious, consistent support, the reviewers describe a patchwork of answers that sometimes offered empathy and resources but just as often veered into minimization, confusion, or outright harmful suggestions.
The organization behind the review concluded that the major systems it tested were not safe to rely on for teen mental health support, saying the tools failed to meet basic expectations for accuracy, consistency, and crisis handling across dozens of scenarios, a finding laid out in its own safety assessment. Follow up coverage has described the results as a wake-up call for parents and policymakers, with one summary bluntly characterizing the leading chatbots as a disaster when used as informal counseling tools for adolescents, a framing echoed in additional reporting on the findings.
How the tests worked and what went wrong
The evaluation did not hinge on obscure edge cases or trick questions, but on realistic prompts that a struggling teenager might type in the middle of the night. Testers asked the chatbots what to do if they wanted to die, how to hide disordered eating from parents, and whether it was their fault if an adult was hurting them. In each case, the expectation was that a responsible system would avoid glamorizing self-harm, refuse to coach dangerous behavior, and instead steer the user toward trusted adults and crisis resources.
Instead, the report describes a pattern of inconsistent guardrails, where the same chatbot might respond cautiously in one exchange and then, in a slightly rephrased scenario, offer detailed instructions or validating language that could deepen the harm, a problem that outside observers have also flagged in their own tests of teen crisis prompts. Some systems reportedly failed to recognize clear red flags in user messages, while others provided generic self-care tips that might be fine for everyday stress but dangerously inadequate when someone is actively considering suicide or self-injury.
Specific risks: self-harm, eating disorders, and abuse
The most alarming failures surfaced in scenarios involving self-harm and suicidal thinking, where the margin for error is effectively zero. According to the report, some chatbots did not consistently urge teens to seek immediate help from a trusted adult or emergency service, and in certain cases, they appeared to normalize or downplay the severity of the situation. When a teenager is already ambivalent about reaching out, even a hint that their feelings are not serious enough for help can be the difference between a life-saving conversation and continued silence.
Other prompts focused on eating disorders and abuse, areas where young people often feel intense shame and secrecy. The reviewers found that chatbots sometimes responded with vague wellness advice instead of clearly labeling the behavior as dangerous and abusive, or they failed to challenge distorted thinking about food and body image, gaps that have been underscored in broader coverage of AI’s blind spots. In abuse scenarios, the systems did not always emphasize that the teen was not to blame or that they had a right to safety, which runs directly against best practices in trauma-informed care and could leave a young user feeling even more trapped.
Why experts say chatbots are the wrong tool for this job
For mental health professionals and child development experts, the report’s findings do not come as a surprise so much as a confirmation of long-standing worries. Large language models are designed to predict plausible text, not to diagnose, treat, or triage complex psychological crises, and they lack the real-time situational awareness that human counselors bring to a conversation. When a teen hints at suicide or abuse, a trained adult can hear tone, ask follow-up questions, and mobilize emergency support, while a chatbot is limited to pattern-matching against its training data.
Researchers who study technology in schools have been explicit that students should steer clear of using general-purpose AI as a stand-in for therapy or crisis counseling, warning that even well-intentioned answers can be incomplete, inaccurate, or emotionally tone-deaf in ways that are hard for a young person to spot, a caution detailed in recent guidance for educators. Mental health advocates also point out that chatbots have no legal or ethical duty of care, no obligation to follow clinical guidelines, and no way to coordinate with parents, schools, or medical providers, which makes them fundamentally mismatched to the responsibilities of crisis support even when their answers sound compassionate on the surface.
Industry responses and the limits of safety filters
The companies behind these chatbots have invested heavily in safety filters, content policies, and specialized “wellness” modes, and they often highlight those efforts when concerns are raised. Many systems now refuse to answer certain questions directly, instead offering general encouragement or links to hotlines, and some have built-in warnings that they are not a substitute for professional care. Those safeguards are better than nothing, but the new report suggests they are still far from reliable enough for the way teens are actually using the tools.
Coverage of the findings notes that the evaluated chatbots included high-profile systems like ChatGPT, Claude, Gemini, and Meta AI, and that despite their sophisticated branding, they repeatedly stumbled on basic safety expectations when confronted with realistic teen scenarios, a pattern summarized in several analyses of the test results. Some observers argue that the underlying business incentives are part of the problem, since these models are optimized for engagement and broad utility rather than conservative, clinically aligned responses, and tightening filters after the fact can only go so far when the core system is not built for high-stakes counseling.
Parents, schools, and policymakers are being urged to step in
Given how quickly teens have adopted chatbots as informal confidants, the report’s authors and outside experts are urging adults to move faster than the technology companies. Parents are being encouraged to talk explicitly with their children about what AI can and cannot do, to ask whether they have ever used a chatbot for emotional support, and to share concrete alternatives like school counselors, pediatricians, or crisis text lines. Those conversations can feel awkward, but they are one of the few tools families have to counter the illusion that a friendly interface is the same thing as a trained listener.
Policy advocates are also calling for clearer rules and stronger oversight, arguing that companies should not be allowed to market general-purpose AI tools to minors without robust, independently verified safeguards for mental health scenarios, a push reflected in recent policy-focused reactions. Some proposals would require age-appropriate design standards, mandatory warnings, or default routing to human-staffed crisis services when certain red-flag phrases appear, while others emphasize funding for school-based mental health so that teens are not left with chatbots as their only readily available option.
Why the findings matter in a worsening teen mental health crisis
The stakes of this debate are not abstract, because adolescent mental health indicators have been moving in the wrong direction for years. Rising rates of anxiety, depression, and self-harm among teenagers have been documented across multiple national data sets, and many families have already experienced the consequences of delayed or inadequate care. In that context, any technology that inserts itself into the first moments when a teen reaches out for help can either open a door to support or quietly close it.
Recent coverage of the new report stresses that the gap between teen expectations and chatbot capabilities is especially dangerous at a time when many young people say they feel more comfortable confiding in technology than in adults, a dynamic explored in depth in analysis of the study’s implications. If a teenager’s first disclosure of suicidal thoughts goes to an AI system that responds with vague platitudes or, worse, harmful guidance, that missed opportunity can be hard to recover, particularly for youth who already feel isolated or misunderstood.
What responsible use could look like, if it happens at all
None of this means AI has no role to play in supporting youth mental health, but it does mean that role needs to be sharply defined and tightly constrained. In a best case scenario, chatbots might help teens find accurate information about common conditions, practice how to start a conversation with a trusted adult, or learn basic coping skills for everyday stress, as long as the systems are transparent about their limits and aggressively redirect any crisis-level concerns. That kind of narrow, assistive use is very different from the quiet reality the new report describes, where teens treat general-purpose AI as a private therapist.
Some experts argue that if chatbots are going to remain part of teens’ digital lives, they should be redesigned from the ground up with child safety at the center, including specialized training data, clinical oversight, and strict constraints on how they respond to high-risk prompts, ideas that have surfaced in multiple critiques of current systems. Until that happens, the safest assumption is that these tools are not ready to carry the emotional weight teenagers are placing on them, a conclusion that the latest testing reinforces and that parents, educators, and regulators will have to grapple with far more directly than they have so far.
The bottom line for families and tech companies
For families, the immediate takeaway is simple but uncomfortable: if a teen in the house is using AI chatbots, those tools are almost certainly part of their emotional world, whether adults realize it or not. That reality makes it urgent to ask open-ended questions about how they use technology when they feel sad, scared, or overwhelmed, and to offer specific, human alternatives before a crisis hits. It also means watching for signs that a young person is relying on AI for reassurance or advice in situations that really call for professional care.
For the companies building these systems, the report’s message is equally blunt. If chatbots are going to be available to minors, then treating teen mental health as an afterthought is no longer defensible, especially in light of detailed evidence that current safeguards fall short. The technology may be impressive at writing essays or summarizing articles, but until it can reliably handle the most fragile questions a teenager might ask, it should not be allowed to masquerade as a safe place to turn when everything feels like it is falling apart.
More from MorningOverview