California lawmakers are pushing legislation that would restrict AI chatbots from providing mental health services, citing safety concerns over untrained systems dispensing therapeutic advice. The bill targets platforms where users increasingly seek counseling through AI tools rather than licensed professionals.
The proposal mandates guardrails on AI systems to prevent them from positioning themselves as therapists or delivering clinical mental health guidance. Supporters argue that AI chatbots lack the accountability, licensure, and training required for mental health work. They point to risks of algorithmic errors causing harm to vulnerable users seeking crisis support or treatment recommendations.
The timing reflects growing adoption of AI mental health tools. Users access platforms like OpenAI's ChatGPT, Google's Gemini, and specialized mental health apps powered by large language models for therapy-like conversations. Some applications explicitly market themselves as therapy alternatives, while others simply enable users to discuss mental health issues with AI agents.
The bill distinguishes between informational content and clinical mental health services. Educational resources about mental health would remain permissible under the proposed law. The restriction specifically targets systems claiming therapeutic capabilities or functioning as de facto mental health providers.
Industry players have raised concerns about overregulation stifling mental health innovation. Proponents counter that without guardrails, vulnerable populations face serious risks. The debate mirrors broader regulatory friction over AI accountability, particularly in sensitive domains like healthcare and mental health.
The Assembly vote represents California's latest move to regulate AI applications. The state has already pursued restrictions on deepfakes, algorithmic transparency, and AI training data practices. If passed, the bill could influence similar legislation across other states and set a precedent for how mental health AI operates in regulated markets.
The measure awaits a full Assembly vote. Its passage would require clarifying enforcement mechanisms and determining how platforms distinguish between permissible informational chatbots and prohibited clinical systems.