What’s the difference between an AI-chatbot and a human therapist? Not much, according to a recently published study looking at recent chatbot use by people with ongoing mental health conditions. Like it or not, we seem to have some competition. How should we respond?
Table of contents
Estimated reading time: 8 minutes
This study flew below my radar when it was published in July 2025. Perhaps it was the snappy title – Large Language Models as Mental Health Resources: Patterns of Use in the United States – that failed to register at the time.
In possession of neither genuine understanding nor consciousness, Large Language Models or LLMs can nonetheless, in the words of the study, “imitate empathy, self-awareness, and personal experience.”
Since the release of ChatGPT in November 2022, many others have followed. They include Anthropic’s Claude, Google’s Gemini, Meta’s Meta AI, DeepSeek-R1 and X’s Grok. Their development, both in terms of sophistication and proliferation, has been breathtaking.
The study in question
Back to the study. It set out to examine “patterns of LLM use specifically for mental health support or therapy-related goals among U.S. residents with mental health conditions.”
Findings are based on the responses of 499 participants who indicated they were a) residents of the US; b) that they had interacted with at least one of 15 listed LLMs; and c) answered “yes” to the question “Do you have – or have you had – a diagnosed, ongoing mental health illness/condition?”
Some broad headlines
The study makes for fascinating reading. It shows how LLMs are finding favour with people facing mental health challenges and who are seeking ongoing support. Among the headline findings:
- All participants reported using LLM’s within the past year (the most frequently used was ChatGPT, used by 96.2%).
- Almost half of participants (n=243; 48.7%) had used LLMs specifically for mental health support or therapy related goals.
- Of this subset, nearly two-thirds (63.7%) had also sought mental health support from a human therapist within the past year.
The last point invites an obvious question. How did respondents rate their relative experiences of human therapist and LLM support? To find out, read on.
And some challenging highlights
Three further findings show the scale of the challenge to our profession that LLMs may be coming to represent. They are based on the responses of the 243 respondents who had used LLMs specifically for mental health support or therapy related goals.
They relate to respondents’ perceptions of improved mental health or wellbeing as a result of LLM use, ratings of LLMs empathy and understanding, and their helpfulness compared with human therapists.
- Participants were asked “Do you feel that using an LLM has improved your mental health or well-being?”
Nearly two-thirds (63.4%) answered “Yes”, while 33.7% answered ” Maybe/it is complicated”.
2. Participants were asked “How would you rate the quality of the LLM’s responses in terms of empathy or understanding?” on a scale from 1 (very poor) to 5 (excellent).
30% gave a rating of 5, and 44.4% a rating of 4.
3. Participants were asked “Have you ever received psychotherapy from a human therapist?” 212, or 87.2% responded “Yes”.
They were further asked “If you have had a human therapist, how would you compare the helpfulness of a LLM compared to human therapy?” on a scale from 1 (much less helpful) to 5 (much more helpful).
74.5% rated the helpfulness of an LLM from neutral to much more helpful, with over one-third of ratings being 4 or 5.
Pretty empathic for a piece of code…
To summarise, the participants who had used LLMs for mental health support or therapy related goals experienced LLMs as broadly effective and highly empathic.
Of those that had also had a human therapist, those that favoured the LLM in terms of helpfulness (35.8%) outnumbered those (25.5%) that favoured the human therapist.
The relative helpfulness finding begs some questions. Were the therapists whose clients favoured the LLM doing something wrong? Were the therapists whose clients favoured them doing something right? Were LLM users construing affirmation and lack of challenge (two common observations of AI-chatbots) as helpful? If only we knew.
Like it or not, we have some competition
Whatever the answers, these findings – if they are even remotely representative of a wider population – suggest that we have some competition.
LLMs are only going to get more sophisticated, more ‘empathic’ and more able to mimic real life therapy interactions. As learners, they are probably rather quicker than most of us. They are also available 24/7, anonymous, and either free or substantially cheaper than therapy with a human.
From a consumer point of view I would have to ask ‘Why would I pay a substantial amount for a 50 minute session at the same time each week with a human therapist, when I can have unlimited support at no or low cost, at any time of the day or night?’
So where do we add value?
I’m really not sure that basing our offer on creating a warm, safe and non-judgemental space where we can explore x, y or z is going to cut it for many. I’m of the belief that what potential clients want is a compelling argument why choosing you is going to make a tangible impact on their concerns.
It’s going to help if we have evidence of our impact. Drawing on my evaluation data I’m able to point to three indicators:
- My dropout rates are below average – only one in twenty clients on average fail to complete therapy, and eighty percent of clients show a demonstrable improvement. (image below)
2. My overall effect size (image below) is significantly above the average estimated for therapy in general (Cohen’s d = 0.8).
3. My improvement and effect size rates are achieved using fewer sessions on average than most peers. (image below)
I share these data with you not to blow my own trumpet, but to illustrate that when push comes to shove with AI-chatbots, outcome data might really help. The evidence base for therapy in general shows therapy to have a large effect. The evidence base for therapy-specific bots and LLMs in general shows only small and occasional medium effect sizes. That may change but currently impact isn’t their strongest suit.
How do you stand out?
We operate in an increasingly crowded market where it’s getting harder than ever to stand out. Whether we see our main competition as human or AI, being able to show evidence of our individual impact as therapists can make a powerful statement about the difference we’re capable of helping to bring about.
Building an evidence base does require that we routinely monitor our outcomes, but I don’t see this as its primary goal. It’s been clearly demonstrated that purposefully using the feedback from measures can significantly improve our outcomes. For me, that comes first and always will. The evidence base that emerges from that practice, however, might come to be an increasingly valuable commodity.
We’ll be posting more on these themes in forthcoming blogs, as well as the tools we’re developing to help therapists engage with them. Do stay tuned….
