We may be created equal, but when it comes to therapy outcomes, we’re far from it. Just how much do therapist and service outcomes vary, and what contributes to that variance? A recent study provides some answers.
Not every client that enters therapy hopefully can be assured that they will come away from the process with a demonstrable benefit. Outcomes vary – from client to client, from therapist to therapist, and from service to service.
Within our individual caseloads we know that some clients will derive a greater benefit than others. Within services, there can be similarly large variations in the mean level of outcome across the therapist team.
How much of this variance can be attributed (for whatever reason) to therapists? To what extent do we individually vary in our ability to demonstrate a significant impact for our clients? A recent study has highlighted that the most effective therapists have almost double the rate of recovery of the least effective. We outline the details below.
The study outline
The study in question set out to investigate two key areas:
To estimate the size of therapist and service effects on outcomes in a naturalistic sample of clients receiving psychological therapy.
To assess the impact of particular demographic and process variables on the clinic and therapist effects.
The study used a three-level model of service / therapist / client to explore outcomes and the variables that might serve to explain the variance in outcome.
The client level variables were: pre-treatment CORE-OM score, age, employment status, ethnic origin, sessions planned, sessions attended, and the percentage of planned sessions attended. A total of 22 possible explanatory variables were derived from the seven client-level variables, an aggregate of the client-level variables at the therapist and service levels (14 in total) and the sector in which the service was located.
Sample drawn from the CORE National Research Database
26,888 clients, seen by
462 therapists (counsellors, psychotherapists and clinical psychologists) across
30 psychological therapy services in the UK
Data drawn from five sectors: primary care, secondary care, university, voluntary and workplace
Assessment and outcome was based on CORE Outcome Measure (CORE-OM) scores
The mean pre-therapy CORE-OM score of clients was 17.8 (scored from 0 – 40 where 40 is the highest possible level of distress)
And the study found….?
The effectiveness of therapy was determined by pre- to post-therapy CORE outcome scores and recovery rates. At the client level, the post-therapy outcome score was 8.8. This represented a pre- to post-therapy change of 9.0.
Of clients that scored above the clinical cut-off (10 or over) 58.4% reached the criteria for recovery (reliable and clinically significant improvement or RCSI). In other words, they started therapy above the clinical cut-off, ended below it, and also showed the five-point change required to demonstrate reliable improvement.
The mean recovery rate across services was 57.5% and the range between them 23.4% – 75.2%. Startling though the range first appears, a number of the explanatory variables helped to explain this variance.
The mean recovery rate for therapists was 57.3%, and the range across the 462 therapists between 6.7% and 100%. When only data for the therapists who saw 50 or more clients was considered, however, the range was reduced to between 15.5% and 91.1%.
The impact of explanatory variables on outcomes
A number of the variables under consideration proved to be significantly associated with the variance in outcome. The key points and findings are summarised below:
At the outset, in the absence of the explanatory variables, the 3-level model indicated a service effect of 8.2%, which was significantly larger than the therapist effect of 3.2%. In other words, the variation in outcome across services was significantly greater than that across therapists.
Of the seven patient-level variables outlined earlier, five (pre-treatment severity, employment status, ethnic origin, age and session attendance) were found to be significant predictors of outcome. Thus, clients who showed greater severity at intake, were not employed, were of non-White ethnicity, older and attended a lower proportion of planned sessions on the whole achieved poorer outcomes. Sessions planned and sessions attended were found not to be significant.
Intake severity explained by far the largest proportion of the variance in outcomes at service, therapist and client levels. Other variables explained little of the variance at the therapist and client levels.
There were no significant differences between primary care, university, voluntary and workplace sectors compared with secondary care. This model showed that secondary care services were associated with poorer outcomes.
Therapist and service effects in the final model
In the final model the five significant variables outlined at 2. above were controlled for. Taken together, these variables explained 76.8% of the initial service effect. Client severity was the largest contributor, accounting for 29.6% of the variance at a service level. The contribution of client severity and other significant variables is shown in the table below. Controlling for these variables served to reduce the service effect from 8.2% to 1.9%.
Table: Proportion of residual variance from an empty 3 level model explained by each additional fixed effect variable, in order of input to an incremental 3 level model.
None of the variables proved to be significant at the therapist level. In the final model the therapist effect was 3.4%, marginally greater than in the unadjusted model.
Therapist and service recovery rates in the final model
After controlling for the five variables that impacted significantly on outcome, what were the rates of recovery across services and practitioners?
While there was a wide range of recovery rates across the 30 services, for most the differences were not sufficient to be significantly different from the average. Services within this average range achieved an average recovery rate of 55.7%.
Four services (13.3%) were found to be significantly more effective than the average, and 2 (6.7%) to be less effective. The average recovery rate for the more effective services was 69.7%, and for the less effective services 48.5%.
A similar picture was found for therapists, though the difference between those with the highest and lowest rates of recovery was more pronounced. Of the 462 therapists in the study, 18 (3.9%) were significantly above the average range of recovery, and 18 (3.9%) were below. The recovery rate for therapists in the average range was 58%. The recovery rates for those therapists above and below the average were 77.2% and 41.4% respectively.
Within this study, then, the variance in outcome that was apparent at the service level was in large part explained by differences in five key variables: pre-treatment severity, employment status, ethnic origin, age and session attendance. Together these variables accounted for more than three-quarters of the variance in outcome seen between services.
At the therapist level, however, none of these variables proved to be significant. In other words, other factors were responsible for producing the variance apparent in therapists’ outcomes.
While this study may not have identified variables that significantly impacted on the variance among therapists, it was probably never likely to. We know from other studies, however, a range of factors that do impact our outcomes, many associated with the relational aspects of therapy. Previous blogs have explored a number of them, including the therapeutic alliance and attending to client preferences within our therapeutic work.
We look forward to continuing to bring these and more to your attention in future blogs, and thanks for reading!
Leave a comment
We thrive on feedback, and we’d love yours. Please leave your thoughts on what you’ve read in the comments section below.
Subcribe to the bulletin
If you haven’t already, subcribe to our regular (approximately fortnightly) bulletin for information about our latest blog and other news. You’ll find details here: