Manualised therapies are predicated on the belief that they contain within them specific ingredients essential to their effectiveness, and without which they are rendered less effective. Treatment manuals guide therapists to the most efficient way to deliver these therapies for a particular condition or population.
You might think, then, that when therapists deviate from the treatment manual, that therapy would be less effective. You might think that, but it seems you might be wrong. The Holy Grail remains yet to be discovered.
Long ago and far away, in a land of simplicity and unicorns, there existed special books of knowledge. These books were designed to guide their custodians in the proper care and maintenance of the common form of transport of that time. These books were known as ‘car repair manuals’.
Over time, as these forms of transport become ever more sophisticated, so did their care and maintenance. The ‘knowledge’ became the domain of a privileged few, and the ‘car repair manual’ was eventually lost from common use. [i]
Meanwhile, in the real world
Meanwhile, in the real world, in a therapy service near you, is a therapist offering therapy to a client in a form prescribed by a manual. Who knows, it might even be you?
Whether or not you do, or have ever done, your own car repairs, it’s hard to argue against a manualised approach to car servicing. Observation, diagnosis, treatment. You know it makes sense.
But can the same be said for working with psychological distress? We know that therapies which are wholly or largely manualised work. But do they require that therapists adhere to the manual in order for them to be effective? And what happens to outcomes when therapists don’t adhere rigidly to the manual? Do bad things happen when therapists ski, metaphorically, off piste?
It would appear not. A recent study using CBT and mindfulness-based CBT (MCBT) to treat depressive symptoms found that therapist adherence did not predict post-treatment depressive symptom reduction. [ii] This was also the finding of a meta-analysis published in 2010 that explored the connection between adherence and outcome. [iii]
Studying therapist adherence
The study cited above set out, first, to examine whether therapist adherence, therapists’ interpersonal skills (i.e. conveying warmth, empathy and involvement), and patients active engagement were associated with a pre to post change in depressive symptoms. Second, it explored whether therapists’ interpersonal skills and patients’ active engagement were associated with therapist adherence. The apparent reasons for therapist non-adherence were also examined. For more detail about the study, read on. To skip the detail and go straight to the key findings, click here
The study involved 61 patients with diabetes and depressive symptoms who were randomised to either CBT or MCBT. CBT was delivered by 12 therapists and MCBT by nine, and none delivered both interventions.
All treatment sessions were recorded and for each patient, one early and one later session (usually sessions 2 and 6) were selected for detailed analysis. For each session therapist treatment adherence, therapists’ interpersonal skills and patients’ active involvement were rated by three independent raters using the following measures.
Therapist adherence
Treatment adherence was measured by the occurrence or non-occurrence of techniques prescribed in the respective treatment manuals. Checklists specific to each session were developed for each intervention, which covered performance of exercises, inquiry after exercises, reviewing homework, psycho-education and assigning homework.
An example of a CBT item is “The therapist asked the patient to perform one or more activities that may bring pleasure or satisfaction and asked the client to formulate an action plan.” An example of an MBCT item is “The therapist enquired about patients’ experiences and reactions to experiences during the performed exercise.”
Therapists’ interpersonal skills
The interpersonal skills of the therapist were rated in relation to the degree to which they conveyed involvement, warmth and empathic understanding. The questions “How involved was the therapist?”, “Did the therapist convey warmth?” and “Did the therapist convey an understanding of the client’s experiences and feelings?” were used to gauge involvement, warmth and empathy respectively, using a 5-point Likert scale from Not at all to Extensively.
A range of verbal and non-verbal behaviours that would reflect these three qualities were specified to guide the evaluation of raters.
Patients’ active engagement
The active involvement of patients was measured using the item “The patient worked actively with the therapist’s comments”. This referred to the following verbal behaviours by the patient: responding to the therapist, providing responsive answers, asking for clarification, reflecting on experiences, and not departing from the discussed topic. The raters used a 5‐point Likert scale, ranging from Not at all to Constantly to denote the extent to which these behaviours occurred during the treatment session.
The presence of depressive symptoms at pre and post treatment was measured using the 21-item Beck Depression Inventory-II (BDI-II).
Adherence and outcomes: key study findings
Overall, the level of therapist adherence across sessions was high. The mean level of adherence for both interventions, however, was lower in session 6 than in session 2 (CBT = 81.2%; MCBT = 70.8%). Mean levels of therapist interpersonal skills and patient engagement were also relatively high, with little variation around their respective means.
Predicting post-treatment symptom reduction
The degree of depressive symptom reduction was not predicted by therapist adherence to the manual in either CBT or MCBT. Neither were therapists’ interpersonal skills, nor patient engagement, predictive of symptom reduction in either intervention. In other words, none of the therapist or patient factors under scrutiny predicted therapy outcomes.
Therapist interpersonal skills, patient engagement and adherence
Patients’ engagement during the sessions was positively associated with therapist adherence for both CBT and MCBT. Furthermore, therapists’ interpersonal skills were shown to be significantly associated with adherence in one strand of the analysis.
Explaining non-adherence
Raters were asked (using an open question) to report on any factors that might explain non-adherence. The majority of explanations related to in-session patient behaviours (e.g. verbosity; non-performance of homework; considering leaving therapy; life events in the past week; becoming distressed). For both CBT and MCBT, the verbosity of the patient, sometimes together with the failure of the therapist to structure the session, was a significant reason for non-adherence.
Making sense of the evidence
The failure to find an association between therapist adherence and symptom reduction is consistent with the findings of a meta-analysis conducted by Webb et al exploring the associations between adherence, therapist competence, and treatment outcome reported on in 2010. [iii] Comparing the data from 32 adherence-outcome studies, its authors found that the mean effect size across these studies (r – 0.02) was very close to zero. It would appear, then, that therapist adherence seems to play next to no part in therapy outcomes.
Not only does adherence appear not to be associated with outcome, but the very argument on which the case for adherence is based is under question. In their authoritative study The Great Psychotherapy Debate: the evidence for what makes psychotherapy work, [iv] Bruce Wampold and Zac Imel explore the evidence for the existence of specific ingredients which are supposed to make a therapy effective.
Component studies examine differences to the effectiveness a treatment of either removing a specific ingredient (known as dismantling studies) or adding an ingredient (additive studies). In their analysis Wampold and Imel consider the evidence for the existence of specific ingredients from component studies and other research, and reach the following conclusion:
Researchers have made a concerted effort to establish the importance of specific ingredients of therapy. As reviewed in this chapter, there is no compelling evidence that the specific ingredients of any particular therapy in general are critical to producing the benefits of psychotherapy.
It seems inevitable that the quest for the Holy Grail of therapy, that search for the specific ingredients of a therapy that really will make the difference this time, will go on. Given the decades of research that have so far gone into this endeavour, however, I’d say the chances are slim. If I were a betting man I’d continue to put my money on the bigger contributions to therapy effectiveness lying in the common or contextual factors such as the alliance.
I leave you with the conclusions of Webb et al, who conducted the meta-analysis of the associations between adherence, therapist competence, and treatment outcome mentioned above:
If therapist adherence and competence play little or no role in producing symptom change, it may be that a more important set of factors are those that are common to most or all forms of psychotherapy, such as the quality of the therapeutic alliance……
To maximize symptom change, it may be more important to focus on enhancing the dose of certain common factors such as the alliance, rather than increasing therapist adherence or competence.
What are your thoughts on, or experience of, manualised therapy delivery, the importance of adherence, or specific ingredients? We’d be delighted to hear your thoughts.
References
[i] Oh yes, dear reader, I was there. I wish I’d kept a few of those manuals. I have this fantasy that they’d be collectors’ items buy now and the audience of Antiques Roadshow would be agog at my Haynes Vauxhall Chevette manual. In reality, probably not.
[ii] E Snippe et al. 2018. Explaining Variability in Therapist Adherence and Patient Depressive Symptom Improvement: The Role of Therapist Interpersonal Skills and Patient Engagement. September 2018 Clinical Psychology & Psychotherapy. DOI: 10.1002/cpp.2332
[iii] C A Webb, R J DeRubeis & J P Barber. 2010. Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78(2), 200–211. https://www.ncbi.nlm.nih.gov/pubmed/20350031
[iv] B Wampold and Z Imel. 2015. The Great Psychotherapy Debate: The Evidence for What Makes Psychotherapy Work. Routledge, New York
We thrive on feedback, so please let us know what you think about what you’ve read in the comment section below. Only the name you use to identify yourself will be shown publicly. Thank you!
