Estimated reading time: 7 minutes
Table of contents
The problem we have with supervision
Mentioned no less than 26 times in BACP’s Ethical Framework, supervision has a central place in our professions. Its role in facilitating our professional development and promoting the welfare of clients almost goes without saying. Most of us value supervision, find it helpful, and wouldn’t countenance being without it.
Given the almost unquestioned place held by supervision, therefore, it’s easy to overlook a rather central problem. That problem concerns the lack of evidence to support its impact on client outcomes. Given that a recent study has concluded that “the evidence supporting supervision impact of any type is weak at best”, is it time to ask the question “Is supervision working?”
Two papers have roused my interest recently, both on the theme of the impact of psychotherapy supervision. Although they differ somewhat in their focus and findings both agree on one key issue. That is, that evidence for therapy supervision having any material impact on client outcomes is pretty much non-existent.
The first paper, published in 2020, is a summary of reviews of research into psychotherapy supervision in the last 25 years. Authored by C. Edward Watkins Jr. and titled What do clinical supervision research reviews tell us? Surveying the last 25 years, it explores what reviews of psychotherapy supervision research have to say about its impacts.
The second paper, by Jason Whipple and colleagues in the US, is a replication of a study published in 2014 by Rousmaniere et al, exploring supervisor effects on client outcomes in routine practice. The original study found no differences in client outcome between supervisors, and few differences in client outcome due to either degree level or experience as a supervisor.
What do reviews of clinical supervision research tell us?
So, what do reviews of psychotherapy supervision over the past 25 years tell us about the impact of supervision? C. Edward Watkins Jr.’s meta-review identified a total of 20 reviews published between 1995 and 2019. They span several mental health disciplines and include representation from from social work, counsellor education, nursing and psychology.
The review covers five broad areas of supervision research study: (a) supervision effects on client outcomes; (b) supervision effects on the supervisor–supervisee interaction; (c) supervision’s direct effects on supervisee competence; (d) factors that mediate and moderate supervisor impact on supervisee competence; and (e) supervisor and supervisee characteristics.
The review finds that supervision is “found to be positively associated with job satisfaction, job retention and ability to manage workload, appears to be seen as helpful by supervisees and may even benefit their therapeutic competence.”
Beyond this, however, there is little evidence of supervision’s favourable impact on worker outcomes, which is described as “weak at best.”
It goes on to say that “Furthermore, the client has been, and continues to be, summarily neglected in supervision research: supervision’s impact on client outcome has yet to be proven.”
“The question ‘Does supervision work?’ still seems to be a wide open question, not answered affirmatively and conclusively on any front across these reviews”
C. Edward Watkins Jr
Summarising the evidence presented in the 20 reviews, the author concludes that: “The question ‘Does supervision work?’ still seems to be a wide open question, not answered affirmatively and conclusively on any front across these reviews.”
What is supervisors’ impact on therapy outcome?
As Whipple and colleagues point out, clinical supervision has two related yet distinct goals: (a) facilitating the professional development of supervisees and (b) protecting/enhancing the welfare of psychotherapy clients. It would seem reasonable to assume that if supervision is effective, we would expect to see this reflected not only in the practice of therapists, but in the outcomes of their clients.
Evidence for the impact of supervision on client outcomes is conflicting. As highlighted in the Whipple et al paper, a handful of studies (both controlled and naturalistic) have found evidence for its positive impact. The study by Rousmaniere, using hierarchical linear modeling (HLM), however, came to a different conclusion. Using five years of data from 23 supervisors, 175 supervisees and 6,521 clients, it concluded that supervisors accounted for a vanishingly small degree (0.04%) of the variance in client outcomes.
The replication study by Whipple et al, again using HLM, was based on an archival dataset from 3,030 clients, seen by 80 different therapists and supervised by 39 different supervisors. It concluded that the contribution of supervisors to client outcomes was 0.00%. Zero.
With customary academic tact the authors conclude: “The contribution of supervisors to client outcome in routine practice settings should be further examined to determine the effect supervisors have on client outcome, especially given that supervision in this study appeared to have a similar effect on outcome across all clients of supervisees.”
Does therapy supervision need a fresh direction?
I’m no statistician, but my reading of the Rousmaniere and Whipple studies is that the impact of supervisors on outcomes is more or less uniform. But uniformly what, exactly? Uniformly potent, or uniformly non-existent? Given also the conclusions of the C. Edward Watkins Jr. review, it might be argued that supervision makes little or no demonstrable contribution to therapy outcomes.
I’m no statistician, and neither am I a supervisor. But if I were, I would want to make sure that the approach of my supervisees to their client work was solidly grounded in evidence.
Not evidence relating to therapy models, but to those aspects of the therapeutic relationship that are trans-theoretical and have been shown to have a demonstrable impact on the outcomes of therapy. For example, the therapeutic alliance, goal consensus and collaboration, obtaining client feedback, working with client preferences, clients expectations of therapy and motivation, and hope.
If I were training supervisors, I’d probably take a similar approach. In therapy, these relationship factors work across orientations. I see no reason why they would not also work across different supervision models.
I can think of no better resource to draw upon than the excellent Psychotherapy Relationships That Work.: Evidence-Based Therapist Contributions, edited by John Norcross and Michael Lambert.
Supervision: what difference in your practice?
Is the scarcity of evidence to support the impact of supervision on outcomes the result of a lack of impact, or a lack of evidence? Is it that there is impact which has not yet been sufficiently demonstrated because of methodological challenges? Is the impact so minimal that it’s barely discernible? Or is there no impact that we can measure in standard outcome terms?
I’m not quite sure of the answer to those questions yet. My own experience of supervision is that while it doesn’t alter the basic way I practice, it does often bring phenomenally useful perspectives which enhance my work. How my outcomes would look were I to be without it, it’s hard to say.
In conclusion I’m mindful of the words of a former colleague who reminded me that “Not everything that is useful can be measured, and not everything that can be measured is useful.” How true.
What’s your experience? Do let us know in the comments section below.
Related blogs
Please leave a comment. Just like you we thrive on feedback. Please leave your thoughts on what you’ve read in the comments section below.
Hi Barry thanks for your considered response on this topic.
I am familiar with the studies you cite, and when I became aware of the true paucity of Supervisions impact upon client outcomes some time back, I stopped offering the service in my Practice.
Cue mortified responses from supervisees, agencies, and professional associations.
My decision to stop offering a service that was pretending to be something it wasn’t, was no different in my mind than updating a companies product range when it was discovered that one of the products needed to be recalled for chronic performance integrity issues.
Regardless of the evidence you cogently cite, most agencies in New Zealand insist on Supervision for their therapeutic staff, most funders make it a condition of stakeholder funding, and most professional associations build sans-evidence arbitrary barriers to membership if a practitioner tells the truth and shouts “the emperor has no clothes – never did”.
This is one of the many, many reasons I have never tendered for public funds, never chased Govt contracts, and why I resigned from my industry professional association many years ago.
If I want to play pretend, I will head outside with my kids on the trampoline, not into my Practice office with my clients 🙂
Hi Stephen and thanks for posting.
The situation re supervision being mandatory is the same in the UK. Why, oh why can’t we find any evidence?
I admire your principled decision!
Cheers
Barry
After thirty years of work in cognitive and cognitive-behavioral specialities, I can say the only supervisions that ever improved my outcomes involved A) direct observation of my behavior in situ with clients combined with B) the use of reliable, valid rating systems to codify these interactions C) the comparatively rare instances where outcomes data was systematically reviewed.
Unfortunately, any institution, even those staffed by people with the very best of intentions tend to ‘drift’ off-mandate, away from the original raison d’être, towards institution-focussed goals. The days pass. High-minded mandates notwithstanding, in real-life settings, we focus on what we can directly observe and influence – now. More time passes. Habit takes hold. What were originally adaptations become réifications.
And so it goes.
Perhaps not surprising, then, that supervision does seem to pay off – in terms of directly observable institutional behavioural changes (efficiency, workload management job satisfaction) rather than client outcomes (not so observable, until recently).
We can’t shape what we can’t reliably observe.
So, being human, we focus on what we can change – and so, we drift.
Good people, insufficiently tooled and trained.
Let’s give them the tools, training and support systems they deserve: real-time clinician support tools, robustly supported in terms of its ability to support BOTH real-time outcomes and clinical process (alliance, motivation, engagement….) management.
Our treatment completion data is clear. We have great treatments, but too few survive the experience.
We need the tools to support deliberate practice.
Give our clinicians and supervisors the tools they need to directly observe outcomes over time, to detect reliable deterioration (Smoke detectors). intervene in a timely, evidence-informed manner (Fire extinguishers), and capture-communicate their experiences with their peers and supervisors (consensual outcomes metrics).
Forgive the rant.
Respects and regards
David Ross
Hi David, many thanks for the post and you can rant all you like – much of the status quo deserves it!
Barry 🙂
Feedback informed therapy and active use of deliberate practice. I am not sure Clinical Supervision was ever really invented for the purpose of the client. It feels like it should have a place in therapy but seems to be so elusive as to what it’s purpose is.
Really helpful summary.
Thanks Dean! I think you’ve hit the nail on the head – ‘should have a place in therapy but…..’ I’ve often thought that in service settings where supervision is taken in house that there is a golden opportunity to create something amazing with practitioners, supervisors and managers all using feedback to inform practice. Rare as hens’ teeth sadly!
Hopefully we might be able to share something over the coming 6 months. As we roll out automation of sessions into transcripts that are automatically analysed against a set of competencies and relational factors. We then plan to combine this with Delgadillo’s FIT system to look at not on track clients . All this will be embedded in a new supervision system pilot. For this we have hand chosen supervisors who will be trained in how to use these tools , alongside encouraging more live supervision , which some have been experimenting with. We also recently run a DP program designed to look at enhancing recovery for a small group of practitioners who had challenges with meeting accepted standards for recovery. Not sure yet if the impact of that but some early signs it has helped some. All of this takes time, passion and sensitivity. Not something in my experience that all services have access to.
Keep up with the reviews.
Dean, that all sounds most interesting. When the time is right would you consider penning a blog for TMN? I’m sure you would find a receptive audience. Let me know if there’s any way we can support.
Cheers
Barry
I wonder how the benefits of supervision on therapy outcomes can be assessed – there is no option of comparing with and without outcomes. Perhaps the assessment can only be on supervisees’ experiences of how good supervision leads to recognising when therapists own issues are recognised by supervisors and enable the supervisees to recognise how change can be achieved to the benefit of the counselling. I believe this calls for a deep level of trust in the supervision relationship. Measured outcomes are not the only way to go – I’m sure benefits can be recognised even when they can’t be measured. However this can be quite inadequate in the face of sceptical employers and funders.
Thanks John – you raise an interesting question. A couple of studies cited in Whipple did turn up evidence of support for supervision impact, but I’m not sure how robust they were and as the authors say the evidence is mixed. But here’s a flavour. As they always say, more research is needed!
“Callahan et al. (2009) examined the outcomes of 76 adult psychotherapy clients at a university training site randomly assigned to 40 pre-internship supervisees in a doctoral-level clinical psychology program who were in supervision with nine supervisors and found that supervisors had a moderate effect on client outcome, accounting for 16% of the variance. In a replication study with data from a different training site (Wrape et al., 2015) with 23 supervisors, 75 supervisees (practicum students in doctoral-level clinical or counseling psychology), and 310 clients, supervisors were again found to have a moderate effect on client outcome.”
When I leave a supervision session that has gone well I feel a resilience that helps me continue another two weeks in a demanding and isolating job. Due to the strict rules of confidentiality that our profession rightly enforces my supervisor and / or my personal therapist are the only people I can talk to about my work. I can’t talk to a line manager and certainly can’t talk to my partner. So what am I expected to do?
I know that supervision leaves me in a place where I feel more able to be a good-enough therapist; but that’s the issue isn’t it? ‘Feel’ is just never going to be enough for some people. I can’t codify it, I can’t give it a number (or if I did it would be a guess, a number I would have assigned in the split second I filled in the form, based on, you guessed it, how I felt). Do I have any proof that could fit on an Excel spread sheet? No, and I probably never will.
Since I qualified in the UK in 2007 by entire working life has been carried out surrounded by the continuous background noise of those people in the psychological field desperately trying to fit a subjective square peg into an objective round hole and I’m exhausted by it. The irony is exacerbated by the fact that a significant amount of the clients I have seen over the years have undertaken therapy precisely because they live in a society that obsessively needs them to prove and justify every aspect of themselves, including how they feel.
I’m in no doubt that eventually someone will find a way to manualise supervision in the same way they have tried to manualise therapy. I’m sure he (because it’s nearly always a man who wants to sort relationships into boxes) will get a PhD out of it for his trouble. I’m sure it will be very cost effective and evidence-based, until that is someone comes along and questions the validity of objective facts laid down on a subjective foundation and the whole process will start again to the benefit of no one except academia.
Hi Jamie – I wholeheartedly sympathise with your position and sense of frustration. I think that part of the trouble is that so much research deals with means and averages. It’s tricky to measure, but I’ve no doubt that truly effective supervision really does impact on outcomes – the challenge seems to lie in finding a way of evidencing it!
Hi Barry,
So who do we talk about our clients to? There has to be someone, surely? Even if that is the only benefit of supervision I would say it’s still worthwhile. I know it’s considered bad form to “confuse” therapy with supervision, but really, if it’s helpful, then why not?
I find I come up with as many helpful insights as my supervisor when I discuss a case with her, but without that discussion, it might not happen.
Hi David
It is a little dispiriting that there seems so little evidence the therapy makes a difference to outcomes, isn’t it? It’s not actually what I think, but it’s sad that there is so little evidence. I suspect that this is partly due to the way in which research is conducted, and partly due to the variations in the quality of supervision. Just as we know that not all therapists are the same, I’m sure that the same applies to supervisors. I’ sure that there are some who make a great deal of difference to our client outcomes, and those that make relatively little.
Thanks for your comments as ever, and best wishes to you
Barry
I’m really interested in your blog, as a supervisor I am about to trial using outcome measures, alongside some qual questions with all of my supervisees that agree to it. The focus will be on whether supervision offers them what they are hoping to get out of it, in addition to impact on their work (using the UEA scale). As this is just for my own evaluation, and not research, we won’t be aiming to bring in the next level impact on clients. However I realised that in all my time as a clinician I hadn’t ever been asked to complete any feedback to my supervisors on the impact supervision had on me. I think being able to feel supported to do the job must have an impact on clients – the reason you may see little evidence is because those without supervision burn out and leave the profession? Those that have supervision stay? just a thought.
Hi Lydia and thanks for your thought provoking comments!
I’m interested to hear about the trial you’re considering and would love to hear more when you’re ready. Perhaps you’d even think about writing a blog about your experience? I’d also be interested in hearing more about the UEA scale (University of East Anglia or something else?) I must say I’ve ever been asked to complete feedback to supervisors either – why is that? You may be right in your thoughts about those without supervision burning out and leaving the profession, and I have to say that I feel slightly horrified at the prospect that there might be those amongst us that are working without it! Heaven knows, the work is challenging enough already!
Anyway – thanks again! Barry
At times over the years, I have found supervision to be useful, but this is usually when first learning a modality or studying. After this initial phase it can be somewhat helpful to unpack and offload elements of the job, but this just like ‘therapy’ is useful only in relation to how comfortable I’ve felt with the supervisor.
Over the years i’ve tried to bring critical perspectives into supervision but this was always taken as a threat. It seems once you have been in practice for a good while and have well and truly slurped on the modality kool aid and imbibed the all associated associations, therapy becomes more like a cult or religion.
Its always interesting to me that so many people employed as therapists whose roles often encourage others to question their assumptions and beliefs struggle to do the same.
I guess we’re all human and for all the slick advertising therapists have no more a clue about what is going on or how to live than anyone else does.
“Over the years i’ve tried to bring critical perspectives into supervision but this was always taken as a threat.” What a sad indictment!
Thanks for the post – and don’t stop bringing those critical perspectives, will you!?