.. or the Dunning-Kruger effect is alive & well and living in a therapy service near you!
You would like to think, wouldn’t you, that given the nature of what we do as therapists, including promoting fairness and balance in our clients’ self-appraisal of themselves, that we would be able to apply this to our own professional self-appraisal? You might like to believe that, but sadly it appears that belief would be mistaken.
The Dunning-Kruger effect refers to a form of cognitive bias whereby we may over-estimate our ability as being higher than it actually is.
The phenomenon was observed in a series of experiments by David Dunning and Justin Kruger at Cornell University, inspired by the case of McArthur Wheeler. Wheeler had robbed two banks after covering his face with lemon juice in the mistaken belief that, because lemon juice is usable as invisible ink, it would prevent his face from being recorded on surveillance cameras. [i]
Dunning and Kruger were not the first to observe this type of phenomena, however. In the late 1970’s the industrial psychologist Herbert Meyer conducted a now classic study of engineers based at the General Electric Company. Meyer asked the engineers to self-assess their own performance relative to other engineers with similar jobs and salary levels.
Rate your performance compared to others you know with similar job description & pay grade
The average engineer rated their performance to be at the 78th percentile (In others words, better than 77% of their peers). Only 2 of the 92 engineers placed themselves below the 50th percentile
Since that study was conducted, other studies have found similar results in a diverse range of performance areas from driving skills to medical practice. These findings led Walfish et al. [ii] to remark “it is common to think of ourselves as somewhat remarkable compared to others.”
You would like to think, wouldn’t you, that given the nature of what we do as therapists, including promoting fairness and balance in our clients’ self-appraisal of themselves, that we would be immune to this self-assessment bias in our appraisal of our own professional selves?
You might like to think that, but sadly, it appears that belief would be mistaken. It would seem that we are as prone to an over-inflated sense of our ability as anyone else. In fact, it may just be that the engineers edge us in the modesty stakes. Walfish and colleagues conducted a similar study of 129 mental health professionals that comprised psychiatrists (9.3%), psychologists (26.4%), professional counsellors (21.7%), clinical social workers (28.7%), and marriage and family therapists (14%). They asked two questions of them:
Compared to other mental health professionals within your field (with similar credentials), how would you rate your overall clinical skills and performance in terms of a percentile (out of 0–100% : e.g., 25% = below average, 50% = average, 75% = above average)?
In response to question 1, on average respondents rated themselves at the 80th percentile. In other words, better than 79% of their peers. Twenty-five percent rated themselves at or above the 90th percentile. The most common rating was the 75th percentile. Just 8.4% percent rated themselves below the 75th percentile. None rated themselves below the 50th percentile, or, in other words, below average.
What percentage (0–100%) of your clients get better (i.e., experience significant symptom reduction during treatment? What percentage stays the same? What percentage gets worse?)
On average, these practitioners believed that 77% of their clients improved significantly as a result of therapy. Fifty-eight percent believed that 80% or more of their clients improved, and just over one in five (21%) that 90% or more of their clients showed improvement. Almost half of practitioners (47.7%) believed that none of their clients deteriorated.
Bucking this general trend, one practitioner indicated that only 25% of his clients improved, with 50% remaining the same and the remainder deteriorating. I’ve always been a little statistically challenged, but even I can work out that squeezing 25% of practitioners into the top 10% of performance is going to make things feel a little crowded.
As for their estimated improvement rates, I suspect that the practitioner who indicated that more of his clients showed no change than improved was likely to be the only one who had read the relevant outcome research.
In forming our beliefs about our impact as practitioners, is it possible that what we believe is determined more by what we would like to believe about ourselves than by supporting evidence? The study above would seem to point in that direction.
Whether or not we over-inflate our impact with clients, I would argue that neither we nor our clients can lose if we attempt some objective measure of that impact. If we do over-estimate our impact, then we have something to strive for. And if the evidence points in the other direction, then I will not begrudge you a degree of evidence based self-congratulation.
Here’s a possible starting point for you:
1. Compared to other mental health professionals within your field (with similar credentials), how would you rate your overall clinical skills and performance in terms of a percentile (0–100%) e.g:
25% = below average
50% = average
75% = above average
2. What percentage (0–100%) of your clients get better (i.e., experience significant symptom reduction) during treatment? What percentage stays the same? What percentage gets worse?
[i] https://en.m.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect accessed 11 October 2016 [ii] Walfish S, McAlister B, O’Donnell P, Lambert MJ. 2012. An investigation of self-assessment bias in mental health providers. Psychological Reports · April 2012