A common criticism of standardised measures is that they don’t always capture the changes that have taken place in therapy. The fascinating and moving story of a client named Cora is a case in point. The question is, what lessons should we draw from it?
Some months ago, I was contacted by a mental health researcher and writer by the name of Elitsa Dermendzhiyska. Ellie was in the process of writing a piece on what makes therapy effective and had found the Therapy Meets Numbers blogs in the course of her research. A little later we met, and I shared my views about what I had learned over the years, from both research and practice, about what works in therapy.
Just before Xmas, I received an email from Ellie with a link to her finished article. She explained that her research had taken ‘some unexpected turns’ and that she had ended up with a focus on attachment in the therapeutic relationship. Ordinarily I probably wouldn’t have read the article had I just happened across it under my own steam. But I’m very glad I did read it, because I tells a quite captivating story of a fascinating client/therapist relationship.
Cora’s story
What is it about expert therapists that helps clients to change? This was the question that a team of researchers from the University of Oslo set out, in 2006, to try and establish. Led by Professor Michael Rønnestad, the team followed 50 client-therapist pairs in minute detail, attempting to identify what it was that made the therapists so effective. Extensive qualitative and quantitative data were collected throughout the study for each pair, as outlined in the panel below.
Data collected in the study
Qualitative data
Post-therapy interviews: independent semi-structured interviews with clients and therapists were conducted by two different interviewers within a month of completing therapy
Session evaluations: a three-item modified version of the Helpful Aspects of Therapy form was completed immediately after each session by both client and therapist.
Audio-Recordings and Session Transcripts: Every session was audio-recorded. Based on post-therapy interviews and session evaluations of both participants, sessions of particular note were selected and transcribed verbatim.
Quantitative data:
The 12-item Working Alliance Inventory— Short Revised (WAI-SR) was completed by therapist and client after sessions 3, 6, 12 and 20, and then at every 20th session.
The 45-item Outcome Questionnaire (OQ-45.2) was completed by clients after session 1, and then after sessions 3, 6, 12 and 20. From there at every 20th session, and finally after the last session.
The 64-item Inventory of Interpersonal Problems (IIP-C-64) was administered with the same frequency as the OQ-45.2.
Margrethe Halvorsen was the post-doc given the task of interviewing the clients at the end of their therapy to learn more of their experience. This was how she met Cora, whose case is profiled in Ellie’s article. A woman in her 40’s, single and childless, Cora had suffered repeated sexual abuse at the hands of her mother and her mother’s friends. Prior to therapy Cora had habitually self-harmed and made several attempts to take her own life. It was a pre-condition of Cora’s for entering into therapy that she be allowed the right to take her own life if she chose.
Interviewing Cora at the end of her three years and 121 sessions of therapy, Margrethe Halvorsen asked her to describe her experience of therapy in a picture or a word. Cora’s response was simply to say: ‘It saved my life.’
Halvorsen was so intrigued that she invited three colleagues to help her to explore Cora’s case in greater depth and discover what had happened during her therapy. Following initial interviews with Cora and her therapist, they worked their way through the 242 summary notes the pair had written after each session, and from this selected and transcribed verbatim 25 sessions that seemed of particular significance. That’s an eye-popping level of detail, which is profiled in a paper which I can thoroughly recommend.
Subjective Experiences of Change
Cora’s therapy was an undoubted success. After 121 session over three years, her therapy was concluded. As Ellie says in the conclusion to her article: “Cora was by no means cured by her therapy: her trauma ran too deep. But she was saved. She was ready to live and to keep healing.”
As well as being alive and ready to live more fully, the authors of paper profiling Cora’s case identify three key areas that were central to Cora’s experience of change.
Being able to trust another human being
Expanding her ability to relate to others
Increased ability to tolerate uncomfortable feelings
For me it was essential that he accepted my ultimatum and my right to kill myself. The same goes for his attitude towards self-harm. He understood my need for hurting myself and he sustained it. All this shame! And when I finally could show it to him, he was not condemning or frightened. This made it possible for me to build trust.
Yes, that is true, and that was exactly what I told her (her friend). Because she has reminded me about it several times, that I am not as alone as I believed all the time. I do have someone. … And especially regarding my sister, you have helped me to realize that she is there. …
Well, it is clear that when I look back, I have…achieved some very big changes. I have to say…because I do not dare to say it with total confidence, but more like that I also have the right to… [inhales] rejoice at things…without immediately feeling guilty about it or put it aside. I dare to hold on to that feeling a bit longer now. …
Key themes in Cora’s therapy
Analysis of the therapy process revealed a number of critical themes which contributed to her successful outcome.
1. Persistence
Both client and therapist talked about how they had needed to persist in their efforts, even through periods when it appeared that little was being achieved. This included persistence on the part of the therapist in consistently reformulating Cora’s shame, self-hatred and self-harm:
There was a rhythmic interaction pattern that took the form of the client in a sense repetitively singing the same kind of verse throughout the whole therapy – a narrative of shame, self-hate, lack of courage, the fear of losing her love for mother as she has to explore hate and anger towards her. The therapist responded through his chorus: a repetitive song of reformulation about seeing the child’s need for love and protection, asking her to look for survival and dignity, offering de-shaming statements.
2. Courage
Both Cora and the therapist used the word courageous to describe their own and the other’s actions and attitudes. On Cora’s part, the courage to immerse herself in the process, take small steps and come to learn to trust the therapist. On the therapist’s part, the risk of involving himself in a therapeutic relationship with an actively suicidal client, often stepping outside of a traditional therapeutic role, but always authentically present:
I have made use of everything I know about being authentically present and trustworthy. To follow up what I say, to mean what I say, dare not to have an answer, dare to give an answer, and dare to challenge her. I have the feeling that I have achieved my very best in most areas I know of.”
3. Symbolic confirmation
A number of events took place throughout the therapy process that seemed to symbolise and reinforce the formation of “an enduring relationship of trust.” I describe one of these below.
The image at the top of this blog shows a carabiner. This is a specialised shackle used in mountaineering to enable climbers to remain securely attached to one another. Cora introduced the idea of a carabiner in her penultimate session as a symbol of trust as she looked to the future and connecting more securely to family and a particular friend. In her final session she gave an actual carabiner as parting gift to her therapist. I find it hard to imagine any object that better symbolises what lies at the heart of the therapeutic endeavour.
On the standard measures used, how much did the needle move?
As I highlighted earlier in this blog, a range of quantitative measures were used throughout the therapy process, including the OQ-45.2 and the IIP-C-64.
Given the very significant changes that Cora was able to make as a result of her therapy, you might expect those changes to be reflected in her overall change scores.
How much did Cora’s scores on these measures change? The answer (somewhat frustratingly for me I admit), is not at all, or at least not significantly. Pre-therapy, on the OQ-45.2 (scoring range 0 – 160; clinical cut-off 63/64), Cora scored 99.2, and post-therapy 99.0. On the IIP-C-64 Cora’s opening score was 1.75 (clinical cut-off 1.37), and her closing score 1.70.
A common criticism of standardised measures is that they don’t always capture the changes that have taken place in therapy. On the face of it, if we were to simply to use the overall scores here to determine whether therapy was ‘effective’, the criticism might be valid.
Should we junk measures entirely?
Given the criticism that the change scores from standard measures may not always reflect the actual changes that take place in therapy, should we be abandoning them entirely? Personally, I think not, but I do think we need to be a little more sophisticated in how we use them. I have three main strands of thought on this.
There’s a danger that we focus more on the exceptions than the norms. In my experience, there is usually a good degree of fit between change in measure scores and clients’ subjective experience of change.
Condition specific measures such as the GAD-7 (anxiety) and PHQ-9 (depression) seem patently unsuitable for capturing the wider changes that clients are seeking to make or indeed achieve through therapy. We need to use measures that are appropriate to the therapy that we’re providing.
Overall change scores, whether clinical, reliable or otherwise, shouldn’t be the end of the story. Measures are comprised of individual items. If we look ‘under the hood’ we may find significant shifts in single items or clusters of items (think of the four CORE-OM clusters: Wellbeing; Problems/symptoms; Functioning and Risk).
I suspect that a closer inspection of the single items, or item clusters, within the OQ-45.2 and the IIP-C-64 would reveal some of the shifts I describe above. If I think of the measure I know best (the CORE-OM) in the context of the changes that Cora was able to make, I can identify a number of items that I imagine would have registered movement:
I have felt terribly alone and isolated (item 1)
I have felt I have someone to turn to for support when needed (item 3)
I have felt able to cope when things go wrong (item 7)
I have thought of hurting myself (item 9)
I made plans to end my life (item 16)
I have felt overwhelmed by my problems (item 17)
I have felt warmth or affection for someone (item 19)
I have thought it would be better if I were dead (item 24)
I have thought I have no friends (item 26)
Unwanted images or memories have been distressing me (item 28)
I have thought I am to blame for my problems and difficulties (item 30)
I have hurt myself physically or taken dangerous risks with my health (item 34)
In Cora’s case, I’m sure that there would have been positive change on some items and negative change on others. In the end these seemed to have evened each other out. In addition, three years is a substantial period of therapy. Cora was changed, and as she changed it’s possible that shifts in her perception of herself and the wider world will have affected the way she interpreted and scored the measure items.
Reading Cora’s story has been a salutary reminder that change doesn’t always show up in overall measure scores. Rather than throwing the baby out with the bathwater, however, I suggest we should attempt to be a little more discriminating in their application and interpretation.
Ellie – thanks for a great article, and for bringing Cora’s case to my awareness. I’ve found it really helpful to reflect further on the subject of why change doesn’t always show up in the numbers!
As ever, your thoughts on these themes are most welcome!
Leave a comment
We thrive on feedback, and we’d love yours. Please leave your thoughts on what you’ve read in the comments section below.
Subcribe to the bulletin
If you haven’t already, subcribe to our regular (approximately fortnightly) bulletin for information about our latest blog and other news. You’ll find details here:
I have for a long time been frustrated with Mental Health Services moves to co-opt “recovery”
Use of standardised outcome measures is a prime example. “recovery” was defined as the personal experience of individuals. standardisation is about removing the uniqueness of the experience and an attempt of measuring whats seen as important to the “professional”
It can be a form of assault.
There may be occasions when an individual says I have these symptoms as per PQ9 etc. and I’d like to reduce them, one could respond with I have this form would it be useful to use it to see how we are doing? How often does that happen?
In particular IAPT has done untold damage to the original principles of “recovery” and it can be compared to some of the worst case instigators of abuse in history.
Hi Michael and thanks for your heartfelt thoughts. I agree with you – we simply can’t tolerate a situation where users are required to fit around services. This simply compounds any original damage.
I feel very humbled reading about Cora’s life. I also feel that she was VERY fortunate that she met that therapist – it is very difficult to work with a client for a length of time when the measures show nothing is happening.
and of course, for you here, that’s the point of this post. My was of dealing with this, when it has happened with (primarily) depressed clients, was to look at measures of relationship attachment. I found them difficult to find, eventually coming across the HAq11. We did very well on that, which was very reassuring.
http://www.med.upenn.edu/cpr/assets/user-content/documents/Luborskyetal1996_HAQ-II.pdf
While I’m on, what about having an open thread, rather like the ones Scott Alexander keeps at SlatestarCodex https://slatestarcodex.com/2020/02/05/open-thread-146-75/
Occasionally things come up which I would like to raise, such as the research on Mentalisation Based Therapy (which opines that therapeutic movement in all therapies depends on the extent to which the client feels understood by the therapist, and seeks to leverage that].
Hi David, I must say I felt humbled by the pair of them! Thanks for the suggestion of the open thread. I’ll look into the resource that you mention. Cheers, Barry
Here! Here!. I work as a person centered therapist in Belfast, dealing with suicide, its aftermath and with addiction. I agree that measures are often a blunt tool compared to client perspective and story, in ascertaining effectiveness of therapy. What is also highlighted for me in this article and resonates with my own experience within client work is the nugget of truth that nothing is cured, but rather a strengthing of and increased awareness for the client occurs. An awareness of self and in relationship to others as well as more understanding of the trauma, introjects and self beliefs that pull against a more complete life.
Hi Nelson and thanks for putting your thoughts so eloquently. It’s unfortunate that stories of change don’t cut it so much with purchasers and commissioners. Maybe the trick is to weave those together into a coherent whole. Numbers may convince, but I don’t think they can move us like stories. I’ve long thought we need a combination of both.