Estimated reading time: 10 minutes

Doing effective therapy is hard enough on its own. Integrating measures of outcome into the process adds additional complexity. Do it well, and the purpose, meaning and value of using measures is integrated into conversations. Do it poorly, and we hear the crunch of gears and a trail of potential alliance fractures. As this recent study neatly illustrates.

---

---

What goes on behind that door?

It’s not often we get an insight into what happens behind the therapy room door, and how those conversations affect client engagement. A study I came across recently offers a glimpse into the dangers that using outcome measures, when poorly integrated into practice, pose for the working alliance and wider engagement of clients.

From measures being administered as a stand-alone task to a lack of consistency in their administration, and measure data being treated as impersonal numbers rather than an opportunity to explore the client experience, this study is studded with examples of poor practice. More of this below, but first a brief outline of the study and its aims.

Study outline

This study looked at the administration and therapeutic use of sessional outcome measures within and between sessions delivered by phone in Step 2 care in five IAPT services across the North and East of England. Participants were 11 patients and 11 practitioners recruited from those services. Measures were the GAD-7, PHQ-9, IAPT phobia scale and the Work and Social Adjustment Scale (WSAS).

Thematic analysis was used to analyse transcripts of session recordings between September 2018 and July 2019. Sixteen transcripts were selected for analysis, nine from session one and seven from session two.

How not to use outcome measures in therapy

Many of the study’s findings are illustrated using extracts from session transcripts. For anyone versed in integrating measures into their practice, many make painful reading. They show, in example after example, practice in administration and use of measures that falls far short of good practice. So much so that at times I found myself wincing and wondering how those transactions might have felt for the client.

Four main themes emerge from the study (each with two or three sub-themes)

Theme 1: Lack of consistency in the administration of outcome measures

Theme 2: Outcome measures administered as a stand-alone task lacking flexibility

Theme 3: Outcome measures as impersonal numbers

Theme 4: Missed opportunities to use outcome measures therapeutically

Theme 1: Lack of consistency in the administration of outcome measures

Analysis revealed inconsistent wording to present questionnaire items and rating options, presentation of conflicting information and inconsistencies in the rationale presented for using measures.

Often, items were shortened or rephrased. For example, item 6 on the PHQ-9 reads:

This was rephrased by one practitioner to simply ‘feeling bad about yourself.’

Item 3 of the IAPT Phobia Scale asks ‘How much do you avoid certain situations because of a fear of particular objects or activities (such as animals, heights, seeing blood, being in confined spaces, driving or flying)’

Contrast this with the question asked by the Psychological Wellbeing Practitioner (PWP) and the resulting dialogue:

Data from session one transcripts often indicated that measures were presented to clients as something that needs to be done at every session, but without providing a rationale.

Theme 2 Outcome measures administered as a stand-alone task lacking in flexibility

Measures tended to be administered as a stand-alone administrative task, often disconnected from the rest of the session. They were commonly framed as an encumbrance rather than an opportunity to understand clients’ experiences, as the following extracts illustrate:

Clients sometimes provided a narrative description of their experiences in response to an item question and this information was omitted, forgotten or disregarded by practitioners. In the following illustration the practitioner jumps from question three to question five of the PHQ-9 without actually getting a client rating of the first question.

Theme 3: Outcome measures as impersonal numbers

Typically, clients’ responses to the measure items were treated as abstract, impersonal numbers, generally devoid of the background significance that would have given scores, or shifts in scores, personal meaning.

There were only four examples of practitioners looking for explanations behind the total scores. Here, practitioners attempted to crosscheck clients’ experiences with total scores using closed questions such as ‘Does that seem accurate?’, ‘Does that feel right?’, ‘Does that kind of fit with how things have been?’, ‘Is that reflective of how you are feeling at the moment?’.

In only one case was there an example of the practitioner using an open question in Session 2 inviting the client to reflect on what they had been doing between sessions that might explain the reduction in total scores: ‘But what do you think has kind of happened over the last week to kind of have such a big change on your scores?’

Initially the client did not know why their scores were reduced but went on to talk about making changes to their sleep routine, eating healthier, relaxing more and making a conscious effort to feel better, and talking about their problems. Subsequently, they indicated that reflecting together on the measures with the practitioner was helpful.

Theme 4: Missed opportunities to use outcome measures therapeutically

If you have experience of using outcome measures in your practice you’ve probably spotted a range of missed opportunities to extract personal meaning and relevance from clients’ responses to individual measure items and their overall scores.

The study highlights that further opportunities were routinely missed. This extract illustrates the chance missed by the practitioner to explore information provided by the client about medication and their perception of its impact on their sleeping patterns.  

These two final extracts demonstrate the opportunities missed by practitioners to explore client responses to the Work and Social Adjustment Scale (WSAS) and IAPT phobia scale.

Meanwhile, in the client’s shoes…

How do you imagine some of these exchanges were for the client? How would it have been to be asked to reveal highly personal information with no discernible rationale? To try to follow the practitioner’s rambling responses? To offer contextual information and have that ignored? 

I imagine any of those exchanges might have felt somewhere between mildly to moderately discounting, and you won’t need me tell you how toxic that can be for the therapeutic relationship. 

Much as my sympathies lie mainly with the clients, I also feel for the PWP’s involved. Despite the fact that they had an average of three years’ practice experience and were under supervision, they have the air of people thrown in the deep end with little or no preparation, and receiving little by way of ongoing support. That’s pretty unforgivable.

Using measures – Back to basics

If the examples above highlight some basic pitfalls in using measures, what should we be doing instead? Here are my top five principles. 

1. Do a little homework

There’s plenty of research showing what works in therapy, as well as what doesn´t. This includes evidence showing the contribution that routine outcome measurement makes to therapy outcomes. A simple grasp of the basics (plus a bit of practice) is all that´s required to develop a degree of confidence in using measures.

2. Stick to the script

Like them or loathe them, a great deal of care and science goes into the construction of a robust measure. If you’re reading the items to the client, read them in full. If you don’t, then, however the client may respond, they won’t be responding to the item on the measure. You’ll also be undermining the validity of the client’s responses.

3. Offer a coherent rationale

Don’t, ever, ask the client to do anything without providing a good reason. If, during our conversation, you ask me to jump through a hoop, I’d like to know how that’s going to help me. Convince me, then pass the hoop. At its most basic, it’s a respect thing. We’ve even written some guidance to help.

4. Value the client’s responses

Value the client’s responses and make them part of the therapeutic conversation. Hopefully few of us would be as crass as to say “Thanks for completing that form. Now you’ve done that we can get on with the real work.” Might we, however, convey a similar message more subtly? 

5. Use the client’s responses to enhance your understanding of their experience

Above, under Theme 3, are a few of the limited examples where the practitioner has taken the opportunity to crosscheck the client’s score with their felt sense. Better still, there’s an example of the practitioner inviting the client to connect the reduction in their scores with their own actions. 

Here are a couple of further ways we might open up the conversation. Both examples use the measure I know best and use routinely, the CORE-OM. First, a response I might make in trying to understand more about a client´s experience at assessment.

[In response to the CORE-OM item I have had trouble getting to sleep or staying asleep] “I notice that you’ve checked Most or all of the time in response. Can I get you to say a bit more? Would you say the past week is more or less typical?” 

The second example might be typical of a session later in therapy reflecting on a shift in a client’s predominant pattern of self-blame.

[In response to the item I have thought I am to blame for my problems and difficulties] “I notice that your score has reduced from 4 at the start of our work together to 1 now. That means you’ve gone from Most or all of the time to Only occasionally. That’s a big change. Does that feel about right and if so what sense do you make of it? 

Final thoughts

This was a small-scale study of interventions conducted by phone in a particular time period. To what extent its findings are generalisable, either across IAPT, or wider, we just don’t know. If they are, then, for reasons discussed above, that’s a real concern. Clients can be remarkably forgiving of our mistakes, but when we keep on making the same mistake over and over, we’re on borrowed time.

If you use measures in your practice, what do you do to make clients´ responses part of a therapeutic conversation? If you don´t, what do you think of what you´ve read here? Either way, we´d love to hear from you.

---