How can you, and how can we as a profession, demonstrate the value of what we do in a way that shows the effectiveness, efficiency and value for money of the services that we provide? It’s all very well bemoaning the lack of a place at the table when it comes to resource allocation, but if we don’t have our own evidence, we’re not likely to get very far in making our case.

Following our recent blog Is IAPT too big to fail Therapy Meets Numbers subscriber David Solomon set out on a journey to compare his own results with those of IAPT. You can read what he found in his blog How Do My Results Compare With IAPT / NHS Therapy?

Here, we want to show you how you can do the same. Comparing your own data with IAPT nationally is remarkably simple. If you want to have some real fun, however, you can use a very neat tool provided by NHS Digital to drill down to IAPT performance data at a local level. This includes data at Commissioning Region, Clinical Commissioning Group (CCG) and provider levels. More on that in a moment.

Why would I want to compare my data anyway?

As a private practitioner, I have next to no incentive for comparing my performance with IAPT. We’re not in competition, at least as far as I see it. But that’s an individual view which you may not share (please feel free to use the comments section below).

I’m driven by a number of things, curiosity, scepticism and a passion for my profession being among them. I’ve been around long enough to recognise a questionable claim for effectiveness when I see one. And more than long enough to have seen the birth and subsequent growth of IAPT and the resulting carnage in NHS primary care counselling.

In a comment on the IAPT blog David makes his thoughts on IAPT’s performance clear. To say he’s not exactly gruntled would be an understatement. He was prompted to run his own analysis which you can read in his own blog post. He is to be congratulated as he’s clearly doing some excellent work.

In a private communication David told me of his initial motivation for adopting the use of outcome measures. A previous supervisor had apparently questioned his competence, to the extent that he had started to question himself. His CORE outcome data, however, started to show a different and altogether more positive picture. Sadly, this led to the supervisor questioning the validity of the measure itself, a story I’ve heard many times before. I’m grateful to David for allowing me to share his experience with you.

Mapping your client journey

Whatever your motivation, it’s relatively easy to map your own clients’ journeys into, through and out of your service. Below I’ve taken the four stages I used in the IAPT blog and made suggestions how you can simply adapt your own figures.

The journey from referral

For a given and reasonably recent time period  (I’d suggest 12 months) take all the initial client contacts with whom you had some kind of interaction. Include providing further information and an initial assessment if that’s how you work. Don’t include first therapy appointments.

Work out what proportion of those initial contacts had what you would describe as a first session of therapy with you. Compare this to the IAPT blog Stage 1 figure, where 29% of referrals don’t enter therapy.

The journey from entering therapy

Starting with all those clients that had an initial therapy session, work out how many came to what you might describe as a planned ending. My preferred definitions are those used by the CORE System, which are outlined in a previous blog on the topic of dropout.

On the subject of categorising clients as planned or unplanned endings it’s easy to give ourselves the benefit of the doubt. If this describes you, draw up two versions, to include one where you don’t. For a helpful guide the CORE System manual offers relatively precise definitions.

Work out what proportion of clients that started therapy reached a planned ending. Compare this with the IAPT blog figures at Stage 2, where 42% of clients that enter therapy don’t complete.

 The outcome at therapy ending for all clients

If you’ve got this far, you’re doing well. If you also use some form of outcome measure, you’ll now be able to make a comparison of your outcomes with those of IAPT. Be aware, however, that IAPT use three different categories to describe improvements in clients’ GAD-7 and PHQ-9 scores:  recovery, reliable recovery and reliable improvement.

Going into any detail about distinctions between the categories is, sadly, beyond the scope of this blog (but will be the topic of a subsequent one). If you want to know more about the IAPT definitions you can find them in the latest IAPT annual report for 2017 – 18.

In the IAPT blog I’ve used the IAPT’s figures for recovery. Note that recovery in IAPT terms means that clients start therapy above the clinical cut-off level on one or both of GAD-7 and PHQ-9 (at or above 8 and 10 respectively) and finish below on both. If clients are below the cut-off on these measures, they cannot therefore achieve recovery (even if they have otherwise improved).

If you collect this type of outcome data you can compare it with the IAPT blog figures at Stage 3, where 47% of clients achieve recovery. If you use measures other than GAD-7 and PHQ-9 bear in mind that the outcome data from these measures may not be precisely comparable. It will nonetheless give you a decent ballpark figure.

The outcome at therapy ending for clients that were at case level at the start

This stage excludes the recovery data for clients that were below the clinical cut-offs for GAD-7 and PHQ-9 at the start and could not therefore recover.

If you have the means to work out what proportion of your clients that were above case level at the start were below at the end, you can compare this to the IAPT blog stage 4 data, where 51% of clients achieve recovery.

How can I find out how my local IAPT is performing?

You may have noticed that the main blog image contains graphs from two CCG areas: Brighton and Hove and Chorley and South Ribble. Why did I choose those? It’s because Brighton and Hove is my own local CCG, and Chorley and South Ribble is the CCG that covers David’s area. I explain below how you can access data for your local area.

Just as therapists are not all the same, so too with services. There are wide variations across the different IAPT performance indicators at Commissioning Region, Clinical Commissioning Group (CCG) and provider levels.

IAPT used to provide that information in a horrifyingly dense spreadsheet designed to break even the toughest of us. Thankfully, however, most of it is now available in an interactive tool provided by NHS Digital. It takes a few minutes to get familiar with the controls, but the level of detail that is available is seriously impressive. You can find it here.

Clicking on the page counter at the bottom of the tool reveals a page menu. Scroll up and down to find the data you’re interested in. Clicking on Outcome Status (rates) for example, takes you to page 13, where you’ll find data for five outcome categories (recovery, reliable recovery, reliable improvement, no reliable change and reliable deterioration) at a national level, as well as having the option to filter the data at a finer grained level.

Finding data at a local level is straightforward. For example, to find the data for Chorley and South Ribble CCG, select the CCG filter (see below), then scroll down the CCG list to the relevant tab.

Go to page 4 and you can access details of the numbers of clients referred, entering treatment, and finishing a course of treatment. Once again, you can filter by CCG area and provider. Hover a mouse over the bars on the chart (below) and the relevant number is displayed.

A wealth of other data by is available including waiting times, stepped care pathways, therapy type, mean appointments by therapy and so on. One which particularly caught my eye was the Other Activity page (page 8). This includes the mean number of treatment appointments for all referrals that ended (6.8) and the mean for referrals that achieved recovery (7.5).

While this is a relatively small difference, I suspect that if we were to know the mean sessions for clients who do not achieve recovery, we would see a greater difference. This suggests to me that many clients may not be getting the optimal number of sessions to meet their needs.

I’d encourage you to have a play with the tool, it’s rather fun. Happy benchmarking and let us know what you find!