More than ten years on from its launch the Improving Access to Psychological Therapy (IAPT) programme is far from delivering on its targets. Why then, given the astonishing rates of attrition in both IAPT clients and staff, are two of our European neighbours seeking to implement similar models? We re-examine some of the evidence and explore what you see when you only look at a small part of a bigger picture.
The recent translation into Dutch of a book published in 2015 by two of the original proponents of England’s IAPT programme has prompted calls in both the Netherlands and Belgium for the adoption of similar initiatives there.
Closer to home, however, it’s clear that however positive the latest headlines about IAPT may be, all is far from well. Levels of attrition among clients remain high, levels of staff burnout and low morale are at epidemic levels, and services seem to be haemorrhaging staff.
Add to that the fact that Sweden has abandoned its pursuit of a single modality national model, and the interest in replicating IAPT elsewhere starts to look particularly curious. Is there part of the bigger picture that is currently missing?
While you’re here, I have a small task for you.
Your task: Commission a therapy service for your country…….
B: Hello sir, how may I help you?
A: I’d like to purchase a national therapy service please.
B: Of course, sir. What model did you have in mind?
A: Well, I’ve heard that CBT is quite effective?
B: Oh yes, sir, indeed it is. Our model returns a 51% recovery rate. Very impressive performance and a guaranteed return on investment.
A: That’s most impressive. It seems I’ve come to the right place. Does it come with any added extras?
B: Well sir, the latest model comes with a few small extras. CfD, PP and EMDR are all tried and tested and come as standard.
A: It sounds too good to be true! Is there anything else I should know?
B: Well sir, it’s worth being aware that the 51% claimed is based on patients that actually finish a course of treatment and are above caseness level at the start.
A: I see……anything else?
B: Well sir, there is some attrition along the way. All perfectly natural and nothing to be concerned about.
A: How much attrition?
B: Er……42%, sir.
A: 42% of what?
B: 42% of patients that enter treatment only have one session sir.
A: I see. So, of the patients that start therapy, what proportion actually recover?
B: Er…..28%, sir.
A: Only just over one in four patients that enter therapy recover?
B: Yes sir, that’s right.
A: Anything else?
B: Well sir, of the referrals in a year, 29% end without being seen by the service, and 29% have one treatment appointment only.
A: So, of the total referred in a year the proportion that finish treatment is……..?
B: Er…..40%, sir.
A: And the proportion that reach recovery is………..?
B: Er…..19%, sir.
A: Well, I don’t really have anything to compare it to, but given your confidence in the system I’m sure yours is the best model there is. Who should I make the cheque out to?
Is IAPT Thriving?
In 2015, Richard Layard and David Clark published a manifesto for the widespread delivery of empirically supported treatments (EST’s) to the population of England. Thrive is impassioned argument for change in our attitudes to psychological distress and its individual and societal costs. The central proposition is this: spend more money on the large-scale delivery of EST’s such as CBT, and you not only alleviate an enormous amount of human suffering, but the costs will be fully covered by the savings generated. There’s no way round it: it’s a seductive promise.
Thrive has recently been translated into Dutch, and has found its way, not surprisingly, into the affections of CBT adherents in both the Netherlands and Belgium. In June of last year, David Clark presented at a conference in Amsterdam at which the book seems to have been launched. Days later the book and a manifesto were presented to the Secretary of State for Health Paul Blokhuis. The manifesto called for “more and better investments in mental health care” promising that “every euro that is invested in mental health care yields two.”
This coming March, the focus shifts to Belgium, where David Clark will present at a lecture and panel discussion hosted by the Centre for the Psychology of Learning and Experimental Psychopathology. The lecture’s strap line is Investing in psychological treatments, it pays off!
David Clark clearly had a busy year last year, because in May of 2018 the Annual review of Clinical Psychology published a paper by him titled Realizing the Mass Public Benefit of Evidence-Based Psychological Therapies: The IAPT Program. The paper highlights some of the latest key metrics (adjacent). IAPT is clearly being presented as a success in England and also beyond these shores. Indeed, in the foreword to the latest IAPT Manual David Clark lauds the success of the programme.
- IAPT has grown to the point where it is seeing (in 2017) over 960,000 people a year
- IAPT currently treats over 560,000 patients per year
- Outcome data is available for 98.5% of these patients
- 51% of patients treated recover and 66% show reliable improvement
As someone who has spent more than two decades working out what quality therapy services look like, however, I think we need to take a critical look behind the headline figures.
The picture behind the headlines
A short while ago I blogged about the latest performance data from IAPT. I wasn’t awfully flattering about it, but then neither was the data. In that review I looked at two slightly different cohorts of clients: those whose referrals started (but may not have finished) in the 2017 – 18 year, and those whose referrals ended (but may not have started) in that year.
I’ve carried out some re-analysis of the data which is presented below. Here I’ve used only the data for clients whose referrals ended in 2017 – 18, so we’re looking at the same group all the way through their journey into and through IAPT in that year. In the end, the proportions that remain at each stage of the journey are very similar to my previous analysis. The overview can be seen in the image below.
With apologies for some repetition of the previous blog, let’s have a closer look at the data. First, the headlines. In the year 2017 – 18, the IAPT report and supporting datafile show that:
There was a total of 1,376,920 referrals that ended in the year
A reduction of 8,744 on the previous year
Recovery rates were 50.8%
This is up one and a half percentage points on the previous year, and for the first time exceeding the target of 50%. (definitions for recovery and caseness are shown in the panel below).
89.1% of referrals were seen within 6 weeks, and 98.8% within 18 weeks
exceeding their respective targets of 75% and 95%
The term used to describe a referral scoring highly enough on measures of depression and anxiety to be classed as a clinical case. If a patient’s score is above the clinical cut off on either anxiety, depression, or both, that are classed as a clinical case
A patient is ‘recovered’ if they finish treatment and move from caseness to non-caseness by the end of the referral. The patient needs to score below the caseness threshold on both anxiety and depression measures. Referrals that started treatment not at caseness are not included in recovery counts
To reach the point at which they can show recovery, clients that are referred must progress through a series of stages, as follows:
The stages from referral:
29% of referrals don’t enter therapy and 29% have one treatment appointment only.
The stages from entering therapy:
42% of clients that enter therapy don’t complete.
The outcome at therapy ending for all clients:
53% of clients don’t achieve recovery
The outcome at therapy ending for clients that were at case level at the start:
51% of clients achieve recovery. 49% do not.
Below I’ll elaborate on each stage and show data for 2015 – 16, 2016 – 17 and 2017- 18.
The journey from referral – 29% of referrals don’t enter treatment and 29% have only one treatment appointment
The journey starts with all referrals for the three years. The table below shows those referrals as 100%. Subsequent stages in the process, and the proportion of clients referred that still remain are also shown. In 2017 – 18, 71% of those referred entered therapy. This is defined as having one or more sessions of therapy. This is slightly higher than in the previous two years.
As we progress through their journey we can see an emerging story of attrition. 29% of those referred only have one treatment appointment. By the end of therapy, only 40% of clients that were referred now remain, Those that reach recovery represent just 19% of the total referred . That’s less than one in five clients.
Why did only seven in ten referrals enter therapy? I can only speculate, but in doing this it’s important to note that nearly nine in ten referrals were seen within six weeks. Hence length of wait may not be the most important factor. Otherwise, they may no longer have needed therapy, their referral may not have been appropriate, or they may simply not have liked what was offered to them.
Stage 2: The journey from entering therapy – 42% of clients that enter therapy don’t complete
We begin this stage with all clients that entered therapy. That is, all those that had one or more therapy sessions. In 2017 – 18, 58% of those entering therapy are recorded as ending it. This figure has reduced in each of the past two years to the point now that more than four in ten clients do not complete.
Following the journey through, we can see that just 28% of those entering therapy achieved recovery. In other words, clients entering therapy appear to stand just over a one in four chance of recovering.
Why are only 58% of clients recorded as ending therapy? Again, it’s hard to know. In my experience, however, clients more commonly drop out because they feel that therapy isn’t working for them for some reason. Often, they simply disappear without the opportunity to explore this with their therapist, and make adjustments that may better serve their needs.
Stage 3: The outcome at therapy ending for all clients – 53% of clients don’t achieve recovery
Here we start with all the clients that are recorded as finishing a course of treatment. The first point to note is that not all clients were at a case level of symptoms or distress at the outset. No matter how much improvement they make, therefore, they cannot achieve recovery. In the 2017 – 18 population, 93% of clients were at case level, and could potentially recover.
This is slightly higher than in the previous years.
Those reaching recovery represent 47% of all those that finished treatment. For 2017 – 18, this is an increase of nearly 2% over the previous year.
Stage 4: The outcome at therapy ending for clients that were at case level at the start – 51% of clients achieve recovery
The final leg of the journey, starting with clients who finished treatment that were at case level at the point they started. In this population we have all the clients who have the potential to achieve recovery.
In 2017 – 18, 51% of clients achieved recovery, of those that finished therapy and were at case level at the start. To be precise, the figure given in the IAPT report is 50.8% (against 49.3% for 2016-17).
The figure for those achieving recovery for 2016 – 17 is 1.5% higher than in the previous year. This is the first time that the recovery rate has exceeded the target of 50% that was established for the IAPT programme.
While this improvement is clearly welcome, however, it remains the case that almost as many clients do not recover as do. At nearly 51:49, the odds of recovering or not recovering are almost even.
What does good look like?
When judging the quality of a service it’s easy to get so overwhelmed by data that we lose sight of what’s important. In a previous blog I’ve argued that we should start with two key performance indicators, and let the rest fight for their place. For me, the key characteristic of high-quality provision is the ability to enable clients to maintain engagement in therapy to the point where we and the client can point to a tangible and lasting improvement. That leads directly to my two key indicators:
The proportion of clients entering therapy that have a measured ending to their therapy
The proportion of those clients that show a demonstrable improvement
The graphic below plots the proportion of clients that have reliably improved, against clients with a measured ending to their therapy. Each circle represents a service.
In the service circled in green, 84% of clients reached a measured ending to their therapy and 85% of those clients have reliably improved. In the service circled in orange, 75% of clients reach a measured ending and 67% have reliably improved. In the service circled in red, 58% of clients reach a measured ending and 66% improve.
Where does the data come from?
Circled in green: performance data from the Royal College of Nursing Counselling Service from 2003 – 4, where I was Head of Service between 1994 – 2005
Circled in orange: performance data for 2017 – 18 from Mankind, a service I provide consultancy to, that specialises in supporting men affected by unwanted sexual experiences
Circled in red: the mean level of performance across IAPT services
Tearing down the house?
A study from 2006 profiled the improvement rates of 32 primary care counselling services using the CORE Outcome Measure. (CORE-OM). The mean level of reliable improvement (including clients that also recovered) was 72%. Across IAPT, the reliable improvement figure was 66%.
IAPT improvement rates are based on GAD-7 and PHQ-9. It is likely that they would yield somewhat different results from the CORE-OM. Nonetheless, on the basis of the reported reliable improvement rates it would appear that IAPT, after some ten years of operation, falls significantly short of the rate found in the 2006 study.
I worked with a number of the services that donated their CORE data to the 2006 study, and many of them were truly outstanding. Sadly, many no longer exist. They were either decommissioned or subsumed into the emerging IAPT structure.
Governments considering the adoption of a national system of provision of the kind provided by IAPT would do well to look wider than simple recovery rates. These form only a small part of a much bigger picture. They would also do well to consider the experience of Sweden, which undertook enormous investment in CBT to address rising rates of disability due to mental ill-health.
In 2012 Sweden’s National Board of Health and Welfare announced an end to the CBT monopoly. Later, their National Audit Office concluded that “Steering towards specific treatment methods has been ineffective in achieving the objective.”
Another look behind the headlines
Behind the headline recovery and improvement rates for IAPT lies a range of performance by Clinical Commissioning Group area and individual providers. In his paper David Clark highlights the variability across services in the year 2015 – 16. The lowest recovery rate was 21% and the highest 63%. The lowest rate of reliable improvement was 35% and the highest 80%. There are clearly some high performing services out there that we can learn from.
Clark identifies a range of organisational variables that impact on performance. These include waiting times, therapy dose, rates of missed appointments and whether services are predominantly focused on providing therapy. Clark also identifies the critical importance of clinical leadership in setting the tone and culture for services to be learning organisations.
“Improving Access to Psychological Therapies is a politically-driven monster which does not cater for staff feedback/input in any way. All we are told is TARGETS!!! And work harder.”
Respondent to the New Savoy Partnership staff wellbeing survey
This will not come as news for the many service managers and practitioners who have been involved in delivering high quality services for the past couple of decades and more. Was it really necessary to tear down the whole house to now rediscover what was already known to many? As it is, we seem to have taken ten years and a vast amount of expense to get to a point that, in outcome terms, feels some distance short of where we were 2006.
A ‘politically-driven monster’ too big to fail?
Has ‘IAPT’ eaten itself? is the title of a recent article in Mental Health Today. The article profiles September 2018 figures from the Department of Health and Social Care showing that 2,000 mental health staff per month are leaving NHS services, increasing waiting times for both high and low intensity cases. The New Savoy Partnership (NSP) has been conducting annual surveys of mental health staff since 2014 to assess the wellbeing of NHS mental health staff.
Their latest survey, profiled in Mental Health Today, found that 43% of staff report feeling depressed in the prior week; 42% reported feeling like a failure in the past week; 72% think that the service they work in was understaffed and not fit for purpose; and 23% were actively considering leaving the NHS.
One of the respondents to the NSP’s 2015 survey described IAPT thus: “Improving Access to Psychological Therapies] is a politically-driven monster which does not cater for staff feedback/input in any way. All we are told is TARGETS!!! And work harder.” Jeremy Clark, NSP’s founding chair, recently described IAPT as “too big to fail.”
All’s well that ends well?
There was never anything wrong with the aspiration of providing high quality therapy at scale, simply the execution. IAPT has become deeply politicised, and as a result, too big to fail.
As early as 2014 the British Association for Behavioural and Cognitive Psychotherapies (BABCP), lead body for CBT in the UK and Ireland, deplored the “NHS management culture of bullying and intimidation, preventing [members] from openly raising their concerns and undermining their clinical judgement.”
Just like Brexit, it is very hard to know how this will end. I simply hope that we (and others) will not repeat the same mistakes.
If you carry old bricks from your past relationship to your new one, you will build the same house that fell apart before.Unknown
We thrive on feedback, so please let us know what you think about what you’ve read in the comment section below. Only the name you use to identify yourself will be shown publicly. Thank you!
Subcribe to the bulletin
If you haven’t already, subcribe to our regular (approximately fortnightly) bulletin for information about our latest blog and other news. You’ll find details here: