Routine outcome monitoring (ROM) and clinical feedback systems (CFS) are becoming an increasingly routine part of how therapy is delivered. They are often portrayed as a means by which therapy outcome and efficiency can be enhanced. But how do clients experience their use? This recent systematic review set out to find out.
In the delivery of services, whether NHS or other statutory sector, or in the third sector, the routine monitoring of outcomes has become increasingly commonplace. Routine outcome monitoring (ROM) has taken many forms over the years: pre and post measures; periodic measures during therapy; sessional measures; use of goal based and alliance measures to name a few.
More recently, the data captured by paper-based measures has migrated into software. I first got ‘hands on’ in 2001 when my service bought into the newly developed CORE PC software. From this PC based version, online systems now proliferate: CORE Net; Pragmatic Tracker; IAPTUS; PC MIS; Bac-pac; My Outcomes and so on. While each system has its own unique features, they are all informed by two key principles:
The routine collection of outcome and other data throughout the clients’ contacts with the service.
The belief that the use of ROM has the potential to reduce dropout and enhance outcomes.
In my years working as a consultant I’ve come to see therapists’ relationship with ROM and the use of clinical feedback systems (CFS) as a bit like marmite – they either love them or loathe them. In my experience therapists’ ambivalence or hostility is often based either on little or no experience of using ROM or CFS, or how they imagine clients will react to them.
Onscreen view of client completed CORE-OM as seen in Pragmatic Tracker
It’s refreshing, then, to come across a study that summarises research looking at clients’ actual experiences of ROM and CFS. So, what did the researchers find?
How do therapy clients experience ROM and CFS?
Stig Magne Solstad and colleagues conducted a systematic review [i] of 16 qualitative studies on patient experiences with the use of ROM/CF in mental health services. Among other areas they were seeking answers to the question: What do patients experience as helpful and hindering processes in using ROM/CF as part of their therapies/treatment? Their analysis of the findings showed how mixed clients’ experiences were, and identified four broad themes:
Suspicion towards service providers
Flexibility and support to capture complexity
Empowering patients
Developing collaborative practice.
The first two of these demonstrate some of the real concerns and limitations associated with ROM and CFS, while the latter two highlight some of the potential for positive change that ROM/CFS can bring, if the concerns and limitations can be addressed.
Theme 1: Suspicion towards service providers
A common theme across the studies was a suspicion that ROM/CFS was implemented for the sake of the service provider, either as a way of demonstrating its effectiveness, and/or as a basis of securing funding. Other concerns were commonly expressed:
Confidentiality of data
How information from ROM/CFS would be utilised, or that it would be used to limit or deny access to services
That ROM/CFS would emphasise treatment at the expense of relationship building
That there were implicit assumptions that gains made by clients would be attributed to therapy, thus ignoring other sources of change and failing to register the negative effects of contact with mental health services.
Theme 2: Flexibility and support to capture complexity
Within this theme, concerns were expressed in a range of studies over the inability of measures to capture the complexity of clients’ mental health concerns, and the focus of measures on symptoms at the expense of goals, values and positive aspects of life. Other concerns expressed included:
Clients wanted measures to focus on topics such as therapeutic alliance, family life, social functioning, and functioning in school or at work.
Clients feeling constrained by the rigid format of questionnaire forms and difficulty distinguishing between answer categories (e.g., “sometimes” versus “often”).
Some clients struggled to understand the meaning of the questions or to translate their actual experience into the abstract categories set out in the measure.
Theme 3: Empowering patients
Central to this theme was the desire expressed by clients to be informed about the rationale and process of ROM/CFS, and to be involved in defining their own outcomes. Specifically:
Clients wanted to be more informed and assured that data collected would be used for their benefit.
That ROM/CFS provided clients and carers opportunity for participation in treatment planning.
That measures be more responsive to clients’ needs, respect their rights and promote self-determination.
Theme 4: Developing collaborative practice.
This theme highlighted how in many studies clients experienced the use of ROM/CFS as positive and helpful. In particular, clients expressed that they:
Became more involved in treatment planning, for example in setting goals, evaluating their own progress and needs and observing evidence of progress over time.
Provided structure and focus for therapy by directing attention to areas where change was needed.
Allowed them to express their thoughts and feelings through the measures rather than verbally.
Stimulated self-reflection and self-awareness of feeling states
Promoted discussion between client and therapist, for example about areas that may not previously have been talked about
View of progress tracking chart in Pragmatic Tracker
ROM and CFS: a double-edged sword?
It is clear from the themes which emerged from this review that the application of routine outcome measurement and use of clinical feedback can bring both benefits as well as potential problems. In my experience, four key issues lie behind many of the problems that arise:
The suitability of measures to the setting or client group with whom they are used (for example the use of condition specific measures such as PHQ-9 and GAD-7 in routine practice settings)
Failure to properly explain the rationale behind ROM/CFS to the client (even assuming its potential is understood by the therapist)
Failure to use the feedback provided by ROM/CFS as an opportunity for client and therapist to collaboratively reflect on progress
Where progress is not evident, lack of reflection on the suitability of therapy goals and methods, as well as the quality of the therapeutic relationship
All these issues can be overcome, so that both clients and therapists experience the potential benefits of incorporating ROM and CFS. To bring this about, here’s what I would suggest as a process:
Ensure practitioners understand the key evidence that relates to self-assessment bias, therapist variability and the therapeutic benefits of using ROM and CFS.
Ensure that your measure or measures are fit for purpose in the context of your client group. Consider using not just measures of outcome, but also alliance, goals etc.
Provide sufficient training for practitioners to ensure are competent to use ROM/CFS as part of a collaboration with clients.
Ensure that both clients and practitioners have timely access to progress feedback (such as the tracking chart above) and that practitioners are confident in reviewing their approach with clients where indicated.
Provide oversight and support for practitioners in their use of ROM/CFS, and work to establish a supportive culture within which practitioners feel confident to challenge themselves.
Reference
[i] Solstad SM, Castonguay LG & Moltu C. 2017. Patients’ experiences with routine outcome monitoring and clinical feedback systems: A systematic review and synthesis of qualitative empirical literature. Psychotherapy Research, 2017. https://doi.org/10.1080/10503307.2017.1326645
