3 Tips for Maintaining Outcomes Integrity in Behavioral Health
A big thank you to Tristan Keelan of TenEleven Group for this guest post!
The Behavioral Health field is undergoing a data revolution where electronic capture of daily activity is expected to produce reports that demonstrate, among other things, quality. The initial response to this paradigm shift is to have an electronic health record that can capture data, and produce reporting and analytics capabilities that use the data to demonstrate agency effectiveness. While front-end data capture and back-end information reporting are certainly critical elements of the value based care equation, there is still an element of intervention required by your Quality and Compliance staff to ensure that your data has integrity. To achieve data integrity, you need to make sure that you’re collecting your data in a repeatable manner that provides consistent and accurate data.
Find your outcomes measurement tool
One of the best ways to demonstrate value in behavioral health is to adopt standardized outcomes measurement tools. Tools like the CANS/ANSA, PHQ-9, DLA-20, GAD-7, Columbia Suicide Risk Scale, and many others are designed to allow for the measurement of client populations to be viewed in aggregate for the purposes of demonstrating agency effectiveness. However, the integrity of the outcomes data you capture is going to be critical for framing the story telling that is done once your results are in.
3 tips for maintaining outcomes integrity
The three critical elements that you can manipulate to drive data integrity within your outcomes tools all revolve around creating and enforcing repeatable processes. When your processes are repeatable, you can present your reports with foundational context that improves your presentation. This can turn “here are our numbers,” into “We follow these processes and procedures, which we know drive success, and the proof is here in our numbers.” And the second version is much more convincing.
These three elements are:
1. Determine the cadence that outcomes measures will be captured by, and enforce it.
You want to use standardized outcomes measurement tools so that you can find a way to commonly measure success across your patient population as the progress through treatment over time. Normally when we measure patient outcomes over time, we would make the date the tool was administered the X-axis that would drive our visual; however, in the case of behavioral health outcomes measurement at the agency level we cannot do that.
If you were to look at outcomes scores by date, you end up with a mash up of scattered data because clients are not all seen on the same days. For a visual example, look to the PHQ-9 outcomes scores shown in the graph below:
To rectify this and bring your clients into an “apples to apples” view of progress, you must group the scores into the iteration of the form that was administered – first time, second time, third time, etc. When you do that, you bring a focus to what your outcomes look like at the agency level.
The image below shows the same data set, for the same time period, but it is organized with “Administration Number,” as the X-axis to bring the average improvements of client across the agency into focus.
To make this data tell the story you want, you must be able to articulate what your process for capturing the outcomes measurement is. For example, the measure could be administered every time the customer is seen. Or, every 3rd visit. Or, every 3 months (maybe this coincides with treatment plan reviews). Whatever your agency’s choice is, it should be agreed upon, and enforced. From a quality enforcement standpoint, there should be monitoring reports and procedures to follow-up with therapists who have clients fall out of the agency defined outcomes measurement procedures.
Those monitoring reports can also be run historically to determine how often your process is falling outside the desired timeframes, and thus how far off your data integrity is from the desired. If your monitoring reports show that your clinicians are administering the PHQ-9 at different points in treatment for each client, you will have a low data integrity.
Your agency should strive to be able to speak to outcomes in terms of the example phrase, “the positive trend in PHQ-9 scores reflects our process to administer the measurement tool within 14 days of the treatment planning due date, which we achieve 99% of the time.”
When your Behavioral Health agency can add these types of process statements in conjunction with positive trending outcomes reports, then your value proposition appears substantially stronger to your audience.
2. Acknowledge outcome drift – respect the anchors and keep them front and center
Your clinical staff wants their clients to improve; that’s why they got into this business in the first place. However, this presents a common problem for clinical staff who administer scoring tools. Because the desire is for the client to improve, the previous score can often replace the scoring anchors in the subconscious of the person doing the scoring.
For example, consider the following simplistic version of a hypothetical 3 point symptomatic scale.
In the past month, how often did you experience feelings of anxiety?
In our example, let’s assume the client has scored a 2 on the past two administrations of the tool. The therapist can begin to rationalize that while the client is exhibiting signs that reflect a 2 = Some of the Time, they seem to be doing better overall. Instead of marking a 2 for the third time in a row on this question, the therapist can begin to use the desire for improvement to justify marking a 1 = Very Little because of the appearance of overall improvement.
This type of “Anchor Drift” is natural and should not prevent the use of standardized outcomes tools, but rather should be protected against. There can be a tendency in EHR software systems to streamline forms in a way that removes unnecessary elements that may have existed in the original paper version. This is where it helps to combat that tendency to drift toward improvement by making the anchors present on the form every time it is completed.
In this case if our anchors, “Very Little,” “Some of the Time,” and “All of the Time” are right next to the questions, it will help to subconsciously override the client’s previous score as the anchor. Stripping out the anchors on an electronic form seems like an efficiency gain of space on the screen, but a smart EHR will keep outcomes anchors present on the form to hold the integrity of the tool above the well-meaning desire for clients to improve.
3. Periodically audit your outcome tool with self-assessments
In conjunction with the previous effort to maintain integrity to the outcomes anchors, you can enforce an audit of your clinical staff compared against your client population. Introduce the client to the outcomes anchors, and ask them and the therapist to each complete separate versions of the form at the same point in the treatment. Then you can measure the variance between them at the agency, therapist, and client levels.
As with all things discussed thus far, there are multiple ways to conduct this audit, and it’s really all about establishing and maintaining process. You can dictate that a self-assessment audit is performed every fourth administration of the outcomes tool. Or, call for random self-assessment auditing at different points and have the audit reflect a date in time, versus a static stage of the process that moves with time. Both methods demonstrate that the agency is committed to maintaining anchor integrity. If the anchor drift feels strong, the agency can move towards refresher training or other follow-up methods that are meant to always keep therapists scoring to the anchors.
Show off your good work
When presenting data, you want to be able to articulate the procedural steps that your agency is taking to ensure that showing a positive trend line can be taken as a representation of actual client improvement. This can make all the difference between winning a contract or being overlooked despite your good work.
The views, information and opinions expressed herein are those of the author; they do not necessarily reflect those of the Council on Accreditation (COA). COA invites guest authors to contribute to the COA blog due to COA’s confidence in their knowledge on the subject matter and their expertise in their chosen field.
Tristan Keelan is the Marketing Strategist at TenEleven Group. Tristan holds a BA in English from Elmira College and an MBA from St. Bonaventure University. He uses his experiences working in Government, Non-profit National Service, and Banking to bring business process and analytics driven insights to the behavioral health industry.
You can read blog posts from Tristan here.