A Simplified Method for Routine Outcome Monitoring after Drug Abuse Treatment

EXECUTIVE SUMMARY

The routine collection of drug treatment outcomes to manage quality of care, improve patient satisfaction, and allocate treatment resources is currently hampered by two key difficulties: (1) problems locating clients once they leave treatment; and (2) the prohibitive cost of obtaining meaningful and reliable post-treatment data. This study investigated precise methods for an economical staff-based Routine Outcome Monitoring (ROM) system using an 18-item core measure telephone survey. As implemented at Narconon of Oklahoma, a behavioral and social skills based, residential drug rehabilitation program, the system was determined to be psychometrically adequate for aggregate reporting while providing clinically useful information.

The study analyzed the procedure developed and implemented by Narconon of Oklahoma (“Narconon”) for monitoring the long-term outcomes achieved by the graduates from its drug and alcohol rehabilitation program. This paper describes the precise methods for an economical staff-based Routine Outcome Monitoring (ROM) system using an 18-item core measure telephone survey as used in the Narconon program and the results of the outcome monitoring process.

The goals of routine outcome monitoring are straightforward: to ensure reproducible treatment effectiveness, consistency, and cost-effectiveness; to improve the overall quality of treatment; and to ensure accountability of health service providers to funding sources by monitoring their outcomes and maintaining treatment quality.

The systematic tracking of clients after they have completed a full course of intervention, when they are operating under minimal supervision as a member of their family, workforce and community, is one of the most convincing methods for demonstrating real-world effectiveness of behavioral health programs. Despite stated aims to gather meaningful post-discharge data, most performance-monitoring efforts are still in development stages. Limitations to data collection include: (1) difficulty tracking clients once they leave the treatment setting; (2) using treatment staff to collect follow-up data when their main function is providing chemical dependency services; and (3) relying on personal interviews and other time-consuming protocols that are costly, require substantial staff training, and may result in data drift or loss due to follow-up complexity.

Narconon has developed a streamlined, telephone-based routine outcome monitoring process which is conducted within the context of a continuing care facility. Utilizing a short outcomes survey, it is sufficiently simple to fit within the routines of staff and a smaller facility and has the advantage that it can be implemented by individuals who have not been trained in research methods. The procedure enables the program to obtain rapid feedback and thereby locate and work with clients who have encountered difficulties after leaving the program and, at the same time, enables data to be obtained which is useful for monitoring the overall effectiveness of the rehabilitation program, thereby enabling program managers to make adjustments which may be needed to increase the effectiveness of the program.

The purpose of this study was to evaluate the efficacy of a post-treatment Routine Outcome Monitoring (ROM) system as a tool for measuring, and improving, results from drug rehabilitation services. To achieve this, Narconon International and Psychometric Technologies Incorporated developed a scientifically grounded methodology to obtain useful monitoring data and implemented this in conjunction with enrollment, case management and client follow-up systems which were already in place at Narconon. Initially a 10-item questionnaire was developed but this was expanded during the course of the study based on recommendations of the aftercare and case management staff. Ultimately a survey was finalized which consists of six items directed at self-reported drug and alcohol use in the past 30 days, two items directed at general drug use since leaving treatment, five items directed at quality of life issues in the past 30 days, and several other questions useful in assessing the success of the individual in re-integrating into the community.

For purposes of developing and evaluating this instrument and the ROM methodology this project was limited to those individuals who had completed the full Narconon program (called “graduates”). Narconon staff compiled a list of individuals who had completed the program during the period of 2004 to 2007.

Aftercare specialists then conducted the ROM surveys by telephone. Initially, in addition to telephonic surveys of graduates, surveys were done of close relatives of the graduates in order to assess if the graduate data was reliable. Once it was found that there was a high statistical correlation between the data from the graduates and from their closest relatives, relatives were surveyed to obtain data when, after three attempts, graduates could not be reached directly. Data obtained was de-identified in keeping with Federal and local confidentiality rules and provided to Psychometric Technologies Incorporated for analysis.

Data was obtained from 323 of the 419 subjects who returned to their community; leaving 22.9% with missing data. The first sampling point had an inadequate follow-up rate. By reviewing the successful actions of other researchers in this field¹ improvements were made as follows: (1) Staff were trained to use a simple enrollment form to collect multiple phone and email contact information from the client as well as multiple collateral phone and address contact data; (2) All contact information was verified and updated at discharge and, (3) A written checklist was implemented to organize each step of the follow-up process. As a result of these improvements, post-treatment contact rate consistently improved to upwards of 80 percent.

The data from graduates and relatives (collateral sources) was analyzed statistically and found to be highly consistent. From this the validity of the self-report measures used in the ROM approach were established. Out of the total data set, 72.1% of the data used was self-reported by the graduates.

The data was analyzed to determine the degree to which drug problems recurred among Narconon program graduates. The following results were obtained when graduates (or collaterals) were asked about drug use during the 30 days preceding the survey interview: