Automatic Auditing Software Improves Quality and Consistency of Radiation Oncology Care

A software program designed to improve consistency in planning and delivering radiation therapy reduces override and late plan completion and is an effective tool for identifying bottlenecks in department work streams, according to a recent study. Naichang Yu, PhD, staff medical physicist in Cleveland Clinic Cancer Center’s Department of Radiation Oncology, and one of the program’s designers, presented the study results at the American Society for Radiation Oncology (ASTRO) annual meeting in 2018, and has since continued with system expansion and updates.

The Quantitative Metrics and Automatic Auditing Program, or QMAP, is based on a set of quantitative metrics that rate the timely completion of critical clinical tasks, as defined internally by the Department of Radiation Oncology. QMAP generates alerts or reminders that are very effective in keeping clinicians on schedule with key tasks. “We developed this program because we know that consistency in planning and delivering radiation therapy is critical to delivering safe, high-quality and efficient care,” says Dr. Yu.

QMAP in action

QMAP software automatically captures the historical trend of each metric and provides daily alerts for potential deviations (i.e., misses) from defined processes, based on Cleveland Clinic’s electronic medical record (EMR) system. Alerts are directed to triage teams within a time frame that gives clinicians the opportunity to avoid the deviations.

Prior studies have demonstrated QMAP’s significant positive impact on compliance with three key tasks including completion of the radiation therapy planning process at least four hours prior to the scheduled treatment time for non-emergency patients; review of patient cone-beam computed tomography (CBCT) images the same day they become available; and timely completion of visits to patients (by physicians) according to a department-defined schedule. QMAP has been extended to provide measures of the quality of some critical aspects of the process, as well as the timeliness of the whole process in greater detail.

Key metrics for study

For the current study, researchers examined two metrics — table position overrides during delivery and late plan completion — in two regional radiation oncology departments integrated into the larger enterprise system in 2016. They compared rates of position overrides and late plan completion before and after adoption of the QMAP system. Position overrides included in the study were those in the longitudinal, lateral and vertical directions in any external beam therapy after the first fraction. The study of late plan completion included patients whose simulation occurred at least two business days before the scheduled treatment, and plans ready less than four hours before the scheduled treatment time were considered late.

During the study period, the override rate was reduced from 6.7% pre-QMAP to 3.3% (P < 0.01), and the late plan rate dropped from 27.8% to 9.9% (P < 0.005) post integration into the QMAP system. “After the departments adopted QMAP, they worked on identifying root causes for these gaps and then adopting corrective measures,” says Dr. Yu. “It would be difficult and very time consuming to monitor and aggregate all of this data without an EMR-integrated system like QMAP. With this data, we can make improvements that benefit quality and safety.”

Expansion of QMAP across all Cleveland Clinic hospitals with radiation therapy

Since this data was published in 2018, QMAP has been extended to all Cleveland Clinic regional hospitals. Studies are underway to quantify correlation between compliance with QMAP metrics and clinical outcomes, such as reduction of medical errors and improved clinical efficiency.

“The expansion of QMAP has helped to improve the standardization of radiation oncology practices across all of Cleveland Clinic’s radiation oncology departments,” says John Suh, MD, Chair of the Department of Radiation Oncology, “and has been a useful tool to improve safety, quality, efficiency and engagement in all of the departments.”