New Tool Offers User-Friendly Option for Assessing Colorectal Surgery Competence

A new tool for assessing surgical performance of laparoscopic right colectomy could help to standardize the procedure and improve patient outcomes.

With the tool, developed by an experts from the American Society of Colon and Rectal Surgeons (ASCRS), raters assess performance of videotaped laparoscopic colectomies (LCs). Potential applications include surgical training for residents/fellows and standardized assessments of technical competency at the attending level.

Thus far, the lack of consensus for LC performance assessment has been a major impediment to quality improvement, says Cleveland Clinic colorectal surgeon Bradley J. Champagne, MD, chair of the ASCRS’ Operative Competency Evaluation Committee, which developed the new tool.

“Since about 2005, the standard of care for colon surgery has been a minimally invasive or laparoscopic approach,” says Dr. Champagne, also Chairman of Surgery at Cleveland Clinic Fairview Hospital. “But, with no set standard for how to teach and no standardized approach, everyone has been doing it their own way. More importantly, validated assessment of trainees and attending surgeons learning this technique had never been appropriately established. Therefore, ASCRS worked to make a tool through our committee.”

Results from that effort, published recently in Diseases of the Colon & Rectum, suggest that the new tool is valid, feasible and overcomes many of the limitations of previously-developed assessment modalities.

Excellent consistency

During a pilot phase, four panel members watched de-identified videos of LCs performed by surgeons who were either novices (less than 20 LC cases), intermediate (50-100 cases), or experts (more than 500 LCs). With repeated consultations to decrease variability in their evaluations, they achieved good reliability (κ > 0.8) in average time less than 20 minutes.

Using those results, 20 reviewers each assessed six de-identified videos on eight LC performance steps — pedicle, respect for tissue, time/motion, instrument handling, operation flow, exposure, tissue planes and dissection completeness — and also provided an overall technical skill rating.

Overall internal consistency was excellent, with an overall Cronbach α of 0.98, as was consistency for each step (>0.74). Average scores for the videos of LC experts were significantly better than for those in the intermediate category, who in turn scored significantly better than novices (P = 0.003 and 0.006, respectively).

This new approach offers significant improvement over prior procedure-specific tools that require the observer to be physically present at the operation, or the Observational Clinical Human Reliability Analysis developed for the United Kingdom’s national LC training program, which requires proprietary software and extensive rater training.

A needed tool

Now is a critical time for development of such surgical assessment tools, given the confluence of mandated reduction in work hours — and therefore less opportunity to gain experience — at the same time that payers are increasing scrutiny of outcomes in bundled payment models. Combined with the recent advent of minimally invasive techniques, and “It’s a perfect storm and set-up for failure,” Dr. Champagne says.

The new tool can assist in a variety of ways. Surgeons might submit their videos to state-funded quality improvement programs to receive feedback, and then sign up for remediation if needed. Or, training programs can use the tool to compare their success to those of other institutions.

In addition, surgeons who sign up for industry-funded LC courses can submit before- and after- videos to assess improvement, with feedback provided to both the course attendee and the sponsoring companies. Such a process could have a major impact, given that 80 percent of colon resections are still being performed by general surgeons without a colorectal board certificate, Dr. Champagne notes.

Also, a modified version of the tool is now under development for use as required criteria for certification of future centers of excellence for colorectal cancer, just as bariatric surgery centers exist now.

“We’re actively trying to improve the education and the quality of minimally invasive colon surgery on both a regional and national level by having a standardized universal assessment tool for quality improvement,” Dr. Champagne says. “It really fits the direction of healthcare and the scrutinization of outcomes and quality moving forward.”