Surgical Quality Assurance in COLOR III: Standardization and Competency Assessment in a Randomized Controlled Trial

Alice Y. C. Tsai, Stella Mavroveli, Danilo Miskovic, Stefan van Oostendorp, Michel Adamina, Roel Hompes, Felix Aigner, Antonino Spinelli, Janindra Warusavitarne, Joep Knol, Matthew Albert, George Nassif, Willem Bemelman, Luigi Boni, Henrik Ovesen, Ralph Austin, Andrea Muratore, Gerald Seitinger, Colin Sietses, Antonio M. LacyJurriaan B. Tuynman, H. Jaap Bonjer, George B. Hanna

Research output: Contribution to journalArticleAcademicpeer-review

28 Citations (Scopus)


OBJECTIVE: The aim of this study was to develop an objective and reliable surgical quality assurance system (SQA) for COLOR III, an international multicenter randomized controlled trial (RCT) comparing transanal total mesorectal excision (TaTME) with laparoscopic approach for rectal cancer. BACKGROUND OF SUMMARY DATA: SQA influences outcome measures in RCTs such as lymph nodes harvest, in-hospital mortality, and locoregional cancer recurrence. However, levels of SQA are variable. METHOD: Hierarchical task analysis of TaTME was performed. A 4-round Delphi methodology was applied for standardization of TaTME steps. Semistructured interviews were conducted in round 1 to identify key steps and tasks, which were rated as mandatory, optional, or prohibited in rounds 2 to 4 using questionnaires. Competency assessment tool (CAT) was developed and its content validity was examined by expert surgeons. Twenty unedited videos were assessed to test reliability using generalizability theory. RESULTS: Eighty-three of 101 surgical tasks identified reached 70% agreement (26 mandatory, 56 optional, and 1 prohibited). An operative guide of standardized TaTME was created. CAT is matrix of 9 steps and 4 performance qualities: exposure, execution, adverse event, and end-product. The overall G-coefficient was 0.883. Inter-rater and interitem reliability were 0.883 and 0.986. To enter COLOR III, 2 unedited TaTME and 1 laparoscopic TME videos were submitted and assessed by 2 independent assessors using CAT. CONCLUSION: We described an iterative approach to develop an objective SQA within multicenter RCT. This approach provided standardization, the development of reliable and valid CAT, and the criteria for trial entry and monitoring surgical performance during the trial.
Original languageEnglish
Pages (from-to)768-774
JournalAnnals of surgery
Issue number5
Publication statusPublished - 2019

Cite this