TY - JOUR
T1 - Endoscopists' diagnostic accuracy in detecting upper gastrointestinal neoplasia in the framework of artificial intelligence studies
AU - Frazzoni, Leonardo
AU - Arribas, Julia
AU - Antonelli, Giulio
AU - Libanio, Diogo
AU - Ebigbo, Alanna
AU - van der Sommen, Fons
AU - de Groof, Albert Jeroen
AU - Fukuda, Hiromu
AU - Ohmori, Masayasu
AU - Ishihara, Ryu
AU - Wu, Lianlian
AU - Yu, Honggang
AU - Mori, Yuichi
AU - Repici, Alessandro
AU - Bergman, Jacques J. G. H. M.
AU - Sharma, Prateek
AU - Messmann, Helmut
AU - Hassan, Cesare
AU - Fuccio, Lorenzo
AU - Dinis-Ribeiro, M. rio
N1 - Publisher Copyright: © 2021 American Medical Association. All rights reserved.
PY - 2021
Y1 - 2021
N2 - Background ?Estimates on miss rates for upper gastrointestinal neoplasia (UGIN) rely on registry data or old studies. Quality assurance programs for upper GI endoscopy are not fully established owing to the lack of infrastructure to measure endoscopists' competence. We aimed to assess endoscopists' accuracy for the recognition of UGIN exploiting the framework of artificial intelligence (AI) validation studies. Methods ?Literature searches of databases (PubMed/MEDLINE, EMBASE, Scopus) up to August 2020 were performed to identify articles evaluating the accuracy of individual endoscopists for the recognition of UGIN within studies validating AI against a histologically verified expert-annotated ground-truth. The main outcomes were endoscopists' pooled sensitivity, specificity, positive and negative predictive value (PPV/NPV), and area under the curve (AUC) for all UGIN, for esophageal squamous cell neoplasia (ESCN), Barrett esophagus-related neoplasia (BERN), and gastric adenocarcinoma (GAC). Results ?Seven studies (2 ESCN, 3 BERN, 1 GAC, 1 UGIN overall) with 122 endoscopists were included. The pooled endoscopists' sensitivity and specificity for UGIN were 82?% (95?% confidence interval [CI] 80?%-84?%) and 79?% (95?%CI 76?%-81?%), respectively. Endoscopists' accuracy was higher for GAC detection (AUC 0.95 [95?%CI 0.93-0.98]) than for ESCN (AUC 0.90 [95?%CI 0.88-0.92]) and BERN detection (AUC 0.86 [95?%CI 0.84-0.88]). Sensitivity was higher for Eastern vs. Western endoscopists (87?% [95?%CI 84?%-89?%] vs. 75?% [95?%CI 72?%-78?%]), and for expert vs. non-expert endoscopists (85?% [95?%CI 83?%-87?%] vs. 71?% [95?%CI 67?%-75?%]). Conclusion ?We show suboptimal accuracy of endoscopists for the recognition of UGIN even within a framework that included a higher prevalence and disease awareness. Future AI validation studies represent a framework to assess endoscopist competence.
AB - Background ?Estimates on miss rates for upper gastrointestinal neoplasia (UGIN) rely on registry data or old studies. Quality assurance programs for upper GI endoscopy are not fully established owing to the lack of infrastructure to measure endoscopists' competence. We aimed to assess endoscopists' accuracy for the recognition of UGIN exploiting the framework of artificial intelligence (AI) validation studies. Methods ?Literature searches of databases (PubMed/MEDLINE, EMBASE, Scopus) up to August 2020 were performed to identify articles evaluating the accuracy of individual endoscopists for the recognition of UGIN within studies validating AI against a histologically verified expert-annotated ground-truth. The main outcomes were endoscopists' pooled sensitivity, specificity, positive and negative predictive value (PPV/NPV), and area under the curve (AUC) for all UGIN, for esophageal squamous cell neoplasia (ESCN), Barrett esophagus-related neoplasia (BERN), and gastric adenocarcinoma (GAC). Results ?Seven studies (2 ESCN, 3 BERN, 1 GAC, 1 UGIN overall) with 122 endoscopists were included. The pooled endoscopists' sensitivity and specificity for UGIN were 82?% (95?% confidence interval [CI] 80?%-84?%) and 79?% (95?%CI 76?%-81?%), respectively. Endoscopists' accuracy was higher for GAC detection (AUC 0.95 [95?%CI 0.93-0.98]) than for ESCN (AUC 0.90 [95?%CI 0.88-0.92]) and BERN detection (AUC 0.86 [95?%CI 0.84-0.88]). Sensitivity was higher for Eastern vs. Western endoscopists (87?% [95?%CI 84?%-89?%] vs. 75?% [95?%CI 72?%-78?%]), and for expert vs. non-expert endoscopists (85?% [95?%CI 83?%-87?%] vs. 71?% [95?%CI 67?%-75?%]). Conclusion ?We show suboptimal accuracy of endoscopists for the recognition of UGIN even within a framework that included a higher prevalence and disease awareness. Future AI validation studies represent a framework to assess endoscopist competence.
UR - http://www.scopus.com/inward/record.url?scp=85108512744&partnerID=8YFLogxK
U2 - https://doi.org/10.1055/a-1500-3730
DO - https://doi.org/10.1055/a-1500-3730
M3 - Article
C2 - 33951743
SN - 0013-726X
JO - Endoscopy
JF - Endoscopy
ER -