Evaluating the endoscopic reference score for eosinophilic esophagitis: moderate to substantial intra- and interobserver reliability

Recently the Endoscopic Reference Score (EREFS) for endoscopic assessment of eosinophilic esophagitis was introduced, with good interobserver agreement for most signs. The EREFS has not yet been evaluated by other investigators and intraobserver agreement has not been assessed. The aim of this study was to further validate the EREFS by assessing interobserver and intraobserver agreement of endoscopic signs in patients with eosinophilic esophagitis. High-quality endoscopic images were made of the esophagus of 30 patients with eosinophilic esophagitis (age 36 years, range 23 - 46 years; 5 female), 6 of whom were in remission. At least three depersonalized images per patient were incorporated into a slideshow. Images were scored by four expert and four trainee endoscopists who were blinded to the patients' conditions. Interobserver agreement was assessed. After 4 weeks, the images were rescored in a different order to assess intraobserver agreement. Interobserver agreement was substantial for rings (κ 0.70), white exudates (κ 0.63), and crepe paper esophagus (κ 0.62), moderate for furrows (κ 0.49) and strictures (κ 0.54), and slight for edema (κ 0.12). Intraobserver agreement was substantial for rings (median κ 0.64, IQR 0.46 - 0.70), furrows (median κ 0.69, IQR 0.50 - 0.89), and crepe paper esophagus (median κ 0.69, IQR 0.62 - 0.83), moderate for white exudates (median κ 0.58, IQR 0.54 - 0.71) and strictures (median κ 0.54, IQR 0.33 - 0.70), and less than chance for edema (median κ 0.00, IQR 0.00 - 0.29). Inter- and intraobserver agreement was not substantially different between expert and trainee endoscopists. Using the EREFS, endoscopic signs of eosinophilic esophagitis were scored consistently by expert and trainee endoscopists