The clinical impact of observer variability in lung nodule classification in children with Wilms tumour

Objectives: To investigate the extent to which observer variability of computed tomography (CT) lung nodule assessment may affect clinical treatment stratification in Wilms tumour (WT) patients, according to the recent Société Internationale d'Oncologie Pédiatrique Renal Tumour Study Group (SIOP-RTSG) UMBRELLA protocol. Methods: I: CT thoraces of children with WT submitted for central review were used to estimate size distribution of lung metastases. II: Scans were selected for blinded review by five radiologists to determine intra- and inter-observer variability. They assessed identical scans on two occasions 6 months apart. III: Monte Carlo simulation (MCMC) was used to predict the clinical impact of observer variation when applying the UMBRELLA protocol size criteria. Results: Lung nodules were found in 84 out of 360 (23%) children with WT. For 21 identified lung nodules, inter-observer limits of agreement (LOA) for the five readers were ±2.4 and ±1.4 mm (AP diameter), ±1.9 and ±1.8 mm (TS diameter) and ±2.0 and ±2.4 mm (LS diameter) at assessments 1 and 2. Intra-observer LOA across the three dimensions were ±1.5, ±2.2, ±3.5, ±3.1 and ±2.6 mm (readers 1–5). MCMC demonstrated that 17% of the patients with a ‘true’ nodule size of ≥3 mm will be scored as <3 mm, and 21% of the patients with a ‘true’ nodule size of <3 mm will be scored as being ≥3 mm. Conclusion: A significant intra–inter observer variation was found when measuring lung nodules on CT for patients with WT. This may have significant implications on treatment stratification, and thereby outcome, when applying a threshold of ≥3 mm for a lung nodule to dictate metastatic status.