Observer variability for Lung-RADS categorisation of lung cancer screening CTs: impact on patient management

Sarah J. van Riel, Colin Jacobs, Ernst Th. Scholten, Rianne Wittenberg, Mathilde M. Winkler Wille, Bartjan de Hoop, Ralf Sprengers, Onno M. Mets, Bram Geurts, Mathias Prokop, Cornelia Schaefer-Prokop, Bram van Ginneken

Research output: Contribution to journalArticleAcademicpeer-review

39 Citations (Scopus)


Objectives: Lung-RADS represents a categorical system published by the American College of Radiology to standardise management in lung cancer screening. The purpose of the study was to quantify how well readers agree in assigning Lung-RADS categories to screening CTs; secondary goals were to assess causes of disagreement and evaluate its impact on patient management. Methods: For the observer study, 80 baseline and 80 follow-up scans were randomly selected from the NLST trial covering all Lung-RADS categories in an equal distribution. Agreement of seven observers was analysed using Cohen’s kappa statistics. Discrepancies were correlated with patient management, test performance and diagnosis of malignancy within the scan year. Results: Pairwise interobserver agreement was substantial (mean kappa 0.67, 95% CI 0.58–0.77). Lung-RADS category disagreement was seen in approximately one-third (29%, 971) of 3360 reading pairs, resulting in different patient management in 8% (278/3360). Out of the 91 reading pairs that referred to scans with a tumour diagnosis within 1 year, discrepancies in only two would have resulted in a substantial management change. Conclusions: Assignment of lung cancer screening CT scans to Lung-RADS categories achieves substantial interobserver agreement. Impact of disagreement on categorisation of malignant nodules was low. Key Points: • Lung-RADS categorisation of low-dose lung screening CTs achieved substantial interobserver agreement. • Major cause for disagreement was assigning a different nodule as risk-dominant. • Disagreement led to a different follow-up time in 8% of reading pairs.
Original languageEnglish
Pages (from-to)924-931
JournalEuropean Radiology
Issue number2
Early online date2018
Publication statusPublished - 1 Feb 2019


  • Cancer screening
  • Lung cancer
  • Observer variation
  • Solitary pulmonary nodule
  • X-ray computed tomography

Cite this