CNN Attention Guidance for Improved Orthopedics Radiographic Fracture Classification

Zhibin Liao, Kewen Liao, Haifeng Shen, Marouska F. van Boxel, Jasper Prijs, Ruurd L. Jaarsma, Job N. Doornberg, Anton van den Hengel, Johan W. Verjans

Research output: Contribution to JournalArticleAcademicpeer-review


Convolutional neural networks (CNNs) have gained significant popularity in orthopedic imaging in recent years due to their ability to solve fracture classification problems. A common criticism of CNNs is their opaque learning and reasoning process, making it difficult to trust machine diagnosis and the subsequent adoption of such algorithms in clinical setting. This is especially true when the CNN is trained with limited amount of medical data, which is a common issue as curating sufficiently large amount of annotated medical imaging data is a long and costly process. While interest has been devoted to explaining CNN learnt knowledge by visualizing network attention, the utilization of the visualized attention to improve network learning has been rarely investigated. This paper explores the effectiveness of regularizing CNN network with human-provided attention guidance on where in the image the network should look for answering clues. On two orthopedics radiographic fracture classification datasets, through extensive experiments we demonstrate that explicit human-guided attention indeed can direct correct network attention and consequently significantly improve classification performance.

Original languageEnglish
Pages (from-to)3139-3150
Number of pages12
JournalIEEE journal of biomedical and health informatics
Issue number7
Early online date2022
Publication statusPublished - 1 Jul 2022


  • CNN
  • X-rays
  • deep learning
  • network attention
  • orthopedics fracture classification
  • radiology

Cite this