Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation

Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

3 Citations (Scopus)

Abstract

Detection of Out-of-Distribution (OOD) samples in real time is a crucial safety check for deployment of machine learning models in the medical field. Despite a growing number of uncertainty quantification techniques, there is a lack of evaluation guidelines on how to select OOD detection methods in practice. This gap impedes implementation of OOD detection methods for real-world applications. Here, we propose a series of practical considerations and tests to choose the best OOD detector for a specific medical dataset. These guidelines are illustrated on a real-life use case of Electronic Health Records (EHR). Our results can serve as a guide for implementation of OOD detection methods in clinical practice, mitigating risks associated with the use of machine learning models in healthcare.

Original languageEnglish
Title of host publicationStudies in Computational Intelligence
Subtitle of host publicationA Paradigm Shift in Health Intelligence
EditorsA. Shaban-Nejad, M. Michalowski, S. Bianco
Place of PublicationCham
PublisherSpringer Science and Business Media Deutschland GmbH
Pages137-153
Number of pages17
Volume1060
ISBN (Electronic)9783031147715
ISBN (Print)9783031147708
DOIs
Publication statusPublished - 2022
Event2022 Health Intelligence workshop and associated Data Hackathon/Challenge: co-located with the 36th Association for the Advancement of Artificial Intelligence (AAAI) conference - Vancouver, Canada
Duration: 28 Feb 20221 Mar 2022

Publication series

NameStudies in Computational Intelligence
Volume1060

Workshop

Workshop2022 Health Intelligence workshop and associated Data Hackathon/Challenge
Abbreviated titleW3PHIAI-22
Country/TerritoryCanada
CityVancouver
Period28/02/20221/03/2022

Keywords

  • Density estimation
  • Electronic health records
  • Evaluation guidelines
  • Out-Of-Distribution detection

Cite this