Nuclear Medicine and Artificial Intelligence: Best Practices for Evaluation (the RELAINCE Guidelines)

Abhinav K. Jha; Tyler J. Bradshaw; Irène Buvat; Mathieu Hatt; Prabhat Kc; Chi Liu; Nancy F. Obuchowski; Babak Saboury; Piotr J. Slomka; John J. Sunderland; Richard L. Wahl; Zitong Yu; Sven Zuehlsdorff; Arman Rahmim; Ronald Boellaard

doi:https://doi.org/10.2967/jnumed.121.263239

Nuclear Medicine and Artificial Intelligence: Best Practices for Evaluation (the RELAINCE Guidelines)

Abhinav K. Jha, Tyler J. Bradshaw, Irène Buvat, Mathieu Hatt, Prabhat Kc, Chi Liu, Nancy F. Obuchowski, Babak Saboury, Piotr J. Slomka, John J. Sunderland, Richard L. Wahl, Zitong Yu, Sven Zuehlsdorff, Arman Rahmim, Ronald Boellaard

Research output: Contribution to journal › Article › Academic › peer-review

36 Citations (Scopus)

Abstract

An important need exists for strategies to perform rigorous objective clinical-task-based evaluation of artificial intelligence (AI) algorithms for nuclear medicine. To address this need, we propose a 4-class framework to evaluate AI algorithms for promise, technical task-specific efficacy, clinical decision making, and postdeployment efficacy. We provide best practices to evaluate AI algorithms for each of these classes. Each class of evaluation yields a claim that provides a descriptive performance of the AI algorithm. Key best practices are tabulated as the RELAINCE (Recommendations for EvaLuation of AI for NuClear medicinE) guidelines. The report was prepared by the Society of Nuclear Medicine and Molecular Imaging AI Task Force Evaluation team, which consisted of nuclear-medicine physicians, physicists, computational imaging scientists, and representatives from industry and regulatory agencies.

Original language	English
Pages (from-to)	1288-1299
Number of pages	12
Journal	Journal of nuclear medicine
Volume	63
Issue number	9
DOIs	https://doi.org/10.2967/jnumed.121.263239
Publication status	Published - 1 Sept 2022

Keywords

PET
SPECT
artificial intelligence
best practices
clinical decision making
clinical task
evaluation
generalizability
postdeployment
technical efficacy

Access to Document

https://doi.org/10.2967/jnumed.121.263239

Cite this

Jha, A. K., Bradshaw, T. J., Buvat, I., Hatt, M., Kc, P., Liu, C., Obuchowski, N. F., Saboury, B., Slomka, P. J., Sunderland, J. J., Wahl, R. L., Yu, Z., Zuehlsdorff, S., Rahmim, A., & Boellaard, R. (2022). Nuclear Medicine and Artificial Intelligence: Best Practices for Evaluation (the RELAINCE Guidelines). Journal of nuclear medicine, 63(9), 1288-1299. https://doi.org/10.2967/jnumed.121.263239

@article{276d4b06b8a5407d924565820ffb753b,

title = "Nuclear Medicine and Artificial Intelligence: Best Practices for Evaluation (the RELAINCE Guidelines)",

abstract = "An important need exists for strategies to perform rigorous objective clinical-task-based evaluation of artificial intelligence (AI) algorithms for nuclear medicine. To address this need, we propose a 4-class framework to evaluate AI algorithms for promise, technical task-specific efficacy, clinical decision making, and postdeployment efficacy. We provide best practices to evaluate AI algorithms for each of these classes. Each class of evaluation yields a claim that provides a descriptive performance of the AI algorithm. Key best practices are tabulated as the RELAINCE (Recommendations for EvaLuation of AI for NuClear medicinE) guidelines. The report was prepared by the Society of Nuclear Medicine and Molecular Imaging AI Task Force Evaluation team, which consisted of nuclear-medicine physicians, physicists, computational imaging scientists, and representatives from industry and regulatory agencies.",

keywords = "PET, SPECT, artificial intelligence, best practices, clinical decision making, clinical task, evaluation, generalizability, postdeployment, technical efficacy",

author = "Jha, {Abhinav K.} and Bradshaw, {Tyler J.} and Ir{\`e}ne Buvat and Mathieu Hatt and Prabhat Kc and Chi Liu and Obuchowski, {Nancy F.} and Babak Saboury and Slomka, {Piotr J.} and Sunderland, {John J.} and Wahl, {Richard L.} and Zitong Yu and Sven Zuehlsdorff and Arman Rahmim and Ronald Boellaard",

note = "Funding Information: Sven Zuehlsdorff is a full-time employee of Siemens Medical Solutions USA, Inc. Nancy Obuchowski is a contracted statistician for QIBA. Tyler Bradshaw receives research support from GE Healthcare. Ronald Boellaard is (unpaid) scientific advisor for the EARL PET/CT accreditation program. Piotr Slomka has a research grant from Siemens Medical Solutions, is a consultant for IBA, and receives royalties from Cedars-Sinai for nuclear cardiology software. No other potential conflict of interest relevant to this article was reported. Publisher Copyright: {\textcopyright} 2022 Society of Nuclear Medicine Inc.. All rights reserved.",

year = "2022",

month = sep,

day = "1",

doi = "https://doi.org/10.2967/jnumed.121.263239",

language = "English",

volume = "63",

pages = "1288--1299",

journal = "Journal of nuclear medicine",

issn = "0161-5505",

publisher = "Society of Nuclear Medicine Inc.",

number = "9",

}

Jha, AK, Bradshaw, TJ, Buvat, I, Hatt, M, Kc, P, Liu, C, Obuchowski, NF, Saboury, B, Slomka, PJ, Sunderland, JJ, Wahl, RL, Yu, Z, Zuehlsdorff, S, Rahmim, A & Boellaard, R 2022, 'Nuclear Medicine and Artificial Intelligence: Best Practices for Evaluation (the RELAINCE Guidelines)', Journal of nuclear medicine, vol. 63, no. 9, pp. 1288-1299. https://doi.org/10.2967/jnumed.121.263239

TY - JOUR

T1 - Nuclear Medicine and Artificial Intelligence

T2 - Best Practices for Evaluation (the RELAINCE Guidelines)

AU - Jha, Abhinav K.

AU - Bradshaw, Tyler J.

AU - Buvat, Irène

AU - Hatt, Mathieu

AU - Kc, Prabhat

AU - Liu, Chi

AU - Obuchowski, Nancy F.

AU - Saboury, Babak

AU - Slomka, Piotr J.

AU - Sunderland, John J.

AU - Wahl, Richard L.

AU - Yu, Zitong

AU - Zuehlsdorff, Sven

AU - Rahmim, Arman

AU - Boellaard, Ronald

N1 - Funding Information: Sven Zuehlsdorff is a full-time employee of Siemens Medical Solutions USA, Inc. Nancy Obuchowski is a contracted statistician for QIBA. Tyler Bradshaw receives research support from GE Healthcare. Ronald Boellaard is (unpaid) scientific advisor for the EARL PET/CT accreditation program. Piotr Slomka has a research grant from Siemens Medical Solutions, is a consultant for IBA, and receives royalties from Cedars-Sinai for nuclear cardiology software. No other potential conflict of interest relevant to this article was reported. Publisher Copyright: © 2022 Society of Nuclear Medicine Inc.. All rights reserved.

PY - 2022/9/1

Y1 - 2022/9/1

N2 - An important need exists for strategies to perform rigorous objective clinical-task-based evaluation of artificial intelligence (AI) algorithms for nuclear medicine. To address this need, we propose a 4-class framework to evaluate AI algorithms for promise, technical task-specific efficacy, clinical decision making, and postdeployment efficacy. We provide best practices to evaluate AI algorithms for each of these classes. Each class of evaluation yields a claim that provides a descriptive performance of the AI algorithm. Key best practices are tabulated as the RELAINCE (Recommendations for EvaLuation of AI for NuClear medicinE) guidelines. The report was prepared by the Society of Nuclear Medicine and Molecular Imaging AI Task Force Evaluation team, which consisted of nuclear-medicine physicians, physicists, computational imaging scientists, and representatives from industry and regulatory agencies.

AB - An important need exists for strategies to perform rigorous objective clinical-task-based evaluation of artificial intelligence (AI) algorithms for nuclear medicine. To address this need, we propose a 4-class framework to evaluate AI algorithms for promise, technical task-specific efficacy, clinical decision making, and postdeployment efficacy. We provide best practices to evaluate AI algorithms for each of these classes. Each class of evaluation yields a claim that provides a descriptive performance of the AI algorithm. Key best practices are tabulated as the RELAINCE (Recommendations for EvaLuation of AI for NuClear medicinE) guidelines. The report was prepared by the Society of Nuclear Medicine and Molecular Imaging AI Task Force Evaluation team, which consisted of nuclear-medicine physicians, physicists, computational imaging scientists, and representatives from industry and regulatory agencies.

KW - PET

KW - SPECT

KW - artificial intelligence

KW - best practices

KW - clinical decision making

KW - clinical task

KW - evaluation

KW - generalizability

KW - postdeployment

KW - technical efficacy

UR - http://www.scopus.com/inward/record.url?scp=85132964853&partnerID=8YFLogxK

U2 - https://doi.org/10.2967/jnumed.121.263239

DO - https://doi.org/10.2967/jnumed.121.263239

M3 - Article

C2 - 35618476

SN - 0161-5505

VL - 63

SP - 1288

EP - 1299

JO - Journal of nuclear medicine

JF - Journal of nuclear medicine

IS - 9

ER -

Nuclear Medicine and Artificial Intelligence: Best Practices for Evaluation (the RELAINCE Guidelines)

Abstract

Keywords

Access to Document

Other files and links

Cite this