A foundation model for generalizable disease detection from retinal images

Yukun Zhou; Mark A. Chia; Siegfried K. Wagner; Murat S. Ayhan; Dominic J. Williamson; Robbert R. Struyven; Timing Liu; Moucheng Xu; Mateo G. Lozano; Peter Woodward-Court; Yuka Kihara; Naomi Allen; John E. J. Gallacher; Thomas Littlejohns; Tariq Aslam; Paul Bishop; Graeme Black; Panagiotis Sergouniotis; Denize Atan; Andrew D. Dick; Cathy Williams; Sarah Barman; Jenny H. Barrett; Sarah Mackie; Tasanee Braithwaite; Roxana O. Carare; Sarah Ennis; Jane Gibson; Andrew J. Lotery; Jay Self; Usha Chakravarthy; Ruth E. Hogg; Euan Paterson; Jayne Woodside; Tunde Peto; Gareth Mckay; Bernadette Mcguinness; Paul J. Foster; Konstantinos Balaskas; Anthony P. Khawaja; Nikolas Pontikos; Jugnoo S. Rahi; Gerassimos Lascaratos; Praveen J. Patel; Michelle Chan; Sharon Y. L. Chua; Alexander Day; Parul Desai; Cathy Egan; Marcus Fruttiger; David F. Garway-Heath; Alison Hardcastle; Sir Peng T. Khaw; Tony Moore; Sobha Sivaprasad; Nicholas Strouthidis; Dhanes Thomas; Adnan Tufail; Ananth C. Viswanathan; Bal Dhillon; Tom Macgillivray; Cathie Sudlow; Veronique Vitart; Alexander Doney; Emanuele Trucco; Jeremy A. Guggeinheim; James E. Morgan; Chris J. Hammond; Katie Williams; Pirro Hysi; Simon P. Harding; Yalin Zheng; Robert Luben; Phil Luthert; Zihan Sun; Martin McKibbin; Eoin O’Sullivan; Richard Oram; Mike Weedon; Chris G. Owen; Alicja R. Rudnicka; Naveed Sattar; David Steel; Irene Stratton; Robyn Tapp; Max M. Yates; Axel Petzold; Savita Madhusudhan; Andre Altmann; Aaron Y. Lee; Eric J. Topol; Alastair K. Denniston; Daniel C. Alexander; Pearse A. Keane

doi:https://doi.org/10.1038/s41586-023-06555-x

A foundation model for generalizable disease detection from retinal images

Yukun Zhou, Mark A. Chia, Siegfried K. Wagner, Murat S. Ayhan, Dominic J. Williamson, Robbert R. Struyven, Timing Liu, Moucheng Xu, Mateo G. Lozano, Peter Woodward-Court, Yuka Kihara, Naomi Allen, John E. J. Gallacher, Thomas Littlejohns, Tariq Aslam, Paul Bishop, Graeme Black, Panagiotis Sergouniotis, Denize Atan, Andrew D. DickCathy Williams, Sarah Barman, Jenny H. Barrett, Sarah Mackie, Tasanee Braithwaite, Roxana O. Carare, Sarah Ennis, Jane Gibson, Andrew J. Lotery, Jay Self, Usha Chakravarthy, Ruth E. Hogg, Euan Paterson, Jayne Woodside, Tunde Peto, Gareth Mckay, Bernadette Mcguinness, Paul J. Foster, Konstantinos Balaskas, Anthony P. Khawaja, Nikolas Pontikos, Jugnoo S. Rahi, Gerassimos Lascaratos, Praveen J. Patel, Michelle Chan, Sharon Y. L. Chua, Alexander Day, Parul Desai, Cathy Egan, Marcus Fruttiger, David F. Garway-Heath, Alison Hardcastle, Sir Peng T. Khaw, Tony Moore, Sobha Sivaprasad, Nicholas Strouthidis, Dhanes Thomas, Adnan Tufail, Ananth C. Viswanathan, Bal Dhillon, Tom Macgillivray, Cathie Sudlow, Veronique Vitart, Alexander Doney, Emanuele Trucco, Jeremy A. Guggeinheim, James E. Morgan, Chris J. Hammond, Katie Williams, Pirro Hysi, Simon P. Harding, Yalin Zheng, Robert Luben, Phil Luthert, Zihan Sun, Martin McKibbin, Eoin O’Sullivan, Richard Oram, Mike Weedon, Chris G. Owen, Alicja R. Rudnicka, Naveed Sattar, David Steel, Irene Stratton, Robyn Tapp, Max M. Yates, Axel Petzold, Savita Madhusudhan, Andre Altmann, Aaron Y. Lee, Eric J. Topol, Alastair K. Denniston, Daniel C. Alexander, Pearse A. Keane

Research output: Contribution to journal › Article › Academic › peer-review

34 Citations (Scopus)

Abstract

Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders 1. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications 2. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

Original language	English
Pages (from-to)	156-163
Number of pages	8
Journal	NATURE
Volume	622
Issue number	7981
Early online date	2023
DOIs	https://doi.org/10.1038/s41586-023-06555-x
Publication status	Published - 5 Oct 2023

Access to Document

https://doi.org/10.1038/s41586-023-06555-x

Cite this

Zhou, Y., Chia, M. A., Wagner, S. K., Ayhan, M. S., Williamson, D. J., Struyven, R. R., Liu, T., Xu, M., Lozano, M. G., Woodward-Court, P., Kihara, Y., Allen, N., Gallacher, J. E. J., Littlejohns, T., Aslam, T., Bishop, P., Black, G., Sergouniotis, P., Atan, D., ... Keane, P. A. (2023). A foundation model for generalizable disease detection from retinal images. NATURE, 622(7981), 156-163. https://doi.org/10.1038/s41586-023-06555-x

@article{7fa894175da742a6983ff806441ffa23,

title = "A foundation model for generalizable disease detection from retinal images",

abstract = "Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders 1. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications 2. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.",

author = "Yukun Zhou and Chia, {Mark A.} and Wagner, {Siegfried K.} and Ayhan, {Murat S.} and Williamson, {Dominic J.} and Struyven, {Robbert R.} and Timing Liu and Moucheng Xu and Lozano, {Mateo G.} and Peter Woodward-Court and Yuka Kihara and Naomi Allen and Gallacher, {John E. J.} and Thomas Littlejohns and Tariq Aslam and Paul Bishop and Graeme Black and Panagiotis Sergouniotis and Denize Atan and Dick, {Andrew D.} and Cathy Williams and Sarah Barman and Barrett, {Jenny H.} and Sarah Mackie and Tasanee Braithwaite and Carare, {Roxana O.} and Sarah Ennis and Jane Gibson and Lotery, {Andrew J.} and Jay Self and Usha Chakravarthy and Hogg, {Ruth E.} and Euan Paterson and Jayne Woodside and Tunde Peto and Gareth Mckay and Bernadette Mcguinness and Foster, {Paul J.} and Konstantinos Balaskas and Khawaja, {Anthony P.} and Nikolas Pontikos and Rahi, {Jugnoo S.} and Gerassimos Lascaratos and Patel, {Praveen J.} and Michelle Chan and Chua, {Sharon Y. L.} and Alexander Day and Parul Desai and Cathy Egan and Marcus Fruttiger and Garway-Heath, {David F.} and Alison Hardcastle and Khaw, {Sir Peng T.} and Tony Moore and Sobha Sivaprasad and Nicholas Strouthidis and Dhanes Thomas and Adnan Tufail and Viswanathan, {Ananth C.} and Bal Dhillon and Tom Macgillivray and Cathie Sudlow and Veronique Vitart and Alexander Doney and Emanuele Trucco and Guggeinheim, {Jeremy A.} and Morgan, {James E.} and Hammond, {Chris J.} and Katie Williams and Pirro Hysi and Harding, {Simon P.} and Yalin Zheng and Robert Luben and Phil Luthert and Zihan Sun and Martin McKibbin and Eoin O{\textquoteright}Sullivan and Richard Oram and Mike Weedon and Owen, {Chris G.} and Rudnicka, {Alicja R.} and Naveed Sattar and David Steel and Irene Stratton and Robyn Tapp and Yates, {Max M.} and Axel Petzold and Savita Madhusudhan and Andre Altmann and Lee, {Aaron Y.} and Topol, {Eric J.} and Denniston, {Alastair K.} and Alexander, {Daniel C.} and Keane, {Pearse A.}",

note = "Funding Information: We thank P. Rawlinson for project management, C. Green and L. Wickham for information governance expertise, and A. Wenban, S. St John-Green and M. Barnfield for information technology support. This work is supported by Engineering and Physical Sciences Research Council grant nos. EP/M020533/1, EP/R014019/1 and EP/V034537/1, as well as the NIHR UCLH Biomedical Research Centre. S.K.W. is supported by a Medical Research Council Clinical Research Training Fellowship (grant no. MR/TR000953/1). P.A.K. is supported by a Moorfields Eye Charity Career Development Award (grant no. R190028A) and a UK Research & Innovation Future Leaders Fellowship (grant no. MR/T019050/1). For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising. Publisher Copyright: {\textcopyright} 2023, The Author(s).",

year = "2023",

month = oct,

day = "5",

doi = "https://doi.org/10.1038/s41586-023-06555-x",

language = "English",

volume = "622",

pages = "156--163",

journal = "NATURE",

issn = "0028-0836",

publisher = "Nature Publishing Group",

number = "7981",

}

Zhou, Y, Chia, MA, Wagner, SK, Ayhan, MS, Williamson, DJ, Struyven, RR, Liu, T, Xu, M, Lozano, MG, Woodward-Court, P, Kihara, Y, Allen, N, Gallacher, JEJ, Littlejohns, T, Aslam, T, Bishop, P, Black, G, Sergouniotis, P, Atan, D, Dick, AD, Williams, C, Barman, S, Barrett, JH, Mackie, S, Braithwaite, T, Carare, RO, Ennis, S, Gibson, J, Lotery, AJ, Self, J, Chakravarthy, U, Hogg, RE, Paterson, E, Woodside, J, Peto, T, Mckay, G, Mcguinness, B, Foster, PJ, Balaskas, K, Khawaja, AP, Pontikos, N, Rahi, JS, Lascaratos, G, Patel, PJ, Chan, M, Chua, SYL, Day, A, Desai, P, Egan, C, Fruttiger, M, Garway-Heath, DF, Hardcastle, A, Khaw, SPT, Moore, T, Sivaprasad, S, Strouthidis, N, Thomas, D, Tufail, A, Viswanathan, AC, Dhillon, B, Macgillivray, T, Sudlow, C, Vitart, V, Doney, A, Trucco, E, Guggeinheim, JA, Morgan, JE, Hammond, CJ, Williams, K, Hysi, P, Harding, SP, Zheng, Y, Luben, R, Luthert, P, Sun, Z, McKibbin, M, O’Sullivan, E, Oram, R, Weedon, M, Owen, CG, Rudnicka, AR, Sattar, N, Steel, D, Stratton, I, Tapp, R, Yates, MM, Petzold, A, Madhusudhan, S, Altmann, A, Lee, AY, Topol, EJ, Denniston, AK, Alexander, DC & Keane, PA 2023, 'A foundation model for generalizable disease detection from retinal images', NATURE, vol. 622, no. 7981, pp. 156-163. https://doi.org/10.1038/s41586-023-06555-x

TY - JOUR

T1 - A foundation model for generalizable disease detection from retinal images

AU - Zhou, Yukun

AU - Chia, Mark A.

AU - Wagner, Siegfried K.

AU - Ayhan, Murat S.

AU - Williamson, Dominic J.

AU - Struyven, Robbert R.

AU - Liu, Timing

AU - Xu, Moucheng

AU - Lozano, Mateo G.

AU - Woodward-Court, Peter

AU - Kihara, Yuka

AU - Allen, Naomi

AU - Gallacher, John E. J.

AU - Littlejohns, Thomas

AU - Aslam, Tariq

AU - Bishop, Paul

AU - Black, Graeme

AU - Sergouniotis, Panagiotis

AU - Atan, Denize

AU - Dick, Andrew D.

AU - Williams, Cathy

AU - Barman, Sarah

AU - Barrett, Jenny H.

AU - Mackie, Sarah

AU - Braithwaite, Tasanee

AU - Carare, Roxana O.

AU - Ennis, Sarah

AU - Gibson, Jane

AU - Lotery, Andrew J.

AU - Self, Jay

AU - Chakravarthy, Usha

AU - Hogg, Ruth E.

AU - Paterson, Euan

AU - Woodside, Jayne

AU - Peto, Tunde

AU - Mckay, Gareth

AU - Mcguinness, Bernadette

AU - Foster, Paul J.

AU - Balaskas, Konstantinos

AU - Khawaja, Anthony P.

AU - Pontikos, Nikolas

AU - Rahi, Jugnoo S.

AU - Lascaratos, Gerassimos

AU - Patel, Praveen J.

AU - Chan, Michelle

AU - Chua, Sharon Y. L.

AU - Day, Alexander

AU - Desai, Parul

AU - Egan, Cathy

AU - Fruttiger, Marcus

AU - Garway-Heath, David F.

AU - Hardcastle, Alison

AU - Khaw, Sir Peng T.

AU - Moore, Tony

AU - Sivaprasad, Sobha

AU - Strouthidis, Nicholas

AU - Thomas, Dhanes

AU - Tufail, Adnan

AU - Viswanathan, Ananth C.

AU - Dhillon, Bal

AU - Macgillivray, Tom

AU - Sudlow, Cathie

AU - Vitart, Veronique

AU - Doney, Alexander

AU - Trucco, Emanuele

AU - Guggeinheim, Jeremy A.

AU - Morgan, James E.

AU - Hammond, Chris J.

AU - Williams, Katie

AU - Hysi, Pirro

AU - Harding, Simon P.

AU - Zheng, Yalin

AU - Luben, Robert

AU - Luthert, Phil

AU - Sun, Zihan

AU - McKibbin, Martin

AU - O’Sullivan, Eoin

AU - Oram, Richard

AU - Weedon, Mike

AU - Owen, Chris G.

AU - Rudnicka, Alicja R.

AU - Sattar, Naveed

AU - Steel, David

AU - Stratton, Irene

AU - Tapp, Robyn

AU - Yates, Max M.

AU - Petzold, Axel

AU - Madhusudhan, Savita

AU - Altmann, Andre

AU - Lee, Aaron Y.

AU - Topol, Eric J.

AU - Denniston, Alastair K.

AU - Alexander, Daniel C.

AU - Keane, Pearse A.

N1 - Funding Information: We thank P. Rawlinson for project management, C. Green and L. Wickham for information governance expertise, and A. Wenban, S. St John-Green and M. Barnfield for information technology support. This work is supported by Engineering and Physical Sciences Research Council grant nos. EP/M020533/1, EP/R014019/1 and EP/V034537/1, as well as the NIHR UCLH Biomedical Research Centre. S.K.W. is supported by a Medical Research Council Clinical Research Training Fellowship (grant no. MR/TR000953/1). P.A.K. is supported by a Moorfields Eye Charity Career Development Award (grant no. R190028A) and a UK Research & Innovation Future Leaders Fellowship (grant no. MR/T019050/1). For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising. Publisher Copyright: © 2023, The Author(s).

PY - 2023/10/5

Y1 - 2023/10/5

N2 - Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders 1. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications 2. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

AB - Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders 1. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications 2. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

UR - http://www.scopus.com/inward/record.url?scp=85171296317&partnerID=8YFLogxK

U2 - https://doi.org/10.1038/s41586-023-06555-x

DO - https://doi.org/10.1038/s41586-023-06555-x

M3 - Article

C2 - 37704728

SN - 0028-0836

VL - 622

SP - 156

EP - 163

JO - NATURE

JF - NATURE

IS - 7981

ER -

A foundation model for generalizable disease detection from retinal images

Abstract

Access to Document

Other files and links

Cite this