Generative adversarial network for coronary artery plaque synthesis in coronary CT angiography

Detection and quantification of atherosclerotic plaque in the coronary arteries is important in cardiovascular risk analysis. Atherosclerotic plaque can be visualized using coronary computed tomography angiography (CCTA). Manual identification and segmentation of plaque is a complex task that requires high level of expertise. Hence, several automatic approaches have been designed. Automatic methods employing deep learning have shown to outperform conventional approaches, but they are hampered by the requirement on the availability of large and diverse training data. To address this, we designed a method that synthesizes calcified and non-calcified atherosclerotic plaque in the coronary arteries. First, we generate plaque geometry using conventional image analysis approach that varies the radius, length and angle of a plaque and we use this to generate a crude inpainting of a plaque on a target artery. Thereafter, we employ a conditional generative adversarial network (GAN) to synthesize the plaque texture in CCTA. The generator is trained to generate fake images with realistic appearance. The discriminator is trained to distinguish the synthesized fake and the real images. The data set for training and evaluation of the plaque synthesis contained CCTA scans of 102 patients (50 training, 52 testing) with manually annotated calcified and non-calcified plaque. To evaluate performance of the synthesis method, we compared CCTA patches with real and synthesized plaque. The evaluation resulted in mean (standard deviation) structural similarity index of 0.99 (0.01), peak signal noise ratio of 73.99 (5.52) and mean absolute error of 5.56 (3.23) HU. To evaluate whether synthesized data enables plaque segmentation, an additional set of CCTA scans of 92 patients without visible plaque was collected. In these scans, plaque was synthesized using the developed approach, containing in total 615 calcified and 544 non-calcified plaque lesions. The synthesized data was used to train a 3D UNet for segmenting calcified and non-calcified plaque lesions. Automatic segmentation which was trained with real data only resulted in Dice coefficients of 0.68 and 0.35 for calcified and non-calcified plaque, respectively. This was significantly improved by pretraining the network with synthetic data and refining it with real data, which resulted in a Dice coefficients of 0.70 (p=0.03) and 0.36 (p=0.02) for calcified and non-calcified plaque, respectively. The results demonstrate that training with CCTA scans with automatically synthesized calcified and non-calcified plaque improves the performance of plaque segmentation.