Objective To investigate the coexistence of anterograde and retrograde trans-synaptic axonal degeneration, and to explore the relationship between selective visual pathway damage and global brain involvement in longstanding multiple sclerosis (MS). Methods In this single-centre, cross-sectional study, patients with longstanding MS (N=222) and healthy controls (HC, N=62) were included. We analysed thickness of retinal layers (optical coherence tomography), damage within optic radiations (OR) (lesion volume and fractional anisotropy and mean diffusivity by diffusion tensor imaging) and atrophy of the visual cortex and that of grey and white matter of the whole-brain (structural MRI). Linear regression analyses were used to assess associations between the different components and for comparing patients with and without optic neuritis and HC. Results In patients with MS, an episode of optic neuritis (MSON) was significantly associated with decreased integrity of the ORs and thinning of the peripapillary retinal nerve fibre layer (pRNFL) and macular ganglion cell complex (GCC). Lesion volume in the OR was negatively associated with pRNFL and GCC thickness in patients without optic neuritis (MSNON). The pRNFL and GCC showed associations with integrity of the OR, thickness of the primary visual cortex (only in patients with MSON), and also with global white and grey matter atrophy. In HCs, no such relationships were demonstrated. Interpretation This study provides evidence for presence of bidirectional (both anterograde and retrograde) trans-synaptic axonal degeneration in the visual pathway of patients with MS. Additionally, thinning of the retinal pRNFL and GCC are related to global white and grey matter atrophy in addition to pathology of the visual pathway. © 2014 by the BMJ Publishing Group Ltd.