OBJECTIVE: To evaluate the association between brain amyloid β (Aβ) and cerebral small vessel disease (CSVD) markers, as well as their joint effect on cognition, in a memory clinic study. METHODS: A total of 186 individuals visiting a memory clinic, diagnosed with no cognitive impairment, cognitive impairment no dementia (CIND), Alzheimer dementia (AD), or vascular dementia were included. Brain Aβ was measured by [11C] Pittsburgh compound B-PET global standardized uptake value ratio (SUVR). CSVD markers including white matter hyperintensities (WMH), lacunes, and cerebral microbleeds (CMBs) were graded on MRI. Cognition was assessed by neuropsychological testing. RESULTS: An increase in global SUVR is associated with a decrease in Mini-Mental State Examination (MMSE) in CIND and AD, as well as a decrease in global cognition Z score in AD, independent of age, education, hippocampal volume, and markers of CSVD. A significant interaction between global SUVR and WMH was found in relation to MMSE in CIND (P for interaction: 0.009), with an increase of the effect size of Aβ (β = -6.57 [-9.62 to -3.54], p < 0.001) compared to the model without the interaction term (β = -2.91 [-4.54 to -1.29], p = 0.001). CONCLUSION: Higher global SUVR was associated with worse cognition in CIND and AD, but was augmented by an interaction between global SUVR and WMH only in CIND. This suggests that Aβ and CSVD are independent processes with a possible synergistic effect between Aβ and WMH in individuals with CIND. There was no interaction effect between Aβ and lacunes or CMBs. Therefore, in preclinical phases of AD, WMH should be targeted as a potentially modifiable factor to prevent worsening of cognitive dysfunction.