Enlarged perivascular spaces (EPVS) are a feature of cerebral small vessel disease (cSVD) and have been related to cSVD severity. A higher number of EPVS were related to decreased cognition in healthy elderly, but this has never been investigated in patients at high risk of cSVD. We included 189 patients with a high risk of cSVD (hypertensive patients and lacunar stroke patients). Patients underwent brain MRI and extensive neuropsychological assessment. EPVS were rated in the basal ganglia (BG) and centrum semiovale (CSO). Correlation analyses between EPVS and cognitive domains were adjusted for white matter lesions (WMLs), age, sex and symptomatic stroke. Negative correlations were found between EPVS in the BG and all cognitive domains, independent of WMLs. After correction for age, results remained significant for information processing speed (IPS) only. No independent correlation was found between EPVS in the CSO and cognition. We demonstrated that more BG EPVS were associated with a decrease in IPS, independent of age and WMLs. This emphasizes that specifically EPVS in the BG are associated with cSVD, and with cSVD-related decreases in cognition.