To improve the detection limits of a portable total reflection X-ray fluorescence (TXRF) spectrometer using white X-rays (i.e., both characteristic X-rays and continuum X-rays) from a 5 W X-ray tube, the measurement was performed in vacuum. The TXRF spectrum measured in vacuum was compared with that measured in air. The spectral background was significantly reduced when the scattering of the incident X-rays from air was reduced using a vacuum pump, leading to improvement in the detection limit. A detection limit of 8 pg was achieved for Cr when measuring in vacuum.