We use an interferometric detection scheme to directly detect single gold nanoparticles with a diameter as small as 5 nm in an aqueous environment. We demonstrate both confocal and wide-field detection of nanoparticles and study signal strength as a function of particle size. Furthermore, we demonstrate a detection speed up to 2 micros. We also show that gold nanoparticles can be readily distinguished from background scatterers by exploiting the wavelength dependence of their plasmon resonances. Our studies pave the way for the application of this detection scheme for particle tracking in biological systems.