We report a time-resolved normal-incidence photoemission electron microscope with an imaging time-of-flight detector using ∼7-fs near-infrared laser pulses and a phase-stabilized interferometer for studying ultrafast nanoplasmonic dynamics via nonlinear photoemission from metallic nanostructures. The interferometer's stability (35 ± 6 as root-mean-square from 0.2 Hz to 40 kHz) as well as on-line characterization of the driving laser field, which is a requirement for nanoplasmonic near-field reconstruction, is discussed in detail. We observed strong field enhancement and few-femtosecond localized surface plasmon lifetimes at a monolayer of self-assembled gold nanospheres with ∼40 nm diameter and ∼2 nm interparticle distance. A wide range of plasmon resonance frequencies could be simultaneously detected in the time domain at different nanospheres, which are distinguishable already within the first optical cycle or as close as about ±1 fs around time-zero. Energy-resolved imaging (microspectroscopy) additionally revealed spectral broadening due to strong-field or space charge effects. These results provide a clear path toward visualizing optically excited nanoplasmonic near-fields at ultimate spatiotemporal resolution.