Lateral flow assays (LFAs) are the cornerstone of point-of-care diagnostics. Although rapid and inexpensive, they are 1000-fold less sensitive than laboratory-based tests and cannot be used for definitive negative diagnosis. Here, we overcome this fundamental limitation by employing plasmonically-enhanced nanoscale colorimetric and fluorescent labels. Plasmonic LFAs (p-LFAs) enabled ultrasensitive detection and quantification of low abundance analytes, without compromising the direct visual detection of conventional LFAs. Dynamic ranges and limits of detection were up to 100-fold superior to "gold standard" ELISA (enzyme-linked immunosorbent assay). p-LFAs had sample-to-answer time of 20 min, compared to 4 hours for ELISA, while achieving over 95% analytical sensitivity and 100% analytical specificity for antibodies and antigens of SARS-CoV-2 in human specimens. We also demonstrate that the p-LFAs enable quantitative detection of the target analytes in a standard-free manner. p-LFAs offer potential as a broadly adaptable point-of-care diagnostic platform that outperforms standard laboratory tests in sensitivity, speed, dynamic range, ease of use, and cost.