Cell-to-cell variation in gene expression, or noise, is a general phenomenon observed within cell populations. Transcription is known to be the key stage of gene expression where noise is generated, however, how variation in RNA polymerase (RNAP) concentration contributes to gene expression noise is unclear. Here, we quantitatively investigate how variations in absolute amounts of RNAP molecules affect noise in the expression of two fluorescent protein reporters driven by identical promoters. We find that intrinsic noise is independent of variation in RNAP concentrations, whereas extrinsic noise, which is variation in gene expression due to varying cellular environments, scales linearly with variation in RNAP abundance. Specifically, the propagation of RNAP abundance variation to expressed protein noise is inversely proportional to the concentration of RNAP, which suggests that the change in noise that results from RNAP fluctuations is determined by the fraction of promoters that is not occupied by RNAP.