We develop a simple and effective description of the dynamics of dense hard sphere colloids in the aging regime deep in the glassy phase. Our description complements the many efforts to understand the onset of jamming in low density colloids, whose dynamics is still time-homogeneous. Based on a small set of principles, our model provides emergent dynamic heterogeneity, reproduces the known results for dense hard sphere colloids and makes detailed, experimentally-testable predictions for canonical observables in glassy dynamics. In particular, we reproduce the shape of the intermediate scattering function and particle mean-square displacements for jammed colloidal systems, and we predict a growth for the peak of the χ(4) mobility correlation function that is logarithmic in waiting-time. At the same time, our model suggests a novel unified description for the irreversible aging dynamics of structural and quenched glasses based on the dynamical properties of growing clusters of highly correlated degrees of freedom.