The complex frequency-dependent polarizability and π → π* excitation energy of azobenzene compounds are investigated by a combined charge-transfer and point-dipole interaction (CT/PDI) model. To parametrize the model, we adopted time-dependent density functional theory (TDDFT) calculations of the frequency-dependent polarizability extended with excited-state lifetimes to include also its imaginary part. The results of the CT/PDI model are compared with the TDDFT calculations and experimental data demonstrating that the CT/PDI model is fully capable to reproduce the static polarizability as well as the π → π* excitation energy for these compounds. In particular, azobenzene molecules with different functional groups in the para-position have been included serving as a severe test of the model. The π → π* excitation is to a large extent localized to the azo bond, and substituting with electron-donating or electron-attracting groups on the phenyl rings results in charge-transfer effects and a shift in the excitation energy giving rise to azobenzene compounds with a range of different colors. In the CT/PDI model, the π → π* excitation in azobenzenes is manifested as drastically increasing atomic induced dipole moments in the azo group as well as in the adjacent carbon atoms, whereas the shifts in the excitation energies are due to charge-transfer effects.