The output phase and propagation time of an optical signal propagating through a hollow-core optical fiber (HCF) drift with changes in environmental temperature significantly less than in conventional optical fibers. In all earlier experimental studies, however, the simplifying assumption was made that the thermo-optic effect of air was negligible. In this Letter, we present, to the best of our knowledge, the first experimental demonstration that the air inside a HCF core can make an appreciable contribution to the fiber's thermal sensitivity with the performance depending on whether the fiber is open to the atmosphere or sealed at both ends (e.g., spliced to solid fiber pigtails). We measure both the sensitivity of the accumulated phase as well as the signal propagation time for both open and sealed HCF and show that these are opposite in sign. Most importantly, we show that the thermal sensitivity contribution from the air inside an open HCF has the sign opposite to the effect of fiber elongation (which is otherwise the dominant effect responsible for the overall thermal sensitivity of HCF). We then go on to show that these two effects can be used to balance each other out in order to achieve zero thermal sensitivity for both accumulated phase and propagation time. We demonstrate this property experimentally over a large spectral range.