Spectroscopic diffuse optical tomography (DOT) can directly image the concentrations of physiologically significant chromophores in the body. This information may be of importance in characterizing breast tumours and distinguishing them from benign structures. This paper studies the accuracy with which lesions can be characterized given a physiologically realistic situation in which the background architecture of the breast is heterogeneous yet highly structured. Specifically, in simulation studies, we assume that the breast is segmented into distinct glandular and adipose regions. Imaging with a high-resolution imaging modality, such as magnetic resonance imaging, in conjunction with a segmentation by a clinical expert, allows the glandular/adipose boundary to be determined. We then apply a two-step approach in which the background chromophore concentrations of each region are estimated in a nonlinear fashion, and a more localized lesion is subsequently estimated using a linear perturbational approach. In addition, we examine the consequences which errors in the breast segmentation have on estimating both the background and inhomogeneity chromophore concentrations.