Stereotactic breast biopsy (SBB) is the gold standard for minimally invasive breast cancer diagnosis. Current systems rely on one of two methods for needle insertion: A vertical approach (perpendicular to the breast compression plate) or a lateral approach (parallel to the compression plate), While the vertical approach is more frequently used, it is not feasible in patients with thin breasts (<3 cm thick after compression) or with superficial lesions. Further, existing SBB guidance hardware provides at most one degree of rotational freedom in the needle trajectory, and as such requires a separate skin incision for each biopsy target. The authors present a new design of lateral guidance device for SBB, which addresses the limitations of the vertical approach and provides improvements over the existing lateral guidance hardware. Specifically, the new device provides (1) an adjustable rigid needle support to minimize needle deflection within the breast and (2) an additional degree of rotational freedom in the needle trajectory, allowing the radiologist to sample multiple targets through a single skin incision. This device was compared to a commercial lateral guidance device in a series of phantom experiments. Needle placement error using each device was measured in agar phantoms for needle insertions at lateral depths of 2 and 5 cm. The biopsy success rate for each device was then estimated by performing biopsy procedures in commercial SBB phantoms. SBB performed with the new lateral guidance device provided reduced needle placement error relative to the commercial lateral guidance device (0.89 +/- 0.22 vs 1.75 +/- 0.35 mm for targets at 2 cm depth; 1.94 +/- 0.20 vs 3.21 +/- 0.31 mm for targets at 5 cm depth). The new lateral guidance device also provided improved biopsy accuracy in SBB procedures compared to the commercial lateral guidance device (100% vs 58% success rate). Finally, experiments were performed to demonstrate that the new device can accurately sample lesions within thin breast phantoms and multiple lesions through a single incision point. This device can be incorporated directly into the clinical SBB procedural workflow, with no additional electrical hardware, software, postprocessing, or image analysis.