A custom built coherent anti-Stokes Raman scattering (CARS) microscope was used to image prostatic glands and nerve structures from 17 patients undergoing radical prostatectomy. Imaging of glandular and nerve structures showed distinctive cellular features that correlated to histological stains. Segmentation of cell nucleus was performed to establish a cell feature-based model to separate normal glands from cancer glands. In this study, we use a single parameter, average cell neighbor distance based on CARS imaging, to characterize normal and cancerous glandular structures. By combining CARS with our novel classification model, we are able to characterize prostate glandular and nerve structures in a manner that potentially enables real-time, intra-operative assessment of surgical margins and neurovascular bundles. As such, this method could potentially improve outcomes following radical prostatectomy.