The accurate recognition of fetal facial standard plane (FFSP) (i.e., axial, coronal and sagittal plane) from ultrasound (US) images is quite essential for routine US examination. Since the labor-intensive and subjective measurement is too time-consuming and unreliable, the development of the automatic FFSP recognition method is highly desirable. Different from the previous methods, we leverage a general framework to recognize the FFSP from US images automatically. Specifically, instead of using the previous hand-crafted visual features, we utilize the recent developed deep learning approach via very deep convolutional networks (DCNN) architecture to represent fine-grained details of US image. Also, very small (3×3) convolution filters are adopted to improve the performance. The evaluation of our FFSP dataset shows the superiority of our method over the previous studies and achieves the state-of-the-art FFSP recognition results.