A large number of digital images of gene expression pattern have been produced for documenting the atlas of spatio-temporal expression patterns of genes during Drosophila developmental stage ranges. These images have been annotated with anatomical and developmental ontology terms under a controlled vocabulary by human curators. With the rapid accumulation of images, manually annotating becomes more and more infeasible. Thus, it is highly desired to develop computational methods to automate the annotation task. This poses a significant challenge. That is, the annotations are spatially corresponding to local expression patterns of images, yet they are assigned collectively to groups of images and it is unknown which term corresponds to which region of which image in the group. In this paper, we address the annotation problem under a new machine learning framework, Multi-Instance Multi-Label (MIML) learning. First, we disclose that the underlying nature of the annotation task is a typical MIML problem. Then, we propose two support vector machine algorithms under the MIML framework to implement automatic annotation on the gene expression pattern image data set, FlyExpress, which collects gene expression pattern images produced by the Berkeley Drosophila Genome Project. Experimental results show that solving the annotation problem under the MIML framework can lead to performances superior to state-of-the-art methods.