For more than a decade, researchers have been trying to develop non-invasive imaging techniques for the in vivo measurement of viable pancreatic beta cells. However, in spite of intense research efforts, only one tracer for positron emission tomography (PET) imaging is currently under clinical evaluation. To many diabetologists it may remain unclear why the imaging world struggles to develop an effective method for non-invasive beta cell imaging (BCI), which could be useful for both research and clinical purposes. Here, we provide a concise overview of the obstacles and challenges encountered on the way to such BCI, in both native and transplanted islets. We discuss the major difficulties posed by the anatomical and cell biological features of pancreatic islets, as well as the chemical and physical limits of the main imaging modalities, with special focus on PET, SPECT and MRI. We conclude by indicating new avenues for future research in the field, based on several remarkable recent results.