Approximately 30% of patients with ulcerative colitis require a colectomy during their disease course. This substantially reduces colorectal cancer risk, although it is still possible to develop colorectal neoplasia in the remaining rectum. Although clear and well-accepted surveillance guidelines exist for patients with inflammatory bowel disease with an intact colon, specific surveillance recommendations following colectomy are less clear. Here, we aim to summarize the prevalence, incidence, and risk factors for developing colorectal cancer in patients with inflammatory bowel disease who underwent subtotal colectomy with a permanent end ileostomy and rectal stump, or with ileorectal anastomosis. Subsequently, gained insights are integrated into a proposed endoscopic surveillance strategy of the residual rectum.