Adapting to blurred or sharpened images alters the perceived focus of subsequently viewed images. We examined whether these adaptation effects could arise from actual sphero-cylindrical refractive errors, by testing aftereffects in images simulating second-order astigmatism. Image blur was varied from negative (vertical) through isotropic to positive (horizontal) astigmatism while maintaining constant blur strength. A 2AFC staircase was used to estimate the stimulus that appeared isotropically blurred before or after adapting to images with astigmatism. Adaptation to horizontal blur caused isotropically blurred images to appear vertically biased and vice versa, shifting the perceived isotropic point toward the adapting level. Aftereffects were similar for different types of images and showed partial selectivity so that strongest effects generally occurred when testing and adapting images were the same. Further experiments explored whether the adaptation depended more strongly on the blurring or "fuzziness" in the images vs. the apparent "figural" changes introduced by the blur, by comparing how the aftereffects transfer across changes in size or orientation. Our results suggest that strong selective adaptation can occur for different lower order aberrations of the eye and that these may be at least partly driven by the apparent figural changes that blurring introduces into the retinal image.