The experience of body ownership can be successfully manipulated during the rubber hand illusion using synchronous multisensory stimulation. The hypothesis that multisensory integration is both a necessary and sufficient condition for body ownership is debated. We systematically varied the appearance of the object that was stimulated in synchrony or asynchrony with the participant's hand. A viewed object that was transformed in three stages from a plain wooden block to a wooden hand was compared to a realistic rubber hand. Introspective and behavioural results show that participants experience a sense of ownership only for the realistic prosthetic hand, suggesting that not all objects can be experienced as part of one's body. Instead, the viewed object must fit with a reference model of the body that contains important structural information about body parts. This body model can distinguish between corporeal and non-corporeal objects, and it therefore plays a critical role in maintaining a coherent sense of one's body.