Haploid male germ cells package their DNA into a volume that is typically 10% or less that of a somatic cell nucleus. To achieve this remarkable level of compaction, spermatozoa replace most of their histones with smaller, highly basic arginine and (in eutherians) cysteine rich protamines. One reason for such a high level of compaction is that it may help optimise nuclear shape and hence support the gametes' swimming ability for the long journey across the female reproductive tract to the oocyte. Super-compaction of the genome may confer additional protection from the effects of genotoxic factors. However, many species including the human retain a fraction of their chromatin in the more relaxed nucleosomal configuration that appears to run counter to the ergonomic, toroidal and repackaging of sperm DNA. Recent research suggests that the composition of this 'residual' nucleosomal compartment, a generally overlooked feature of the male gamete, is far more significant and important than previously thought. In this respect, the transport and incorporation of modified paternal histones by the spermatozoon to the zygote has been demonstrated and indicates another potential paternal effect in the epigenetic reprogramming of the zygote following fertilisation that is independent of imprinting status. In this review, the most recent research into mammalian spermatozoal chromatin composition is discussed alongside evidence for conserved, non-randomly located nucleosomal domains in spermatozoal nuclei, all supporting the hypothesis that the spermatozoon delivers a novel epigenetic signature to the egg that may be crucial for normal development. We also provide some thoughts on why this signature may be required in early embryogenesis.