Evolutionary encephalization, or increasing brain size relative to body size, is assumed to be a general phenomenon in mammals. However, despite extensive evidence for variation in both absolute and relative brain size in extant species, there have been no explicit tests of patterns of brain size change over evolutionary time. Instead, allometric relationships between brain size and body size have been used as a proxy for evolutionary change, despite the validity of this approach being widely questioned. Here we relate brain size to appearance time for 511 fossil and extant mammalian species to test for temporal changes in relative brain size over time. We show that there is wide variation across groups in encephalization slopes across groups and that encephalization is not universal in mammals. We also find that temporal changes in brain size are not associated with allometric relationships between brain and body size. Furthermore, encephalization trends are associated with sociality in extant species. These findings test a major underlying assumption about the pattern and process of mammalian brain evolution and highlight the role sociality may play in driving the evolution of large brains.