Griffiths, Chater, Norris, and Pouget (2012) argue that we have misunderstood the Bayesian approach. In their view, it is rarely the case that researchers are making claims that performance in a given task is near optimal, and few, if any, researchers adopt the theoretical Bayesian perspective according to which the mind or brain is actually performing (or approximating) Bayesian computations. Rather, researchers are said to adopt something more akin to what we called the methodological Bayesian approach, according to which Bayesian models are statistical tools that allow researchers to provide teleological explanations of behavior. In our reply we argue that many Bayesian researchers often appear to be make claims regarding optimality, and often appear to be making claims regarding how the mind computes at algorithmic and implementational levels of descriptions. We agree that some Bayesian theorists adopt the methodological approach, but we question the value of this approach. If Bayesian theories in psychology and neuroscience are only designed to provide insights into teleological questions, we expect that many readers have misunderstood, and hence there is a pressing need to clarify what Bayesian theories of cognition are all about.