Recent years have witnessed an explosion in computational power, leading to attempts to model ever more complex systems. Nevertheless, there remain cases for which the use of brute-force computer simulations is clearly not the solution. In such cases, great benefit can be obtained from the use of physically sound simplifications. The introduction of such coarse graining can be traced back to the early usage of a simplified model in studies of proteins. Since then, the field has progressed tremendously. In this review, we cover both key developments in the field and potential future directions. Additionally, particular emphasis is given to two general approaches, namely the renormalization and reference potential approaches, which allow one to move back and forth between the coarse-grained (CG) and full models, as these approaches provide the foundation for CG modeling of complex systems.