How muscles are used is a key to understanding the internal driving of fish swimming. However, the underlying mechanisms of some features of the muscle activation patterns and their differential appearance in different species are still obscure. In this study, we explain the muscle activation patterns by using 3D computational fluid dynamics models coupled to the motion of fish with prescribed deformation and examining the torque and power required along the fish body with two primary swimming modes. We find that the torque required by the hydrodynamic forces and body inertia exhibits a wave pattern that travels faster than the curvature wave in both anguilliform and carangiform swimmers, which can explain the traveling wave speeds of the muscle activations. Notably, intermittent negative power (i.e., power delivered by the fluid to the body) on the posterior part, along with a timely transfer of torque and energy by tendons, explains the decrease in the duration of muscle activation towards the tail. The torque contribution from the body elasticity further clarifies the wave speed increase or the reverse of the wave direction of the muscle activation on the posterior part of a carangiform swimmer. For anguilliform swimmers, the absence of the aforementioned changes in the muscle activation on the posterior part is consistent with our torque prediction and the absence of long tendons from experimental observations. These results provide novel insights into the functions of muscles and tendons as an integral part of the internal driving system, especially from an energy perspective, and they highlight the differences in the internal driving systems between the two primary swimming modes.