A theory of the mode-coupling instability (MCI) in a fluid two-dimensional complex plasma is developed. In analogy to the point-wake model of the wake-mediated interactions commonly used to describe MCI in two-dimensional crystals, the layer-wake model is employed for fluids. It is demonstrated that the wake-induced coupling of wave modes occurs in both crystalline and fluid complex plasmas, but the confinement-density threshold, which determines the MCI onset in crystals, virtually disappears in fluids. The theory shows excellent qualitative agreement with available experiments and provides certain predictions to be verified.