Control of membrane fouling is important for more efficient use of membranes in water treatment. Control of physically irreversible fouling, which is defined as fouling that requires chemical cleaning to be cancelled, is particularly important for reduction of operation cost in a membrane process. In this study, a long-term filtration experiment using three different types of MF and UF membranes was carried out at an existing water purification plant, and the evolution of physically irreversible fouling was investigated. The experimental results demonstrated that the extent of physically irreversible fouling differed significantly depending on the membrane type. Cleaning of the fouled membranes with various chemical reagents demonstrated that organic matter was mainly responsible for physically irreversible fouling. Organic matter that had caused physically irreversible fouling in the long-term operation was desorbed from the fouled membranes and was subjected to Fourier transform infrared and 13C nuclear magnetic resonance analyses. These analyses revealed that carbohydrates were dominant in the membrane foulant regardless of the type of membrane. Based on measurements of molecular weight distribution of organic matter in the feedwater and the permeates from the membranes, a two-step fouling mechanism is proposed to explain the dominance of carbohydrates in the foulant: hydrophobic (humic-like) components with small molecular weight are first adsorbed on the membrane and, consequently, narrow the size of micro-pores of membranes, and then hydrophilic (carbohydrate-like) compounds with larger molecular weight plug the narrowed pores or the hydrophilic compounds are adsorbed onto the membrane surface conditioned by the hydrophobic components.