There is no clear consensus in the literature on the metal accumulation pattern and sensitivity of different earthworm species. In the present study, accumulation and toxicity of Cu, Cd, Ni, and Zn in the earthworms Lumbricus rubellus (epigeic), Aporrectodea longa (anecic), and Eisenia fetida (ultra-epigeic) were determined after 28 days exposure in two soils. Metal accumulation and sensitivity were interpreted using the specific traits of different earthworm species. Results showed that for all four metals tested L. rubellus was the most sensitive species, followed by A. longa and E. fetida. At the same exposure concentration, internal concentrations followed the order: L. rubellus > E. fetida > A. longa for Cu and Ni, L. rubellus ≈ E. fetida ≈ A. longa for Cd, and L. rubellus > A. longa > E. fetida for Zn. Langmuir isotherms were used to model metal accumulation at both nontoxic and toxic exposure concentrations. The Cu, Cd, and Zn concentrations in E. fetida generally leveled off at high exposure concentrations but not for the other two species. A. longa showed a high capability of regulating internal Ni concentrations. The traits-based approaches suggested that most likely a group of earthworm traits together determined (differences in) metal accumulation and sensitivity. More research is needed in this respect to build up solid relationships between species-specific responses and traits, enabling cross-species extrapolation of accumulation and toxicity data.