Homogeneous composites, or metamaterials, made of dielectric or metallic particles are known to show magnetic properties that contradict arguments by Landau and Lifshitz [Landau LD, Lifshitz EM (1960) Electrodynamics of Continuous Media (Pergamon, Oxford, UK), p 251], indicating that the magnetization and, thus, the permeability, loses its meaning at relatively low frequencies. Here, we show that these arguments do not apply to composites made of substances with Im square root(epsilon(S)) >> lambda/l or Re square root(epsilon(S)) approximately lambda/l (epsilon(S) and l are the complex permittivity and the characteristic length of the particles, and lambda >> l is the vacuum wavelength). Our general analysis is supported by studies of split rings, one of the most common constituents of electromagnetic metamaterials, and spherical inclusions. An analytical solution is given to the problem of scattering by a small and thin split ring of arbitrary permittivity. Results reveal a close relationship between epsilon(S) and the dynamic magnetic properties of metamaterials. For |square root(epsilon(S))| << lambda/a (a is the ring cross-sectional radius), the composites exhibit very weak magnetic activity, consistent with the Landau-Lifshitz argument and similar to that of molecular crystals. In contrast, large values of the permittivity lead to strong diamagnetic or paramagnetic behavior characterized by susceptibilities whose magnitude is significantly larger than that of natural substances. We compiled from the literature a list of materials that show high permittivity at wavelengths in the range 0.3-3000 microm. Calculations for a system of spherical inclusions made of these materials, using the magnetic counterpart to Lorentz-Lorenz formula, uncover large magnetic effects the strength of which diminishes with decreasing wavelength.