The authors estimated the influence of familial factors and community disadvantage on changes in children's intelligence quotient (IQ) scores from age 6 years to age 11 years. Data were obtained from a longitudinal study of the neuropsychiatric sequelae of low birth weight in two socioeconomically disparate, geographically defined communities in the Detroit, Michigan, metropolitan area. Representative samples of low birth weight and normal birth weight children from the City of Detroit (urban) and nearby middle-class suburbs (suburban) were assessed at age 6 years (in 1990-1992) and age 11 years (in 1995-1997) (n = 717). Children's IQs were measured using the Wechsler Intelligence Scale for Children-Revised. The familial factors considered included maternal IQ, education, and marital status. Multiple regression analysis applying generalized estimating equations was used. The IQs of urban children, regardless of birth weight, declined from age 6 years to age 11 years. The downward shift increased by 50% the proportion of urban children scoring 1 standard deviation below the standardized IQ mean of 100. A negligible change was observed in suburban children. Maternal IQ, education, and marital status and low birth weight predicted IQ at age 6 years but were unrelated to IQ change. Growing up in a racially segregated and disadvantaged community, more than individual and familial factors, may contribute to a decline in IQ score in the early school years.