Triad sampling in household surveys

CITATION: Aldworth, J., & Chromy, J. (2004). Triad sampling in household surveys. In Proceedings of the 2004 Joint Statistical Meetings, American Statistical Association, Section on Survey Research Methods, Toronto, Canada [CD-ROM]. Alexandria, VA: American Statistical Association.

PURPOSE/OVERVIEW: The motivation for selecting three individuals from a dwelling unit (DU) derives from an interest in studying the behavioral relationships among certain triads (e.g., two parents and a child, a parent and two children). Computer-assisted DU screening provides the mechanism for targeted sampling of individuals, pairs, and even triads within a DU. Chromy and Penne (2002) showed how a modification of Brewer’s (1963, 1974) method for samples of size two was used to select samples of 0, 1, or 2 individuals from eligible DUs. They also developed a second adaptation to control the number of pairs selected.

METHODS: Chromy and Penne’s modification of Brewer’s method and their adaptation to control the number of individuals selected from a DU is extended to the case of sampling triads within DUs. Some empirical data on household roster composition, and response rates based on the number of individuals selected, are presented from the National Survey on Drug Use and Health (NSDUH).

RESULTS/CONCLUSIONS: The results of simulations of the sample selection and response process are presented as a means of evaluating alternatives. Possible negative impacts of triad selections on design effect, response rates, and response bias were assessed. The results indicate that although pair selections in the 2002 NSDUH show little negative impact, a field test assessment is needed for triad assessment.

Estimating substance abuse treatment need by state [editorial]

CITATION: Gfroerer, J., Epstein, J., & Wright, D. (2004). Estimating substance abuse treatment need by state [editorial]. Addiction, 99(8), 938–939. [PubMed: 15265089]

PURPOSE/OVERVIEW: This editorial comments on the methods used by the Substance Abuse and Mental Health Services Administration (SAMHSA) to measure the need for the treatment of substance use disorders at the State level.

METHODS: The two methods compared are the one used in the National Survey on Drug Use and Health (NSDUH) and the index method proposed by W. E. McAuliffe and R. Dunn in this issue of Addiction. The authors describe the strengths and weaknesses of each of these methods.

RESULTS/CONCLUSIONS: The strengths of NSDUH are that it uses an independent probability sample for each State, data collection procedures and methods are defined and carried out consistently across States, and the data are current, reflecting well-defined time periods. A validation study also has demonstrated that NSDUH has small biases and allows for estimates of different subgroups. The main limitations are the small sample sizes in each State and possible underreporting and undercoverage. The primary weakness of the index method is that their measures are not well defined and therefore not consistently interpreted. For these reasons, the NSDUH is recognized as more valid than the index method, but the NSDUH results are meant to be interpreted along with the results from other data sources.

Estimating trends in substance use based on reports of prior use in a cross-sectional survey

CITATION: Gfroerer, J., Hughes, A., Chromy, J., Heller, D., & Packer, L. (2004). Estimating trends in substance use based on reports of prior use in a cross-sectional survey. In S. B. Cohen & J. M. Lepkowski (Eds.), Eighth Conference on Health Survey Research Methods (HHS Publication No. PHS 04-1013, pp. 29–34). Hyattsville, MD: U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Center for Health Statistics.

PURPOSE/OVERVIEW: Substance use trends in the United States have shown dramatic shifts since the 1960s. Among youths aged 12 to 17, the rate of past month marijuana use was less than 2 percent in the early 1960s, increased to 14 percent by 1979, then decreased to 3.4 percent in 1992 before rising to 8.2 percent in 1995. Major shifts in prevalence at different points in time and for different age groups have been observed for other substances, including cocaine, LSD, Ecstasy, opiates, cigars, and cigarettes (Substance Abuse and Mental Health Services Administration [SAMHSA], 2003). Accurate measurement of these trends is critical for policymakers targeting limited resources efficiently toward emerging problems. Trend data also are used for assessing the impact of prevention and treatment programs. The typical method used for measuring substance use trends is comparing prevalence estimates across repeated cross-sectional surveys. An alternative approach is to collect data about prior substance use within a cross-sectional survey and to construct prevalence estimates for prior years based on these data. Besides the cost advantages, these retrospective estimates have some analytic advantages. When data are obtained for different periods from the same respondents, trend analyses are more powerful, due to the positive correlation between estimates, as is the case in a longitudinal study. Retrospective estimates also may be the only alternative if estimates are needed for periods for which direct estimates are not available. However, retrospective estimates do have important limitations. Bias due to recall decay, telescoping, and reluctance to admit socially undesirable behaviors could cause underestimation or distort trends (Johnson, Gerstein, & Rasinski, 1998; Kenkel, Lillard, & Mathios, 2003). Bias also could result from coverage errors affecting the capability of the sample to represent the population of interest for prior time periods, due to mortality, immigration, or other changes in the population. This paper discusses several types of retrospective estimates and presents analyses of data from the National Survey on Drug Use and Health (NSDUH) to assess biases in these estimates.

METHODS: Several analyses were undertaken to assess bias in retrospective estimates. One known source of bias in retrospective estimates is the inclusion of data from immigrants who were not living in the United States in some prior years. The authors compared estimates of incidence and lifetime use (for those aged 12 to 17 or 18 to 25) for the full 2002 NSDUH sample with estimates based on the sample excluding these immigrants, according to questions on country of birth and years in the United States.

Trends in incidence estimates for 1965 to 1990 based on 1991 to 1993 data (shortest recall), 1994 to 1998 data, 1999 to 2001 data, and 2002 data (longest recall) were compared. For the 1991 to 1997 period, trends based on the 1994 to 1998 data, 1999 to 2001 data, and 2002 data were compared. Consistency was assessed through visual inspection of curves and with correlations. Because of methodology changes, comparisons of levels from different sets of surveys were not made.

The authors compared 2002-based retrospective lifetime use estimates (excluding immigrants) with direct lifetime use estimates from earlier NSDUHs (for those aged 12 to 17 or 18 to 25) and from the Monitoring the Future (MTF) study, a survey of high school seniors (Johnston, O’Malley, & Bachman, 2003). To reduce the effect of sampling error, the authors combined several years of data, depending on availability, and generated average annual lifetime prevalence for specific time periods. Because 1999 and 2002 survey changes resulted in increased reporting of lifetime use, the authors expected retrospective estimates to be greater than the direct estimates for years before 1999.

The authors compared retrospective lifetime use estimates for 2002, based on 2003 NSDUH data (first 6 months of data currently available), to direct 2002 lifetime use estimates, from the 2002 NSDUH (first 6 months of data, for consistency). Comparisons were made for 19 substances for those aged 12 to 17 or 18 to 25.

To assess the accuracy of these estimates, the authors compared January to June 2003-based retrospective estimates of past year use in January to June 2002 to direct past year estimates from the January to June 2002 data, by age group.

RESULTS/CONCLUSIONS: Marijuana incidence estimates for 1965 to 2001 were 2.5 percent higher when immigrants were included. For most other illicit drugs, the bias was smaller, indicating that very little initiation for these drugs occurs among immigrants prior to their entry to the United States. However, biases for alcohol and cigarette incidence estimates were larger (8 percent for alcohol, 7 percent for cigarettes). In general, they were largest for the years 1979 to 1994 (3.5 percent for marijuana, 11 percent for alcohol, 10 percent for cigarettes) and smallest for years after 1997 (1 percent for marijuana, 3 percent for alcohol, and 3 percent for cigarettes). For lifetime prevalence rates, bias due to including immigrants was negative for nearly every substance because of the low rates of substance use among immigrants. For youth estimates during the period from 1979 to 1990, the inclusion of immigrants resulted in biases of about −14 percent for marijuana, −15 percent for cocaine, −9 percent for cigarettes, and −9 percent for alcohol. Bias was generally worse for estimates for those aged 12 to 17 than for those aged 18 to 25, and there was very little bias in any estimates for years after 1997. Estimates of alcohol use for those aged 18 to 25 including immigrants showed very small but positive bias (1.5 percent) for the period from 1982 to 1993 and a larger positive bias (7 percent) for 1965 to 1981.

A simple evaluation of the imputation procedures used in NSDUH

CITATION: Grau, E., Frechtel, P., Odom, D., & Painter, D. (2004). A simple evaluation of the imputation procedures used in NSDUH. In Proceedings of the 2004 Joint Statistical Meetings, American Statistical Association, Section on Survey Research Methods, Toronto, Ontario, Canada [CD-ROM]. Alexandria, VA: American Statistical Association.

PURPOSE/OVERVIEW: The National Survey on Drug Use and Health (NSDUH) is the primary source of information on drug use in the United States. Since 1999, the predictive mean neighborhood (PMN) procedure has been used to impute missing values for many of the analytical variables. This method is a combination of two commonly used imputation methods: a nearest neighbor hot-deck and a modification of Rubin’s predictive mean matching method. Although PMN has many practical advantages, it has not been evaluated formally.

METHODS: The authors proposed a simple simulation to evaluate PMN. Using only complete data cases, they induced random patterns of missingness in the data for selected outcome variables. Imputations then were conducted using PMN and a weighted nearest neighbor hot deck. This process of inducing missingness and imputing missing values was repeated multiple times. The imputed values using PMN and the weighted hot deck then were compared with the true values found in the complete data across the repeated iterations. In particular, the authors compared the number of matches between the two methods, as well as compared statistics derived from the data, such as drug prevalence estimates.

RESULTS/CONCLUSIONS: This evaluation showed that using PMN provided a modest advantage in the match test, and, in the missing completely at random (MCAR) case, using weighted sequential hot deck (WSHD) provided a modest advantage in the mean test. In each of these cases, however, the degree of difference between the methods, though significant, was not substantial. Not surprisingly, all methods did badly in the mean test when the data were not missing at random (NMAR).

Measuring treatment needs: A reply to Gfroerer, Epstein, and Wright

CITATION: McAuliffe, W. E. (2004). Measuring treatment needs: A reply to Gfroerer, Epstein, and Wright. Addiction, 99(9), 1219–1220. [PubMed: 15317643]

PURPOSE/OVERVIEW: This commentary is a reply to the editorial titled “Estimating Substance Abuse Treatment Need by State” by Joe Gfroerer, Joan Epstein, and Doug Wright. The author commented on the strengths and weaknesses of studies used to measure the treatment need for substance use disorders.

METHODS: The two studies compared were the National Survey on Drug Use and Health (NSDUH) and the Substance Abuse Need Index (SNI) produced by W. E. McAuliffe and R. Dunn (2004). The author discussed the strengths and weaknesses of each approach.

RESULTS/CONCLUSIONS: NSDUH suffers from underreporting, undercoverage, and nonresponse, which likely leads to undercounting users of hard-core drugs, such as cocaine. The SNI more accurately captures the incarcerated and homeless populations, improving their estimates for cocaine use and drug abuse. However, the NSDUH estimates probably are more accurate for marijuana use. The author disagreed with Gfroerer et al. that the SNI was less reliable or valid than NSDUH and expressed the opinion that multiple data sources need to be considered together to get the full picture on drug abuse and need for treatment.

Substance abuse treatment needs and access in the USA: Interstate variations

CITATION: McAuliffe, W. E., & Dunn, R. (2004). Substance abuse treatment needs and access in the USA: Interstate variations. Addiction, 99(8), 999–1014. [PubMed: 15265097]

PURPOSE/OVERVIEW: This paper analyzed two measures for substance abuse treatment needs across States. They were used to explore geographic variations in substance abuse, the causes for substance abuse, the stability of these estimates over time, and whether the severity of substance abuse was correlated to need.

METHODS: This study used alcohol dependency estimates from the National Survey on Drug Use and Health (NSDUH), and drug and alcohol use indexes based on alcohol-related mortality and arrests data, to measure interstate differences between substance abuse treatment needs and treatment services. This study tested the reliability and the validity of the survey measures used. The index for substance abuse treatment then was regressed on the measures for substance abuse need to identify differences in availability of treatment across States.

RESULTS/CONCLUSIONS: The individual indicators of treatment needs and availability of treatment for substance abuse across the United States had reliability and construct validity. Substance use problems in the United States are clustered geographically, with the most severe problems appearing in the western States. The biggest gaps between treatment need and treatment access appeared in the South where there was moderate need for treatment, but very low access. The interstate discrepancies in treatment need versus treatment services indicated that substance use problems in rural areas are often overlooked by treatment services.

Prevalence of adult binge drinking: A comparison of two national surveys

CITATION: Miller, J. W., Gfroerer, J. C., Brewer, R. D., Naimi, T. S., Mokdad, A., & Giles, W. H. (2004). Prevalence of adult binge drinking: A comparison of two national surveys. American Journal of Preventive Medicine, 27(3), 197–204. [PubMed: 15450631]

PURPOSE/OVERVIEW: This paper compares two national surveys—the Behavioral Risk Factor Surveillance System (BRFSS) and the National Survey on Drug Use and Health (NSDUH)—that measure the prevalence of binge drinking across States and overall for the United States. The authors examine methodological differences between the two studies to assess their impact on the survey estimates.

METHODS: The main methodological difference between the two studies is that BRFSS is conducted over the telephone and NSDUH is conducted in-person using audio computer-assisted self-interviewing (ACASI). BRFSS assesses binge drinking using three alcohol questions, and NSDUH uses nine; however, the questions are very similar. Because BRFSS uses a telephone sample and NSDUH is an in-person survey, data from NSDUH were limited to only telephone-households to make comparisons consistent. In addition, NSDUH was restricted to respondents aged 18 years or older to match BRFSS. Response rates, survey size, and other characteristics of the two studies were compared.

RESULTS/CONCLUSIONS: BRFSS binge drinking estimates for the United States and most States were considerably lower than the NSDUH estimates, even after stratifying for sociodemographic variables. The demographic characteristics of the sample were very similar; the majority were male and white, non-Hispanic. However, there were no significant differences for binge drinking in the past 30 days between the two studies. The large differences in estimates was likely due to the use of ACASI in NSDUH, which is perceived as more anonymous and yields higher reporting of sensitive behaviors. Other possible explanations for the differences are the sample size, the number of questions asked about alcohol, and the overall topic of the survey.

A system for detecting interviewer falsification

CITATION: Murphy, J., Baxter, R. K., Eyerman, J., Cunningham, D., & Barker, P. (2004). A system for detecting interviewer falsification. In Proceedings of the 2004 Joint Statistical Meetings, American Statistical Association, Section on Survey Research Methods, Toronto, Ontario, Canada [CD-ROM]. Alexandria, VA: American Statistical Association.

PURPOSE/OVERVIEW: The National Survey on Drug Use and Health (NSDUH) has developed a detailed system for the detection of interviewer falsification. This process includes phone, mail, and in-person field verification procedures, as well as the review of interview and interviewer-level process data to identify cases and interviewers requiring extra verification efforts. Although these components of the system successfully identify a majority of potential falsifiers, more savvy falsifiers may be able to remain undetected if they are aware that their process data are being scrutinized. To address this gap, NSDUH added a new component to the falsification detection system: the regular review of interview response and question-level timing data. This paper details the structure and operationalization of this system and presents examples of its effectiveness on NSDUH.

METHODS: Based on what is known about the area in which an interviewer is working and the types of cases he or she is assigned, a likely range of interview responses is calculated. Response distributions are compared with this likely range at the interviewer level to identify interviewers whose responses appear to be highly unlikely, given their caseloads. These additional measures make it even more difficult for falsifiers to remain undetected because they would need to have a specific understanding of the prevalence and correlates of substance use in order to enter likely responses. Similarly, question-level timings for particular items that require certain interviewer-respondent interactions are compared with a “gold standard” to detect outliers. Once potential falsifiers are identified, the work of the suspected interviewers is subject to 100 percent and/or in-person verification.

RESULTS/CONCLUSIONS: The analyses in this paper showed that falsification detection can be improved through the systematic review of response data and metadata, such as module and item timings. Through early detection and remediation, the threat of falsification to survey bias and increased costs can be reduced. Although the NSDUH system has improved based on these recent enhancements, there are still many more enhancements that could be incorporated. In particular, the authors planned to assess the following techniques for possible adoption: (1) analysis of screening data, (2) analysis of record of calls data and other metadata, (3) a data mining approach, and (4) statistical process control.

Nonresponse among persons age 50 and older in the National Survey on Drug Use and Health

CITATION: Murphy, J., Eyerman, J., & Kennet, J. (2004). Nonresponse among persons age 50 and older in the National Survey on Drug Use and Health. In S. B. Cohen & J. M. Lepkowski (Eds.), Eighth Conference on Health Survey Research Methods (HHS Publication No. PHS 04-1013, pp. 73–78). Hyattsville, MD: U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Center for Health Statistics.

PURPOSE/OVERVIEW: Response rates traditionally have been highest among the youngest respondents and lowest among the oldest, with the lowest rates found in the 50 or older (50+) age group. The introduction in 2002 of a series of methodological enhancements to the National Survey on Drug Use and Health (NSDUH) appeared to improve the response rates for most age groups but had only a small impact on the 50+ age group (Kennet, Gfroerer, Bowman, Martin, & Cunningham, 2003). Because lower response rates make nonresponse bias more likely, and because there is a disturbingly low response rate among the 50+ group, this paper aims to understand why this may be in order to understand how the problem can be ameliorated. This topic is of increasing importance as the proportion of Americans in this age group increases (U.S. Bureau of the Census, 1999). Obtaining unbiased survey estimates will be vital to assess accurately the substance abuse treatment need for older Americans in the coming years. This need is expected to nearly triple by 2020 as the baby boom generation carries its alcohol and drug use into older ages (Gfroerer, Penne, Pemberton, & Folsom, 2002). The purpose of this paper is to provide a better understanding of nonresponse among older sample members in NSDUH in order to tailor methods to improve response rates and reduce the threat of nonresponse error.

METHODS: This paper examines the components of nonresponse (refusals, noncontacts, and/or other incompletes) among the 50+ age group in NSDUH. It also examines respondent, environmental, and interviewer characteristics in order to identify the correlates of nonresponse among the 50+ group, including relationships that are unique to the 50+ group. Finally, this paper considers the root causes for differential nonresponse by age, drawing from focus group sessions with NSDUH field interviewers on the topic of nonresponse among the 50+ group.

RESULTS/CONCLUSIONS: This paper showed that nonresponse in NSDUH was higher among the 50+ group than among any other age group and was primarily due to a high rate of refusals, especially among sample members aged 50 to 69, and a high rate of physical and mental incapability among those 70 or older. Taken together with evidence from interviewer focus groups, it appeared that the higher rate of refusal among the 50+ age group may, in part, have been due to fears and misperceptions about the survey and interviewers’ intentions. Increased public awareness about the study may allay these fears. Although an increase in the incentive amount may not automatically increase response rates among this group, the authors concluded that other protocol changes and methodological enhancements may be effective.

Imputation and unbiased estimation: Use of the centered predictive mean neighborhoods method

CITATION: Singh, A., Grau, E., & Folsom, R. (2004). Imputation and unbiased estimation: Use of the centered predictive mean neighborhoods method. In Proceedings of the 2004 Joint Statistical Meetings, American Statistical Association, Section on Survey Research Methods, Toronto, Ontario, Canada [CD-ROM]. Alexandria, VA: American Statistical Association.

PURPOSE/OVERVIEW: Methods for determining the predictive distribution for multivariate imputation range between two extremes, both of which are commonly employed in practice: a completely parametric model-based approach, and a completely nonparametric approach, such as the nearest neighbor hot deck (NNHD). A semiparametric middle ground between these two extremes is to fit a series of univariate models and construct a neighborhood based on the vector of predictive means. This is what is done under the predictive mean neighborhood (PMN) method, a generalization of Rubin’s predictive mean matching method. Because the distribution of donors in the PMN neighborhood may not be centered at the recipient’s predictive mean, estimators of population means and totals could be biased. To overcome this problem, the authors propose a modification to PMN that uses sampling weight calibration techniques, such as the generalized exponential model (GEM) method of Folsom and Singh to center the empirical distribution from the neighborhood.

METHODS: Empirical results on bias and mean squared error (MSE), based on a simulation study using data from the 2002 National Survey on Drug Use and Health, are presented to compare the centered PMN with other methods.

RESULTS/CONCLUSIONS: Although it had been theorized that bias could be a problem for existing methods, this simulation study was unable to detect any meaningful bias with any of the methods. Furthermore, none of the methods showed a consistent pattern of higher or lower variance beyond what was expected.

Combined-year state-level public use files and single-year nation-level PUFs from the National Survey of [sic] Drug Use and Health (NSDUH) data

CITATION: Wright, D., & Singh, A. (2004). Combined-year state-level public use files and single-year nation-level PUFs from the National Survey of [sic] Drug Use and Health (NSDUH) data. In Proceedings of the 2004 Joint Statistical Meetings, American Statistical Association, Section on Survey Research Methods, Toronto, Ontario, Canada [CD-ROM]. Alexandria, VA: American Statistical Association.

PURPOSE/OVERVIEW: Since 1999, the Substance Abuse and Mental Health Services Administration (SAMHSA) has provided yearly national public use files (PUFs) for the National Survey on Drug Use and Health (NSDUH) data using a procedure based on the micro agglomeration, substitution, subsampling, and calibration (MASSC) system for statistical disclosure limitation. There is a growing demand for State-level data, and SAMHSA is considering providing State-level PUFs based on combining several years of NSDUH data. The authors explore various concerns and approaches to State-level PUFs and indicated how MASSC could address some of them.

METHODS: Releasing combined-year State-level PUFs alongside single-year national PUFs poses several challenges. The most important one is that confidentiality of an individual could be compromised if an intruder were able to match the State-level PUFs with the national PUFs on the basis of various sensitive variables that are typically not perturbed, and thus may succeed in attaching State identifiers to the national PUFs. This problem can be reduced by taking advantage of the randomness in perturbation and suppression used in MASSC.

RESULTS/CONCLUSIONS: In this paper, the authors suggest a way in which State-level PUFs could be created if one is willing to combine data over several years. The State PUFs would provide a wealth of information for each State for the calculation of point estimates and for analyzing relationships within each State.