An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Substance Abuse and Mental Health Services Administration . National Survey on Drug Use and Health: Summary of Methodological Studies, 1971–2014 [Internet]. Rockville (MD): Substance Abuse and Mental Health Services Administration (US); 2014 Nov.
CITATION: Biemer, P. P., & Witt, M. (1997). Repeated measures estimation of measurement bias for self-reported drug use with applications to the National Household Survey on Drug Abuse. In L. Harrison & A. Hughes (Eds.), The validity of self-reported drug use: Improving the accuracy of survey estimates (NIH Publication No. 97-4147, NIDA Research Monograph 167, pp. 439–476). Rockville, MD: National Institute on Drug Abuse. [PubMed: 9243573]
PURPOSE/OVERVIEW: Direct estimates of response bias for self-reports of drug use in surveys require that essentially error-free determinations of drug use be obtained for a subsample of survey respondents. The difficulty of obtaining determinations that are accurate enough for estimating validity is well documented in the literature. Methods such as specimen (e.g., hair, urine) analysis, proxy reports, and the use of highly private and anonymous modes of interview all have to contend with error rates that may be only marginally lower than those of the parent survey. Thus, any methodology for direct validity estimation must rely to some extent on approximations and questionable assumptions.
METHODS: In this chapter, the authors considered a number of methods that rely solely on repeated measures data to assess response bias. Because the assumptions associated with these approaches do not require highly accurate second determinations, they may be more easily satisfied in practice. One such method for bias estimation for dichotomous variables that is considered in some detail provides estimates of misclassification probabilities in the initial measurement without requiring that the second measure be accurate or even better than the first. This methodology does require, however, that two subpopulations exist that have different rates of prevalence but whose probabilities of false-positive and false-negative error are the same. The applicability of these methods for self-reported drug use are described and illustrated using data from the National Household Survey on Drug Abuse (NHSDA).
RESULTS/CONCLUSIONS: In this chapter, a general model for studying misclassification in self-reported drug use was presented, and the model then was extended to the case where two measurements of the same characteristic are available for the sample of respondents. For the two-measurements case, the general model required seven parameters while only three degrees of freedom were available for estimation. Thus, some additional assumptions were required to reduce the set of unknown parameters to three or less. It was shown how the assumptions typically made for test-retest, true value, improved value, and Hui-Walter methods relate to the general model. Further, it was shown how the measures of reliability, measurement bias, estimator bias, mean squared error, false-negative probability, and false-positive probability can be defined in the context of the general model and how they may be estimated under the appropriate study designs.
Finally, the use of Hui and Walter’s method for estimating misclassification error based on two erroneous reports was demonstrated. The reports may be self-reports, biological tests, administrative record values, or any other measure. For the general case of two measurements, the Hui-Walter method used maximum likelihood estimation to obtain estimates of the false-negative and false-positive probabilities associated with each measurement, as well as the error-adjusted estimates of prevalence based on both measurements. The method required that the population be divided into two domains that have markedly different prevalence estimates and that satisfy the assumption of homogeneity of error probabilities.
CITATION: Folsom, R. E., & Judkins, D. R. (1997). Substance abuse in states and metropolitan areas: Model based estimates from the 1991–1993 National Household Surveys on Drug Abuse: Methodology report (HHS Publication No. SMA 97-3140, Methodology Series M-1). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.
PURPOSE/OVERVIEW: This report presents estimates of substance abuse for 26 States and 25 metropolitan statistical areas (MSAs). These estimates were developed from data collected in the National Household Survey on Drug Abuse (NHSDA) combined with local area indicators from a variety of sources. They were produced by the Substance Abuse and Mental Health Services Administration (SAMHSA) to provide State and local area policymakers with information on the prevalence of substance abuse behaviors and problems in their local areas. These estimates were an inexpensive alternative to the direct survey approach for describing substance abuse in State and local areas. They were based on a consistent methodology across areas and were constructed so that they summed to national estimates produced by the NHSDA.
METHODS: These State and MSA estimates were the result of a comprehensive small area estimation (SAE) project that included the development of an innovative methodology based on the methods used by other Federal agencies to meet needs for small area data. The methodology employed logistic regression models that combine NHSDA data with local area indicators, such as drug-related arrests, alcohol-related death rates, and block group–level characteristics from the 1990 census that were found to be associated with substance abuse. Work also was carried out to evaluate the model used to produce these estimates.
RESULTS/CONCLUSIONS: Considering a variety of evidence, the following conclusions were made: (1) The SAE model produced estimates of all measures that were much better than States could achieve by simply applying NHSDA national prevalence rates for demographic subgroups to the population distribution in their States. (2) A preponderance of evidence indicated that the estimates for alcohol, cigarette, and any illicit drug use were good in that they adequately reflected both levels of use and differences across States and MSAs in the level of use. (3) For past month use of any illicit drug but marijuana, past month use of cocaine, past year dependency on illicit drugs and dependency on alcohol, and need for treatment, the limited evidence indicated that these estimates also were reasonably good. (4) For arrests, past year treatment for illicit drug use, and past year treatment for alcohol, the quality assessments resulted in mixed findings, and these small area data may not have been good indicators of either differences between States or between MSAs, except in broad terms, or of the actual levels.
CITATION: Gfroerer, J., Lessler, J., & Parsley, T. (1997). Studies of nonresponse and measurement error in the National Household Survey on Drug Abuse. In L. Harrison & A. Hughes (Eds.), The validity of self-reported drug use: Improving the accuracy of survey estimates (NIH Publication No. 97-4147, NIDA Research Monograph 167, pp. 273–295). Rockville, MD: National Institute on Drug Abuse. [PubMed: 9243566]
PURPOSE/OVERVIEW: A summary of the results of a series of studies of nonresponse and measurement error in the National Household Survey on Drug Abuse (NHSDA) is given in this chapter. Two studies not previously reported, the Skip Pattern Experiment and the Census Match Study, are the primary focus of the chapter. The Skip Pattern Experiment involved a test of a modified NHSDA questionnaire that made extensive use of skip patterns in drug use questions. Compared with the standard NHSDA method, which avoided skip patterns, the modified questionnaire tended to produce lower rates of reported drug use. The Census Match Study involved linking 1990 NHSDA nonrespondent cases with data from the 1990 decennial census. Household and individual data for NHSDA nonrespondents were obtained from the census and used to characterize NHSDA nonresponse patterns in detail.
METHODS: A multilevel logistic model of response propensity identified the important predictors of nonresponse, including characteristics of the sampled person, the selected household, the neighborhood, and the interviewer.
RESULTS/CONCLUSIONS: Drug abuse surveys are particularly vulnerable to nonresponse and measurement error because of the difficulties in accessing heavy drug users and the likelihood that the illegal and stigmatized nature of drug abuse may lead to underreporting. The Skip Pattern Experiment confirmed once again that respondents’ reporting of their drug use behavior was highly sensitive to the conditions under which they report. This conclusion made clear the need to proceed with great caution in interpreting differences in drug use rates obtained in different surveys. It also suggested caution in the implementation of new technologies, such as computer-assisted data collection, that undoubtedly will have some as yet unknown effect on respondents’ willingness to report their drug use. The Skip Pattern Experiment may have implications in the introduction of these new technologies because one of the advantages of computer-assisted interviewing is the ease with which skips can be implemented. The Census Match Study demonstrated that response rates were not constant across various interviewer, respondent, household, and neighborhood characteristics. To the extent that rates of drug use varied by these same characteristics, bias due to nonresponse may be a problem. However, it was not always the case that low response rates occurred in conjunction with high drug use prevalence. Some populations with low response rates (e.g., older adults and high-income populations) tended to have low rates of drug use. On the other hand, some populations (e.g., residents of large metropolitan areas and men) had low response rates and high drug use rates. In estimating overall prevalence, many of these potential sources of bias would be in opposite directions and would therefore tend to cancel each other.
CITATION: Gfroerer, J., Wright, D., & Kopstein, A. (1997). Prevalence of youth substance use: The impact of methodological differences between two national surveys. Drug and Alcohol Dependence, 47(1), 19–30. [PubMed: 9279494]
PURPOSE/OVERVIEW: The purpose of this paper is to analyze the differences in the methodology used in two major federally sponsored surveys of drug use among youths.
METHODS: This study compares the prevalence rates yielded by the Monitoring the Future (MTF) study and the National Household Survey on Drug Abuse (NHSDA) by comparing the differences in response rates, precision, coverage, and data collection methods.
RESULTS/CONCLUSIONS: Although the MTF has a larger sample size of youths, an analysis of precision estimates reveals that its precision is similar to that of the NHSDA. After controlling for ages used, time of data collection, and dropouts, estimates of alcohol and drug use were significantly lower in the NHSDA than in the MTF. Although the exact cause of these differences could not be determined, the most likely reasons were the different interview environments (home vs. school), the questionnaires, and nonresponse bias in the MTF.
CITATION: Harrell, A. V. (1997). The validity of self-reported drug use data: The accuracy of responses on confidential self-administered answer sheets. In L. Harrison & A. Hughes (Eds.), The validity of self-reported drug use: Improving the accuracy of survey estimates (NIH Publication No. 97-4147, NIDA Research Monograph 167, pp. 37–58). Rockville, MD: National Institute on Drug Abuse. [PubMed: 9243556]
PURPOSE/OVERVIEW: Official records offer a relatively inexpensive, nonintrusive strategy for checking on the accuracy of self-reported drug use. The study reported here was designed as a criterion validity test of the National Household Survey on Drug Abuse (NHSDA) procedures. In criterion validity studies, two different measures of the same trait or experience are available: a candidate measure and an external, independent criterion measure that is treated as an error-free measure of the construct.
METHODS: In this study, the underreporting of illegal drug use was investigated in a sample of 67 former drug treatment clients by comparing their survey responses with clinic records on drug problems at time of admission. The criterion measures were based on self-reported marijuana, cocaine, hallucinogens, and heroin use. Drug treatment records were obtained from the files of publicly funded drug treatment programs in three States. The study followed the NHSDA interviewing and questionnaire procedures closely. To avoid bias from interviewer expectations and to protect the respondent’s privacy, the sample of treatment clients was embedded in a larger sample of respondents. Interviewers were not told that the respondents had been treated for drug abuse. Special sample selection directions, tailored to match the target respondent’s age and gender, were used to select the former drug treatment client within the household, simulating the random selection screening instrument used in NHSDA. This analysis compared reports of past year and lifetime use of marijuana, cocaine, hallucinogens, and heroin by the former drug treatment clients with the drugs listed as problematic at time of admission to treatment. This analysis also examined factors that might have influenced the respondents’ willingness or ability to respond accurately, such as the level of privacy during the interview and the amount of time between admission to the program and the interview.
RESULTS/CONCLUSIONS: The accuracy of reports compared with clinic records varied by drug, with the percentage of known users reporting their use highest for marijuana, followed by cocaine and hallucinogens, and lowest for heroin. Almost half of this sample of former treatment clients denied ever receiving drug treatment.
CITATION: Johnson, R. A., Gerstein, D. R., & Rasinski, K. A. (1997). Recall decay and telescoping in self-reports of alcohol and marijuana use: Results from the National Household Survey on Drug Abuse (NHSDA). In Proceedings of the 1997 Joint Statistical Meetings, 52nd annual conference of the American Association for Public Opinion Research, Norfolk, VA (pp. 964–969). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: To a large extent in the mid-1990s, the knowledge of life-cycle patterns of drug use in the United States was based on retrospective self-reports of survey respondents. Although most evidence for validity came from studies comparing self-reports of individuals dependent on narcotics with hospital and criminal justice records and with the results of urine tests (Nurco, 1985), such measures were too expensive to obtain in general population surveys. Instead, many general population surveys used the reinterview design to evaluate response error.
METHODS: The authors used a repeated cross-sectional design to evaluate response errors and analyzed changes in the distribution of responses of the same birth cohorts as measured in cross-sectional surveys conducted in different years. There were advantages and disadvantages to this method. The advantage is that pooling data from 10 National Household Surveys on Drug Abuse (NHSDAs) conducted between 1979 and 1995 allowed birth cohorts for a 16-year period at minimal expense; some disadvantage were the target population and methodological differences.
RESULTS/CONCLUSIONS: The authors used a cross-sectional design to show that NHSDA estimates of alcohol and marijuana incidence were biased downward by response error, especially at ages 10 to 14. The downward bias increased with the retention interval, making stable trend lines look increasing when estimated using a single survey. In addition, there was evidence that suggests forward telescoping, as well as some intentional concealment among older respondents.
CITATION: Miller, P. V. (1997). Is “up” right? The National Household Survey on Drug Abuse [review]. Public Opinion Quarterly, 61(4), 627–641.
PURPOSE/OVERVIEW: For almost a decade, starting in the late 1980s, politicians quoted statistics from the National Household Survey on Drug Abuse (NHSDA) to claim that drug use among teenagers was rising. However, results from the 1996 NHSDA actually showed a decline in teenage drug use. This 1997 review discusses the NHSDA methodology to analyze the assumption that drug use was rising despite statistics to the contrary.
METHODS: A number of methodological factors, such as sampling, response rates, and data collection issues, are examined to assess their impact on prevalence estimates. The NHSDA sampling frame does not include individuals who are homeless (not living in shelters), nor does it include institutionalized individuals. There is no evidence on whether nonresponders to the NHSDA differ from responders in level of drug use. The NHSDA went through a number of methodological changes over the years aimed to improve survey quality and encourage reporting. These factors influence the likelihood of whether the NHSDA underreports drug use.
RESULTS/CONCLUSIONS: This examination of the NHSDA methodology suggests that the survey might have underestimated drug use. However, because no studies on the likelihood of overreporting had been conducted, it was impossible to say with conviction whether the estimates were an underreport. Validation studies do not answer all the questions surrounding data quality issues in the NHSDA; however, they do help to understand how different methods have an impact on survey estimates.
CITATION: Woodward, A., Epstein, J., Gfroerer, J., Melnick, D., Thoreson, R., & Willson, D. (1997). The drug abuse treatment gap: Recent estimates. Health Care Financing Review, 18(3), 5–17. [PMC free article: PMC4194502] [PubMed: 10173122]
PURPOSE/OVERVIEW: In the mid-1990s, it was acknowledged that the National Household Survey on Drug Abuse (NHSDA) underestimated the prevalence of drug use in the United States due to coverage error and social desirability bias. In this article, the authors attempted to correct for the underreporting by applying new estimation procedures.
METHODS: To control for sampling errors and underreporting due to social desirability in the NHSDA, three smaller data sources were used to assist with the ratio estimation. The three data sources were (1) the National Drug and Alcoholism Treatment Unit Survey (NDATUS), later known as the Uniform Facility Data Set (UFDS); (2) the Drug Services Research Survey (DSRS); and (3) the Uniform Crime Report (UCR). Ratio estimation relies on the assumption that estimates can be improved when a known value exists for a related variable. To assess the effects on the need for treatment for drug abuse, the term “need” had to be redefined.
RESULTS/CONCLUSIONS: The new definition for “need” included both a clinical and epidemiological perspective, making the definition more comprehensive and reliable for use in reporting estimates. The ratio estimation procedures improved the NHSDA estimates for the number of people in need of treatment because it corrected for coverage error and underreporting.
CITATION: Wright, D., Gfroerer, J., & Epstein, J. (1997). Ratio estimation of hardcore drug use. Journal of Official Statistics, 13(4), 401–416.
PURPOSE/OVERVIEW: In the mid-1990s, the prevalence of hard-core drug use was consistently underestimated in national surveys. The goal of this paper was to improve the estimates for hard-core drug use from the National Household Survey on Drug Abuse (NHSDA) by using additional sources and applying ratio estimation.
METHODS: Ratio estimation improves an estimate by comparing it with known totals for a related variable. To estimate hard-core drug use, the authors made use of two known populations related to drug use. First was the number of people in treatment centers for drug abuse, which was taken from the National Drug and Alcoholism Treatment Unit Survey (NDATUS). The second known population was the number of arrests during the past year taken from the Federal Bureau of Investigation’s Uniform Crime Reports (UCRs). To get estimates of hard-core drug users, the NHSDA ratio of drug users to people in treatment was multiplied by the known total population of people in treatment. The same procedure was applied to the number of arrests during the past year. These methods yielded two estimates for the number of hard-core drug users. Another step of ratio estimation was to use the two known populations together to create one single, more accurate estimate.
RESULTS/CONCLUSIONS: The standard errors for the ratio estimation were similar to the standard errors for the simple expansion estimator that was alternatively used. The ratio estimates were considered an improvement over other estimation techniques. However, they still did not account for underreporting in people who were not arrested or in treatment centers. An advantage to the estimates provided by ratio estimation is that it could provide more than just the “bottom line” and could yield estimates for such subgroups as race, region, gender, and income.
CITATION: Wright, D., Gfroerer, J., & Epstein, J. (1997). The use of external data sources and ratio estimation to improve estimates of hardcore drug use from the NHSDA. In L. Harrison & A. Hughes (Eds.), The validity of self-reported drug use: Improving the accuracy of survey estimates (NIH Publication No. 97-4147, NIDA Research Monograph 167, pp. 477–497). Rockville, MD: National Institute on Drug Abuse. [PubMed: 9243574]
PURPOSE/OVERVIEW: In the mid-1990s, levels of hard-core drug use were especially difficult to estimate because of the relative rarity of the behavior, the difficulty of locating hard-core drug users, and the tendency to underreport stigmatized behavior. This chapter presents a new application of ratio estimation, combining sample data from the National Household Survey on Drug Abuse (NHSDA) together with population counts of the number of individuals arrested in the past year from the Uniform Crime Report (UCR) and the number of those in drug treatment programs in the past year from the National Drug and Alcoholism Treatment Unit Survey (NDATUS). The population counts served as a benchmark accounting for undercoverage and underreporting of hard-core drug users.
METHODS: In this discussion, the focus was on the ratio estimate’s ability to reduce bias (in particular, the undercounting of hard-core drug users in the NHSDA) given a true population value of a related variable. To make the discussion more concrete, the estimation procedure was applied to four separate, but overlapping, measures of hard-core drug use for 1992: the number of past year users of heroin, weekly users of cocaine in the past year, past year users who were more dependent on some illicit drug, and past year intravenous drug users.
RESULTS/CONCLUSIONS: Overall statements about ratio estimation for the NHSDA include the following: (1) ratio estimation did not fully account for underreporting in the NHSDA; (2) because ratio estimation could be looked at as an adjustment to the NHSDA analytic weights (which were based on a probability sample design), it provided analytic capabilities that were not possible in other methods; (3) the ratio estimation model, as applied in this case, relied primarily on regularly updated and consistently collected data from the NHSDA, NDATUS, and UCR, and a relatively small number of easily understood assumptions; and (4) because ratio estimation relied primarily on the NHSDA sample design and weighting, it was possible to develop estimates of the variances of ratio-adjusted estimates.
All material appearing in this report is in the public domain and may be reproduced or copied without permission from SAMHSA. Citation of the source is appreciated. However, this publication may not be reproduced or distributed for a fee without the specific, written authorization of the Office of Communications, SAMHSA, HHS.
Your browsing activity is empty.
Activity recording is turned off.
See more...
