An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Substance Abuse and Mental Health Services Administration . National Survey on Drug Use and Health: Summary of Methodological Studies, 1971–2014 [Internet]. Rockville (MD): Substance Abuse and Mental Health Services Administration (US); 2014 Nov.
CITATION: Aldworth, J., Colpe, L. J., Gfroerer, J. C., Novak, S. P., Chromy, J. R., Barker, P. R., Barnett-Walker, K., Karg, R. S., Morton, K. B., & Spagnola, K. (2010). The National Survey on Drug Use and Health Mental Health Surveillance Study: Calibration analysis. International Journal of Methods in Psychiatric Research, 19(Suppl. 1), 61–87. doi:10.1002/mpr.312 [PMC free article: PMC7003703] [PubMed: 20527006] [CrossRef]
PURPOSE/OVERVIEW: The Mental Health Surveillance Study (MHSS) was an initiative by the Substance Abuse and Mental Health Services Administration to develop and implement methods for measuring the prevalence of serious mental illness (SMI) among U.S. adults aged 18 or older. The 2008 MHSS used data from clinical interviews that were administered to a subsample of respondents from the National Survey on Drug Use and Health (NSDUH) to accurately calibrate mental health screening scale data for estimating the prevalence of SMI in the full NSDUH sample.
METHODS: This research was based on the mental health scales that were included in the Kessler-6 (K6) screening scale of psychological distress (administered to all respondents) along with two additional measures of functional impairment (each administered to a random half sample of respondents): the World Health Organization Disability Assessment Schedule (WHODAS) and the Sheehan Disability Scale (SDS). The Structured Clinical Interview for DSM-IV (SCID) was administered to a subsample of 1,506 adult NSDUH respondents within 4 weeks of completing the NSDUH interview. Descriptive analyses were conducted to examine the distribution of respondent characteristics in the MHSS to check for imbalances between the two half samples, each of which was assigned to one of the impairment scales. Modeling analyses were conducted to develop algorithms based on the K6 scale and each of the impairment scales in turn, with the goal of identifying the best possible model for each impairment scale. This involved fitting a variety of models using alternative predictors, including different forms of the K6 and impairment variables. For each model, receiver operating characteristic (ROC) analyses were conducted to select the optimal cut point for determining SMI status. Weighted counts were used in the ROC classifications because primary interest is in estimating SMI status in the adult U.S. population. Models to determine SMI were compared and evaluated based on three criteria: (1) model robustness (e.g., preference given to parsimonious models that could be generalized to data beyond that used in the modeling process); (2) minimization of misclassification errors in SMI prediction (i.e., exhibiting reasonable ROC statistics, such as sensitivity and AUC, defined as the area under the ROC curve based on the optimal cut point described above); and (3) reasonable SMI estimates based on the full dataset (i.e., balanced across several demographic subgroups and across the WHODAS and the SDS half samples).
RESULTS/CONCLUSIONS: The authors found that the models with the WHODAS were more robust, while SMI prediction accuracy of the K6 was improved by adding either the WHODAS or the SDS to the prediction equation. The results of the calibration study and methods used to derive prevalence estimates of SMI were also presented.
CITATION: Chen, P., Sathe, N., Jones, M., Dai, L., Laufenberg, J., Folsom, R. E., & Gordek, H. (2010). Use of single years of age in the National Survey on Drug Use and Health (NSDUH) weighting to improve drug prevalence estimates. In Proceedings of the 2010 Joint Statistical Meetings, American Statistical Association, Survey Research Methods Section, Vancouver, British Columbia, Canada (pp. 3055–3066). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: In the National Survey on Drug Use and Health (NSDUH), several age groups (i.e., 12 to 17, 18 to 25, 26 to 34, 35 to 49, 50 to 64, and 65 or older) are currently used in the nonresponse (NR) and poststratification (PS) adjustments. Using 2004 to 2006 NSDUH data, the authors found that the response rate decreased as the age increased in the 12 to 25 age range. Additionally, prevalence rates for illicit drug, alcohol, and tobacco use increased almost linearly between the ages of 12 and 21, reaching a peak at age 21. Because the response rates and drug use prevalence rates changed dramatically between the ages of 12 and 21, use of single years of age instead of age groups for both the NR and PS adjustments should reduce NR bias and the variance of estimates. This paper explores the use of single years of age between 12 and 25 in the 2006 NSDUH weighting process and discusses the prevalence rates and standard errors produced using the new set of weights.
METHODS: The authors adopted the following approach for using single years of age as predictors in the person-level NR and PS adjustments for each of the nine census division-level model groups: (1) added single years of age in the main effect (12, …, 25) for both person-level NR and PS in place of the 12 to 17 and the 18 to 25 age groups; (2) kept all of the other variables in the models; (3) used age groups 12 to 17, 18 to 25, 26 to 34, 35 to 49, and 50 or older in the interactions for NR; and (4) used age groups 12 to 17, 18 to 25, 26 to 34, 35 to 49, 50 to 64, and 65 or older in the interactions for PS. After recalibrating the weights using single years of age as the main effects in the NR and PS adjustments, the authors checked the distribution of the recalibrated weights against the current analysis weights that used age groups as the main effects in the generalized exponential models. Then they examined the estimated numbers of past year users and the prevalence rates for a selected set of outcomes using the recalibrated weights (which used single year of age variables in the NR and PS adjustments) and compared them with the estimates based on the current weights (without the single year of age variables).
RESULTS/CONCLUSIONS: Adding single years of age (12 to 25) in the person-level NR and PS adjustments did not change the weight distribution dramatically. Differences between the estimated numbers and percentages of users for the younger age groups and single years of age (12 to 25) using the two sets of weights were observed, while the impact for other demographic domains was minimal. The effect on the estimated numbers of users (counts) was greater than on the estimated percentages for the 12 to 17 and the 18 to 25 age groups and for the single years of age. There was also evidence that the recalibration with single years of age (12 to 25) reduced errors (lower mean square error using the recalibrated weights) for individuals aged 12 to 25 in the NSDUH sample.
CITATION: Chromy, J. R., Feder, M., Gfroerer, J., Hirsch, E., Kennet, J., Morton, K. B., Piper, L., Riggsbee, B. H., Snodgrass, J. A., Virag, T. G., & Yu, F. (2010, February). Reliability of key measures in the National Survey on Drug Use and Health (Methodology Series M-8, HHS Publication No. SMA 09-4425). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.
PURPOSE/OVERVIEW: Information on data quality is an important output of major Federal surveys because survey data often are used to influence policy decisions. A range of types of error may occur in surveys related to problems with the respondent’s understanding of the questions or the effects of the interviewer. These problems may bring about a disparity between the survey response and a true value. Reinterviewing survey respondents in studies of survey response reliability provides a direct measure of such response variance. As a response to a 2006 directive by the Federal Government’s Office of Management and Budget, a reinterview study of respondents to the National Survey on Drug Use and Health (NSDUH) was conducted.
METHODS: The Reliability Study was embedded within the 2006 NSDUH main study. A subsample of the main study sample of 67,802 was selected such that data from the initial interview were used for both the main study and the Reliability Study. As for the main study, the respondent universe included the civilian, noninstitutionalized population aged 12 or older. In the Reliability Study, the respondent universe excluded residents of Alaska and Hawaii, residents of noninstitutional group quarters (e.g., shelters, rooming houses, dormitories), and individuals who did not speak English. To preserve the results and response rates of the main study, neither the field interviewers (FIs) nor respondents were informed ahead of time about the selection for the second interview. Recruitment scripts for the second interview were added to the end of the first interview and administered to the 3,516 eligible respondents selected for reinterview. Second interviews were obtained from 3,136 respondents, for an 85.6 percent reinterview weighted response rate. Although an incentive of $30 was offered for the first interview, $50 was offered for participation in the second interview. The second interviews were conducted 5 to 15 days following the initial interview with the same questionnaire as for the first interview. A set of follow-up questions was added to the end of the second interview about respondent use of tobacco, alcohol, and other drugs between the two interviews. A substudy was conducted within the Reliability Study—the same versus different interviewer substudy—to examine the potential impact that an FI might have on reliability.
RESULTS/CONCLUSIONS: Responses for substance use in the lifetime had almost perfect reliability, and responses for substance use in the past year showed substantial agreement. Age at first use of specific substances showed mostly moderate reliability, but findings for which a substance was used first were less consistent, with some being of only fair reliability. The reliability of responses to age at last use was generally fair. Comparisons of the consistency of responses among those who were interviewed by the same versus different FIs at the time of the two interviews showed no significant effect of the interviewer on the reliability of survey responses. The consistency of responses among those whose first and second interviews were fewer than 9 days apart was similar to the consistency of responses among those whose interviews were 9 or more days apart. Analyses showed that questions about factual personal events or characteristics were more reliable than questions that asked for a respondent’s personal opinion or intentions or questions that addressed issues that involved perceived discrimination (i.e., carried a social stigma).
CITATION: Colpe, L. J., Barker, P. R., Karg, R. S., Batts, K. R., Morton, K. B., Gfroerer, J. C., Stolzenberg, S. J., Cunningham, D. B., First, M. B., & Aldworth, J. (2010). The National Survey on Drug Use and Health Mental Health Surveillance Study: Calibration study design and field procedures. International Journal of Methods in Psychiatric Research, 19(Suppl. 1), 36–48. doi:10.1002/mpr.311 [PMC free article: PMC7003702] [PubMed: 20527004] [CrossRef]
PURPOSE/OVERVIEW: The Mental Health Surveillance Study (MHSS) was an initiative by the Substance Abuse and Mental Health Services Administration (SAMHSA) to monitor the prevalence of serious mental illness (SMI) among adults in the United States. In 2008, the MHSS used data from clinical interviews to calibrate mental health data from the National Survey on Drug Use and Health (NSDUH) for estimating the prevalence of SMI based on the full NSDUH sample. This paper describes the MHSS calibration study procedures, including information on sample selection, instrumentation, follow-up, data quality protocols, and management of distressed respondents.
METHODS: The clinical interview used in this study was the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV; SCID). NSDUH interviews were administered via audio computer-assisted self-interviewing (ACASI) to a nationally representative sample of the population aged 12 years or older. A total of 46,180 NSDUH interviews were completed with adults aged 18 years or older in 2008. The SCID was administered by mental health clinicians to a subsample of 1,506 adults via telephone. Clinical interviews were conducted by master’s and doctoral level mental health professionals who had been carefully and extensively trained to administer the semistructured clinical interview over the telephone. The study protocol included comprehensive instructions for identifying and managing distressed respondents and for ongoing supervision and interrater training exercises for the clinical interviewers.
RESULTS/CONCLUSIONS: Descriptive analyses of the demographic characteristics of the clinical interview sample indicated that the sample was balanced and consistent with the overall NSDUH sample. A 76 percent unweighted completion rate among those who agreed to the clinical interview was achieved by the study, a commendable rate given the shortness of the 4-week data collection period and the lack of in-person follow-up. Given the success in the execution of the 2008 study, SAMHSA decided to continue to include the K6 and WHODAS scales in the main NSDUH interview and to collect clinical interview data from a subset of NSDUH respondents to monitor the prevalence of SMI among adults in the United States. It was decided that this will allow additional analysis of SMI at the State level, as well as investigations into the prevalence and impact of milder forms of mental illness (e.g., with mild to moderate functional impairment). These continuing calibration activities and the ongoing nature of the study are significant contributions to mental health surveillance in the United States.
CITATION: Gfroerer, J., & Bose, J. (2010). The influence of prior experiences in managing current and future risks during survey transition points on the National Survey on Drug Use and Health (NSDUH). In Proceedings of the 2010 Joint Statistical Meetings, American Statistical Association, Section on Survey Research Methods, Vancouver, British Columbia (pp. 422–430). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: Since its inception in 1971, the National Survey on Drug Use and Health (NSDUH) has experienced many changes, including the transfer of the survey between Federal agencies; changes in government project officers and contractors; modifications to questionnaire content; major changes to the sample design and size; introduction of new modes of data collection and incentives to respondents; changes in the oversight and management of field staff; and introduction of new weighting, editing, and imputation methods. Some of these changes have, not surprisingly, resulted in both intended and unintended consequences, in some cases despite best efforts to control and quantify the effects of these changes. This paper uses examples to illustrate how prior experiences have influenced both ongoing practices and the current approach to redesigning the survey.
METHODS: N/A.
RESULTS/CONCLUSIONS: First, accurate trend measurement with an ongoing cross-sectional survey requires careful monitoring of data collection and estimation procedures to ensure comparability. Caution is needed when sampling errors are small. Small field tests cannot always be relied on to make decisions about design changes. Second, any major redesign of a large ongoing survey is probably going to result in a break in trends. Although it is probably possible to implement some improvements that have a low probability of disrupting the trend, and it may be feasible to implement a redesign under a split-sample design to account for and measure methods effects, there is no guarantee that this will be successful. Promises of trend continuation after a redesign are probably ill advised. The choice often comes down to maintaining the trend with a problematic design and biased estimates versus improving the survey (including possibly saving on costs) but breaking the trend.
CITATION: Hughes, A., Heller, D., & Marsden, M. E. (2010, May). The validity of self-reported tobacco and marijuana use, by race/ethnicity, gender, and age. In L. A. Aday & M. Cynamon (Eds.), Ninth Conference on Health Survey Research Methods: Conference proceedings (pp. 132–142). Hyattsville, MD: U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Center for Health Statistics.
PURPOSE/OVERVIEW: Researchers and policymakers have long been concerned about the validity of self-reported drug use and have provided recommendations for improvement, including the use of biological specimens to validate self-reports (U.S. General Accounting Office, 1993). A National Institutes of Health strategic plan for reducing health disparities in drug abuse and dependence recommended improving the validity of self-reported drug use because minority populations may be differentially affected (National Institute on Drug Abuse, 2004). The goal of this study was to examine the nature and extent of bias and discordance in self-reported estimates of tobacco and marijuana use by race/ethnicity, gender, and age compared with results from urinalysis tests in a nationally representative household survey.
METHODS: This paper used the data collected through the National Household Survey on Drug Abuse (NHSDA) Validity Study. The total number of respondents in the Validity Study was 4,465 over the 2-year data collection period (2000 and 2001), with a 74.3 percent weighted interview response rate. Of those completing the interview, 89.4 percent provided hair, urine, or both; of these, 80.5 percent provided both, 4.7 percent provided only urine, and 4.3 percent provided only hair. In addition to weighted prevalence rates of self-reported 3-day use and positive test results, statistics comparing self-reporting and drug testing also were calculated. These included estimates of discordance, underreporting and overreporting, bias, and correlation between discordance and bias. Two logistic regression models predicting past 3-day discordance of tobacco and marijuana were estimated. Models were fit using covariates including gender, age, race/ethnicity, passive exposure in the past 6 months to tobacco or marijuana, and socioeconomic status (SES).
RESULTS/CONCLUSIONS: All self-reported estimates of tobacco and marijuana use exhibited a downward bias, meaning that underreporting occurred more frequently than overreporting. Blacks and youths aged 12 to 14 had the largest bias compared with others in their respective demographic groups for both tobacco and marijuana use. Unadjusted odds ratios showed that older individuals and blacks were more likely than the youngest age group and whites, respectively, to report discrepant responses compared with urine test outcomes; moreover, females were less likely to misreport than males. However, after controlling on SES, privacy, truthfulness, friends’ use, and other theoretically relevant covariates, these relationships were no longer statistically significant.
CITATION: Iannacchione, V., Morton, K., McMichael, J., Shook-Sa, B., Ridenhour, J., Stolzenberg, S., Bergeron, D., Chromy, J., & Hughes, A. (2010, August). The best of both worlds: A sampling frame based on address-based sampling and field enumeration. Presented at the 2010 Joint Statistical Meetings, American Statistical Association, Section on Survey Research Methods, Vancouver, British Columbia.
PURPOSE/OVERVIEW: Cost savings are the primary advantage of using address-based sampling (ABS) over field enumeration (FE) for in-person surveys of the civilian, noninstitutionalized population. However, the values of cost savings decrease by research that indicates that FE provides more complete coverage than ABS, especially in rural areas. The authors developed and piloted a candidate sampling frame for the National Survey on Drug Use and Health (NSDUH) that uses ABS supplemented with a frame-linking procedure in area segments where they anticipate adequate ABS coverage and FE in segments where they anticipate poor ABS coverage. The objective of the candidate frame is to lower costs without sacrificing coverage levels of the current NSDUH sampling frame, which is based solely on FE.
METHODS: In the first Check for Housing Units Missed (CHUM1) procedure, field interviewers (FIs) start at an address that is contained on the ABS list (the starting dwelling unit [DU]) and follow a predetermined path of travel (usually moving to the left of the starting DU) until they either reach an address that is contained on the ABS list or they return to the starting DU. This allows coverage of DUs that are not on the ABS list but are in the same block as addresses on the ABS list. In the CHUM2 procedure, FIs perform the CHUM procedure from a predetermined start point in a randomly selected area rather than a starting DU. The FIs follow the same path of travel that they do for the CHUM1 procedure, stopping when they either list an address that matches to the ABS list or they return to the start point without finding a match. This procedure enables coverage of DUs that are in blocks where none of the DUs are on the ABS list.
RESULTS/CONCLUSIONS: The gains in coverage afforded by the CHUM procedure are essential to the veracity of the hybrid frame. Similar to the half-open interval (HOI) procedure, the CHUM procedure offers virtually complete coverage of DUs if implemented correctly in the field. The hybrid frame, utilizing a combination of FE, the ABS list, and the CHUM1 and CHUM2 procedures, theoretically provides 100 percent coverage of the target population. The authors report on the trade-offs between coverage and cost savings as area segments are shifted from FE to ABS.
CITATION: Kennet, J., Gfroerer, J., Barker, P., Piper, L., Hirsch, E., Granger, R., & Chromy, J. R. (2010, May). Reliability and data quality in the National Survey on Drug Use and Health. In L. A. Aday & M. Cynamon (Eds.), Ninth Conference on Health Survey Research Methods: Conference proceedings (pp. 107–123) Hyattsville, MD: U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Center for Health Statistics.
PURPOSE/OVERVIEW: High reliability is a necessary condition to guarantee data validity. If a question or set of questions proves unreliable, discussion of disparities in the construct purportedly being measured loses its grounding. This paper presents preliminary results from a reliability study carried out on the National Survey on Drug Use and Health (NSDUH). More than 3,100 respondents participated in the study, and this rich dataset is likely to yield many important findings in the future.
METHODS: The data used in this paper came from the reliability study data obtained in the second and third quarters of 2006, which included about 2,200 respondents. The reliability study sample was drawn from the NSDUH main sample. For practical reasons, respondents in Alaska and Hawaii were not included in the reliability study, nor were non-English speaking respondents. Recruitment scripts were added to the end of the first interview to the eligible respondents selected for reinterview. A $30 incentive was offered at the first interview, and $50 was offered at the second interview. A set of follow-up questions was also added to the end of the second interview about respondent use of tobacco, alcohol, and other drugs between the two interviews.
RESULTS/CONCLUSIONS: Youths appeared less consistent than young adults and older adults in their reporting of substance use, particularly in the cases of lifetime and past year nonmedical prescription drug use and past year alcohol use. Young adults and older adults had fewer kappas below those of the complement of the sample and, in a few cases, appeared more consistent than the other groups. Other patterns that appeared were related to race/ethnicity. White respondents appeared to be more consistent than others in reporting past year substance use. A general pattern also appeared to be present in the case of income. Greater income was generally associated with greater response consistency. This result would only be surprising if educational level did not exhibit the same general pattern of association, which it did. These findings point toward the importance of efforts to lower the reading level of the instrument and to improve the comprehensibility of the supporting materials that, among other topics, describe the survey, the uses of the data, and measures taken to enhance confidentiality.
CITATION: Kroutil, L. A., Vorburger, M., Aldworth, J., & Colliver, J. D. (2010). Estimated drug use based on direct questioning and open-ended questions: Responses in the 2006 National Survey on Drug Use and Health. International Journal of Methods in Psychiatric Research, 19(2), 74–87. doi:10.1002/mpr.302 [PMC free article: PMC6878274] [PubMed: 20222148] [CrossRef]
PURPOSE/OVERVIEW: Substance use surveys may use open-ended items to supplement questions about specific drugs and obtain more exhaustive information on illicit drug use. However, these questions are likely to underestimate the prevalence of use of specific drugs. Little is known about the extent of such underestimation or the groups most prone to underreporting.
METHODS: Using data from the 2006 National Survey on Drug Use and Health (NSDUH), a civilian, noninstitutionalized population survey of individuals aged 12 or older in the United States, the authors compared drug use estimates based on open-ended questions with estimates from a new set of direct questions that occurred later in the interview.
RESULTS/CONCLUSIONS: For these drugs, estimates of lifetime drug use based on open-ended questions often were at least 7 times lower than those based on direct questions. Among adults identified in direct questions as substance users, lower educational levels were consistently associated with nonreporting of use in the open-ended questions. Given NSDUH’s large annual sample size (approximately 67,000 interviews), combining data across future survey years could increase the understanding of characteristics associated with nonreporting of use in open-ended questions and allow drug use trends to be extrapolated to survey years in which only open-ended question data are available.
CITATION: LeBaron, P. A., Granger, R., Park, H., Heller, D., Dean, E., & Bettinger, G. (2010). Questionnaire design considerations when expanding a survey target population to include children. In Proceedings of the 2010 Joint Statistical Meetings, American Statistical Association, AAPOR, Chicago, IL (pp. 6259–6273). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: When designing a questionnaire, survey practitioners are challenged with developing questions that are appropriate for all respondents within the target population. This task becomes increasingly difficult when the same survey instrument will be administered to both children and adults. This paper examines factors to be considered when asking children to respond to the same survey items as adults.
METHODS: In order to evaluate the impact on questionnaire design and nonresponse of expanding a target population to include children under 12 years of age, the authors used data from the National Survey on Drug Use and Health (NSDUH). The authors examined differences in cognitive ability, nonresponse, and timing data between minors in the NSDUH sample. The differences in respondent behavior of the single years of age within the 12 to 17 year old age group provided insight into the effects of administering a survey to respondents under 12 years of age.
RESULTS/CONCLUSIONS: For expanding a survey target population to include children under 12 years of age, the authors suggested expanding the target population in such a way that would not require the development of new or significantly altered screening or interview instruments, such as only including children aged 10 or 11. This would potentially result in only minor adjustments to current data collection protocols while limiting the impact on data quality, instrumentation changes, and costs. Extensive usability testing and cognitive interviewing would need to be conducted to inform this decision and to ensure that children aged 10 or 11 could comprehend the questionnaire items and respond accurately to sensitive questions.
CITATION: McMichael, J. P., Ridenhour, J. L., Shook-Sa, B. E., Morton, K. B., & Iannacchione, V. G. (2010). Predicting the coverage of address-based sampling frames prior to sample selection. In Proceedings of the 2010 Joint Statistical Meetings, American Statistical Association, Section on Survey Research Methods, Vancouver, British Columbia (pp. 4852–4859). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: The current sampling frame for the National Survey on Drug Use and Health (NSDUH) relies on field enumeration (FE) supplemented with the half-open interval (HOI) procedure (Kish, 1965). Because of the costs associated with FE, the size of the area segments is small—usually, about 100 dwelling units (DUs) in a rural area and 150 DUs in an urban area. Several national in-person surveys are looking at address-based sampling (ABS) instead of or in conjunction with FE due to the lower costs associated with ABS. In 2009, RTI conducted a field study for NSDUH aimed at investigating the cost implications and coverage properties of a sampling frame based on ABS in area segments with adequate ABS coverage and FE elsewhere. The objective of the NSDUH field study was to develop and test an ABS/FE hybrid frame that provides cost savings without sacrificing coverage.
METHODS: The field study was implemented by subsampling 200 NSDUH segments from the 2009 quarter 1 sample. The sample had 3,878 screened and eligible sampled DUs in a subsample of 200 NSDUH segments. Segments in Alaska and Hawaii were excluded from the field study sampling frame. To develop a hybrid frame of DUs, the authors attempted to match the addresses of eligible sampled DUs obtained from NSDUH’s FE process and updated during the screening to a list of mailing addresses purchased from a commercial vendor. The authors classified the segments by coverage threshold to theoretically evaluate how these segments would be allocated to FE or ABS under the hybrid frame.
RESULTS/CONCLUSIONS: Under a hybrid frame, correctly predicting whether segments should utilize ABS or FE is essential to retaining desired coverage properties (e.g., minimizing coverage bias) while achieving cost savings. Cost savings are achieved by appropriately allocating segments that have sufficient ABS coverage to ABS. Coverage is maintained by allocating segments where ABS coverage is low to FE. Allocating correctly, in both cases, reduces costs.
CITATION: Morton, K., McMichael, J. P., Ridenhour, J. L., & Bose, J. (2010). Address-based sampling and the National Survey on Drug Use and Health: Evaluating the effects of coverage bias. In Proceedings of the 2010 Joint Statistical Meetings, American Statistical Association, Section on Survey Research Methods, Vancouver, British Columbia (pp. 4902–4907). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: A common concern of survey researchers is whether the coverage properties of an address-based sampling (ABS) frame create outcome bias in the estimates from in-person surveys. This paper evaluates basic demographics and several drug use and mental health measures obtained from 1,725 respondents in a probability sample of 200 area segments from the National Survey on Drug Use and Health (NSDUH).
METHODS: The evaluation compared outcomes from respondents covered by the NSDUH’s field enumeration (FE) frame with those covered by an ABS frame derived from the United States Postal Service Computerized Delivery Sequence (USPS CDS) file. After poststratifying the weights to populations with known ABS undercoverage, the authors tested for significant differences in outcomes between the two frames.
RESULTS/CONCLUSIONS: Estimates based on the ABS-only frame were limited to small, but statistically significant differences when compared with the FE frame. Use of the Check for Housing Units Missed (CHUM) frame-linking procedure to supplement the ABS frame helped to mitigate some of these differences. It is notable that these comparisons have the statistical power to declare very small differences in the overall prevalence estimates statistically significant because the estimates based on the FE frame and the estimates based on the ABS frame share a large portion of their cases. For example, for a prevalence estimate of 0.01 percent, a difference of 0.002 percent can be detected with 80 percent power and a significance level of 0.10 assuming an ABS coverage rate of 95 percent. Hybrid frames (such as the one investigated for NSDUH) would share an even larger proportion of cases with the FE frame because segments below the designated threshold would be field enumerated. Therefore, the hybrid FE and ABS frame would have even less coverage bias. For the hybrid frame investigated for NSDUH, the authors examined coverage bias among several subgroups, including rural areas and group quarters, and found no substantive differences in the estimates.
CITATION: Novak, S. P., Colpe, L. J., Barker, P. R., & Gfroerer, J. C. (2010). Development of a brief mental health impairment scale using a nationally representative sample in the USA. International Journal of Methods in Psychiatric Research, 19 (Suppl. 1), 49–60. doi:10.1002/mpr.313 [PMC free article: PMC7003704] [PubMed: 20527005] [CrossRef]
PURPOSE/OVERVIEW: A psychometric analysis was conducted to reduce the number of items needed to assess the disability associated with mental disorders using the World Health Organization Disability Assessment Schedule (WHODAS). The WHODAS was to be used in the Substance Abuse and Mental Health Services Administration’s National Survey on Drug Use and Health (NSDUH), beginning in 2008, as part of a screening algorithm to produce estimates of the prevalence of serious mental illness (SMI) in the U.S. adult population. The goal of this paper was to create a parsimonious screening scale from the full 16-item WHODAS that was administered to 24,156 respondents aged 18 or older in the 2002 NSDUH.
METHODS: The authors used the 2002 NSDUH containing multiple measures of psychiatric symptoms, impairments, and mental health service use to include the following: (a) a series of disorder-specific items that had been determined to predict with certain probability meeting 12-month criteria for Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV), disorders from the World Health Organization (WHO) Composite International Diagnostic Interview Short Form (CIDI-SF) scales; (b) the Kessler-6/Kessler-10 (K6/K10) scales of nonspecific psychological distress; and (c) NSDUH’s adult mental health service utilization module. These measures were used to identify and exclude members of the study sample who would likely not meet the criteria for a DSM disorder because associated mental health impairments would not apply to those respondents. The authors restricted the analyses to those respondents with the following characteristics: (1) endorsing a disorder-specific item from the CIDI-SF screener, (2) endorsing “most” or “some of the time” for any one of the nonspecific distress items on the K6/K10 scale, (3) seeing a mental health professional in the past 12 months, (4) staying in a hospital or facility overnight for mental health issues in the past 12 months, (5) receiving outpatient treatment for mental health issues in the past 12 months, and (6) taking prescription medication for a mental condition in the past 12 months.
RESULTS/CONCLUSIONS: Exploratory factor analyses showed that WHODAS responses were unidimensional. A two-parameter polytomous Item Response Theory model showed that all of the 16 WHODAS items had good item discrimination (slopes greater than 1.0) for each response option. Analysis of item difficulties and differential item function across sociodemographic categories was then used to select a subset of 8 items to create a short version of the WHODAS. The Pearson correlation between scores in the original 16-item and reduced 8-item WHODAS scales was 0.97, documenting that the vast majority of variation in total scale scores was retained in the reduced scale.
CITATION: Paschall, M. J., Ringwalt, C. L., & Gitelman, A. M. (2010). The validity of State survey estimates of binge drinking. American Journal of Preventive Medicine, 39(2), 179–183. doi:10.1016/j.amepre.2010.03.018 [PMC free article: PMC2904643] [PubMed: 20621266] [CrossRef]
PURPOSE/OVERVIEW: State survey-based estimates of binge drinking are useful to estimate the need for alcohol prevention and treatment services and to evaluate the effects of State alcohol control policies. However, because of declining survey response rates, there is growing concern about the validity of State survey estimates of binge drinking. This study examines the construct validity of State survey-based prevalence estimates of binge drinking.
METHODS: The authors used the State prevalence estimates of binge drinking in the past 30 days for 1999, 2001, 2003, 2005, and 2007 from published reports or public use data for the National Survey on Drug Use and Health, the Behavioral Risk Factor Surveillance System Survey, and the Youth Risk Behavior Survey. Construct validity was assessed in 2009 by examining correlations between these survey estimates and State per capita alcohol consumption levels (based on sales data for beer, wine, and spirits) and the percentage of drivers with a blood alcohol concentration of at least 0.08 who were in fatal motor vehicle crashes.
RESULTS/CONCLUSIONS: An estimated 88 percent of the correlations between State survey-based binge drinking estimates and per capita alcohol sales data were significant and moderate to strong (r ≥ 0.30, range = 0.16 – 0.60). Similarly, 86 percent of the State survey binge drinking estimates were moderately or strongly correlated with the percentage of drivers in fatal crashes who had a blood alcohol concentration that was greater than or equal to 0.08 (range = 0.11 – 0.60). Based on the results, the authors concluded that State survey-based estimates of binge drinking have construct validity and therefore can be used to investigate relationships between State alcohol policies and other State characteristics and the prevalence of this behavior.
CITATION: Shook-Sa, B. E., McMichael, J. P., Ridenhour, J. L., & Iannacchione, V. G. (2010). The implications of geocoding error on address-based sampling. In Proceedings of the 2010 Joint Statistical Meetings, American Statistical Association, Survey Research Methods Section, Vancouver, British Columbia, Canada (pp. 3303–3312). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: In-person surveys that use address-based sampling (ABS) are often based on area segments defined by census geography rather than postal geography. Census geography enables more accurate inclusion of demographic information in the sample selection procedures and the use of frame supplementation methods to increase coverage. However, area frames based on census geography contain more frame error than frames based on postal geography because addresses must be allocated (i.e., geocoded) into area segments. When addresses are incorrectly geocoded into area segments, sampling inefficiencies occur.
METHODS: The authors examined data from the 2009 National Survey on Drug Use and Health to determine the extent of geocoding error in sampled segments and its implications on coverage and efficiency of area frame samples.
RESULTS/CONCLUSIONS: Geocoding accuracy at the segment level was quite poor and varied significantly by urbanicity. Rural segments had a much higher rate of undercoverage geocoding error (23.4 percent) compared with urban segments (7.5 percent). Geocoding accuracy improved significantly at the census block group level for both rural and urban segments, with 99.3 percent of addresses geocoding into the correct census block group (99.8 percent urban, 96.5 percent rural). These findings should be considered when designing ABS studies that are based on census geography. Geocoding error can be a significant source of undercoverage and sampling inefficiencies if segments are smaller than census block groups, especially in rural areas. Several characteristics of addresses and segments are related to segment-level geocoding error. Segment-level geocoding is more accurate in urban areas than in rural areas. Geocoding error also varies by census division, postal route and delivery type, the area of the segment, the proportion of new homes, and median home values within the segment.
All material appearing in this report is in the public domain and may be reproduced or copied without permission from SAMHSA. Citation of the source is appreciated. However, this publication may not be reproduced or distributed for a fee without the specific, written authorization of the Office of Communications, SAMHSA, HHS.
Your browsing activity is empty.
Activity recording is turned off.
See more...
