An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Substance Abuse and Mental Health Services Administration . National Survey on Drug Use and Health: Summary of Methodological Studies, 1971–2014 [Internet]. Rockville (MD): Substance Abuse and Mental Health Services Administration (US); 2014 Nov.
CITATION: Barker, P., Gfroerer, J., Caspar, R., & Lessler, J. (1998). Major design changes in the National Household Survey on Drug Abuse. In Proceedings of the 1998 Joint Statistical Meetings, American Statistical Association, Survey Research Methods Section, Dallas, TX (pp. 732–737). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: This paper reports on interim results of methodological research carried out to test design changes eventually introduced in the 1999 National Household Survey on Drug Abuse (NHSDA). The main focus is on a feasibility test conducted in 1996 that compared administering the survey through paper-and-pencil interviewing (PAPI) and computer-assisted interviewing (CAI). The paper also briefly describes laboratory testing of 12-month frequency of use questions and plans to increase the NHSDA sample sizes in 1999 to produce State-level estimates.
METHODS: In the 1996 CAI Feasibility Experiment, 400 interviews were carried out using either PAPI or CAI. For the CAI version, core items on substance use were administered using audio computer-assisted self-interviewing (ACASI). The CAI version consisted of two versions, one that incorporated skip patterns into the survey instrument and another that did not and thus, “mirrored” the PAPI version. Interviewer observation questions also were asked.
RESULTS/CONCLUSIONS: The main result of the 1996 CAI Feasibility Experiment was that a CAI approach to gathering NHSDA data was quite feasible and had a number of benefits to offer over the PAPI approach, including increased reporting of past year and past month marijuana and cocaine use and increased perceptions of privacy. It also was found that the skip pattern version of CAI was shorter than the CAI version that “mirrored” PAPI by about 10 minutes. The cognitive laboratory testing of the 12-month frequency of use items revealed that these questions were difficult to answer due to a long recall period and that the response categories combined total number of days with periodicity estimates. An alternative response format was developed that allowed the respondent to select the unit for reporting days of use.
CITATION: Brittingham, A., Tourangeau, R., & Kay, W. (1998). Reports of smoking in a national survey: Data from screening and detailed interviews, and from self- and interviewer-administered questions. Reports About Smoking: Annals of Epidemiology, 8(6), 393–401. [PubMed: 9708875]
PURPOSE/OVERVIEW: Using data from the 1994 National Household Survey on Drug Abuse (NHSDA), the authors tested the primary hypothesis that using self-administered questionnaires versus interviewer-administered questionnaires would increase reports of cigarette smoking among adolescents. The study also compared respondent and proxy responses during a brief screening interview with the responses collected during a more detailed interview later.
METHODS: Approximately 22,000 respondents aged 12 or older were interviewed using a national area probability sample. During the screening interview, basic demographic information and current smoking status were collected for all household members. Using this information, one or more respondents from the household were selected to complete the main interview. Sometimes the respondent who completed the screening interview also was selected for the main interview. Sample members were randomly assigned to receive either interviewer-administered smoking questions or self-administered smoking questions. Logistic regression was used to compare the rates of smoking by interview mode and differences in reporting of smoking between the screening and the main interview.
RESULTS/CONCLUSIONS: Although the self-administered questions showed higher reports of smoking than interviewer-administered questions, the differences were not significant for the overall population. When the analyses were restricted to adolescents, there was a marginally significant main effect for mode of interview, with self-administered questions yielding higher reporting. The proportion of current smokers reported in the screening interview was significantly lower than in the main interview. Discrepancies between the screening interview and the main interview occurred 4 times more often when the screening respondent was different from the interview respondent. This supports the notion that the screening respondent was a proxy and was not necessarily cognizant of the smoking status of the rest of the household. When the same respondent was interviewed in the screener and in the main interview, the highest rate of discrepancy occurred in the adolescent age group, supporting the hypothesis that smoking is sensitive for adolescents but not adults. In addition, asking multiple questions about smoking leads to higher reports of smoking, particularly in respondents who may be reluctant to admit their smoking status.
CITATION: Caspar, R. A., Lessler, J. T., & Penne, M. A. (1998). Testing ACASI procedures to reduce inconsistencies in respondent reports in the NHSDA: Results from the 1997 experimental field test. In Proceedings of the 1998 Joint Statistical Meetings, American Statistical Association, Survey Research Methods Section, Dallas, TX (pp. 750–755). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: One of the most significant potential benefits of converting the National Household Survey on Drug Abuse (NHSDA) to a computer-assisted format is the opportunity to resolve inconsistent data at the time of the interview rather than editing the data to deal with inconsistencies after the fact. However, maintaining the privacy benefits of the audio computer-assisted self-interviewing (ACASI) component of the interview requires that the respondent be able to resolve inconsistencies for many items on his or her own. Thus, one of the goals of the NHSDA conversion work was to develop a method for resolving inconsistent data that the respondent could easily understand and complete without significant intervention by the interviewer.
METHODS: Based on the authors’ own hypotheses about how inconsistencies should be identified and resolved, they developed a resolution methodology that combined two components. First, at the verify stage, respondents are asked whether an answer they have entered is in fact correct. So, for example, when a 20-year-old respondent reports that she was 51 the first time she drank alcohol (a clearly inconsistent answer), the computer was programmed to verify that this information was correct. If the respondent indicated that the information was incorrect, she was routed back to answer the question again (perhaps this time entering the age of her first drink as 15). A second component incorporates the resolution of seemingly inconsistent answers. For example, a respondent who indicated drinking alcohol on 15 days in the past 12 months, but then reported drinking alcohol on 25 days in the past 30 days, first would be asked to verify the last entry keyed. If the respondent indicated that the entry was correct, then he or she was routed to a question that identified the inconsistency and provided the respondent with an opportunity to correct one or both of the entries.
RESULTS/CONCLUSIONS: Based on the results reported, the authors feel that the inconsistency resolution methodology employed in the 1997 CAI Field Experiment was successful. The methodology improved the consistency of the data collected without adversely affecting respondent cooperation or burden. Using this methodology in future implementations of the NHSDA will allow the Substance Abuse and Mental Health Services Administration (SAMHSA) to capitalize on the numerous benefits of the ACASI technology while minimizing one of the potential pitfalls—that respondent errors and inconsistencies are not identified and corrected at the time of interview. As of 1998, work is under way to incorporate inconsistency resolution screens into the 1999 computer-assisted personal interviewing (CAPI) and ACASI NHSDA instrument. The authors anticipate that future research in this area will be conducted to determine whether respondents can resolve inconsistencies between items that are not closely spaced in the interview. If this proves to be possible, the authors feel that future NHSDA instruments may incorporate inconsistency resolution screens of this type as well.
CITATION: Epstein, J. F., & Gfroerer, J. C. (1998, May). Estimating substance abuse treatment need from a national household survey. In J. Epstein (Ed.), Analyses of substance abuse and treatment need issues (HHS Publication No. SMA 98-3227, Analytic Series A-7, pp. 113–125). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.
PURPOSE/OVERVIEW: For purposes of planning future demand on the health care system in general and the substance abuse treatment system in particular, it is important to be able to develop estimates of the number of people needing treatment for substance abuse (i.e., treatment need) on a regular basis. In order to do this, the measurement of treatment need must distinguish low-intensity substance use from drug use that requires intervention. The overall prevalence of substance use is a poor absolute measure of problem substance use. Estimating treatment need is a difficult problem. Drug and alcohol consumption patterns and their consequences are complicated and dynamic. The modalities and philosophies of treatment are diverse. The applicability of even well-ensconced and tested diagnostic criteria must be reestablished as new drugs and ways of administering them appear. Measuring treatment need involves both a scientific and clinical understanding of the substance use problem. Estimating how many people in the general population need treatment is a different problem from diagnosing the need for treatment for an individual based on history taking, physical examination, and information in previous records. In most household surveys, it would be impractical to perform physical examinations or take a detailed history. In addition, household surveys may not fully cover key populations that may have a significant number of people needing treatment, such as incarcerated and homeless individuals. The main focus of this paper describes attempts in the 1990s to estimate treatment need based on the National Household Survey on Drug Abuse (NHSDA). The NHSDA is potentially a valuable source for estimating treatment need because it is the only ongoing large national survey of substance use covering most of the population. Thus, if a reasonable method for estimating treatment need from the NHSDA could be developed, it would provide more timely estimates than other sources and could be used to measure changes in treatment need over time.
METHODS: To estimate the effects of changes in the questionnaire and variable construction, both the new and old versions of the survey were fielded in 1994. The “old” version of the survey, which was actually the same as the 1993 questionnaire, is referred to as 1994-A, while the “new” version of the survey is referred to as 1994-B.
RESULTS/CONCLUSIONS: Comparisons between the new questionnaire (1994-B) and the old questionnaire (1994-A) showed that estimates of the rate of lifetime illicit drug use from the new questionnaire were approximately 8 percent less than estimates from the old questionnaire. Estimates of the rate of past year illicit drug use were approximately 12 percent less with the new questionnaire. On the other hand, estimates of the rate of past month illicit drug use were approximately 5 percent higher with the new questionnaire. The new questionnaire produced approximately 29 percent higher estimates of the rate of weekly cocaine use and approximately 6 percent higher estimates of the rate of daily marijuana use.
CITATION: Groves, R. M., & Couper, M. P. (1998). Nonresponse in household interview surveys. New York City, NY: Wiley.
PURPOSE/OVERVIEW: The 1990 National Household Survey on Drug Abuse (NHSDA) was one of six large Federal or federally sponsored surveys used in the compilation of a dataset that then was matched to the 1990 decennial census for analyzing the correlates of nonresponse. In addition, data from surveys of NHSDA interviewers were combined with those from these other surveys to examine the effects of interviewer characteristics on nonresponse.
METHODS: Information from the NHSDA, such as interviewer notes, segment listing sheets, and segment maps, as well as other data sources were used to match sampled housing units from the 1990 NHSDA with housing units from the 1990 decennial census. Variables on household characteristics from the 1990 census and area characteristics were used as predictors of cooperation on the NHSDA and the other surveys in the dataset. For characteristics of interviewers, a survey was administered to NHSDA interviewers during training. Among the items asked about were (1) education; (2) primary reason for their interviewing job and the existence of another paid job; (3) experience, including years worked and number of organizations worked for in the last 5 years; and (4) questions regarding interviewer attitudes and expectations. These variables were used as predictors of cooperation on the NHSDA as well as the other surveys.
RESULTS/CONCLUSIONS: Overall, 97.2 percent of NHSDA sample housing units (a total of 4,619 units) were successfully matched to the decennial census, with 97.8 percent of interviewed housing units (n = 1,407) and 96.7 percent of nonresponse housing units (n = 2,681) successfully matched. A total of 280 NHSDA interviewers completed the survey of interviewer characteristics. The main findings were as follows: (1) Once contacted, those with lower socioeconomic status were no less likely to cooperate than those with higher socioeconomic status; there was instead a tendency for those in high-cost housing to refuse survey requests, which was partially accounted for by residence in high-density urban areas. (2) The tendency for individuals in military service and in non-English-speaking households to cooperate appeared to be a function of household composition. (3) Households with children or young adults were more likely to cooperate; single-adult households were less likely to cooperate. (4) After controlling for the effects of household size, households with elderly residents tended to cooperate. (5) The tendency of racial and ethnic minorities to participate was accounted for by lower socioeconomic status. (6) Densely populated, high-crime urban areas had lower cooperation rates, which could be accounted for by other ecological variables, as well as different household compositions in urban versus rural areas. (7) Interviewers with more interviewing experience tended to achieve higher cooperation rates than those with less experience, but it was not clear if this was due to higher attrition among less successful interviewers or additional skills gained by experienced interviewers. (8) There was some evidence that interviewers with higher levels of confidence in their ability to gain participation achieved higher cooperation rates.
CITATION: Johnson, R. A., Gerstein, D. R., & Rasinski, K. A. (1998). Adjusting survey estimates for response bias: An application to trends in alcohol and marijuana use. Public Opinion Quarterly, 62(3), 354–377.
PURPOSE/OVERVIEW: Lifetime prevalence estimates and trends in drug use are primarily based on self-reports. Validity testing usually compares the reports given by special populations with administrative records and biochemical tests. However, these types of validity measures are not possible in a general population study; therefore, most studies rely on test-retest procedures.
METHODS: Instead of the traditional reinterview method for assessing validity, this study used a repeated cross-sectional design. This method utilized birth cohorts and examined changes in the distribution of their responses over individual cross-sectional surveys. Using data from 9 years of the National Household Survey on Drug Abuse (NHSDA) from 1982 to 1995, trends in birth cohorts were evaluated over 13 years. Changes in survey design in the NHSDA over the 13 years may have introduced bias in assessing response error.
RESULTS/CONCLUSIONS: Following birth cohorts using a repeated cross-sectional design is cost-effective in assessing self-report error in studies of drug and alcohol use. Using an exponential decay model of response bias revealed lower reporting of drug use as the time between the drug use and survey response increased. This bias distorts trends in alcohol and drug use, making stable drug use trends misleadingly appear increasing. Further analysis revealed that respondents tended to forward telescope the age for first alcohol use. In addition, intentional nondisclosure of marijuana use at a young age increased as the respondent increases in age, which led to underreporting of lifetime prevalence rates for marijuana use.
CITATION: Lessler, J. T., Witt, M., & Caspar, R. (1998). Development of computer assisted interviewing procedures for the National Household Survey on Drug Abuse (NHSDA): Design and operation of the 1997 CAI field experiment. In Proceedings of the 1998 Joint Statistical Meetings, American Statistical Association, Survey Research Methods Section, Dallas, TX (pp. 738–743). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: A large-scale field experiment that examined the use of computer-assisted interviewing (CAI) for the National Household Survey on Drug Abuse (NHSDA) was conducted in the last quarter of 1997. The design of the 1997 field experiment was based on the results of a 1996 CAI feasibility experiment and subsequent cognitive laboratory testing, power calculations, and discussions as to the operational feasibility of various designs. The authors compared alternative versions of the audio computer-assisted self-interviewing (ACASI) portion of the CAI interview in a factorial design.
METHODS: The authors compared these alternatives with each other and with the results from the methodology employing a combination of a paper-and-pencil interviewing (PAPI) and self-administered answer sheets. They conducted the experiment in the fourth quarter of 1997 and used the Quarter 4 1997 NHSDA survey results as a comparison group.
RESULTS/CONCLUSIONS: The screening and interview response rates in the 1997 experimental field test were lower than those achieved in the main study. The overall screening response rates was 86.8 percent, which was about 7 percent lower than that achieved in similar areas in the national NHSDA. About 2.5 percent of this shortfall was due to the failure to obtain access to restricted housing; 3.5 percent was due to increased refusals. It is unlikely that the electronic screener contributed to the failure to obtain access to restricted housing. However, the authors were not able, from this study alone, to verify that using the Newton screener did not contribute to increased refusals to the screening.
CITATION: Miller, P. V. (1998). Discussion of issues facing the NHSDA. In Proceedings of the 1998 Joint Statistical Meetings, American Statistical Association, Survey Research Methods Section, Dallas, TX (p. 762). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: The papers in this session report on the results of a field test that compared paper-and-pencil questionnaires with several versions of a computer-assisted interviewing (CAI) measurement of drug use. The study discussed is one of many studies conducted to assess the effects of converting the National Household Survey on Drug Abuse (NHSDA) from paper-and-pencil interviewing (PAPI) to CAI.
METHODS: N/A.
RESULTS/CONCLUSIONS: The results support a priori views that CAI has the potential to improve data quality in NHSDA. Findings include indications that a streamlined CAI version of the questionnaire can produce higher reporting of drug use and that inconsistency checking built into the CAI instrument can improve reporting. The authors also noted the success of ACASI technology in providing respondent privacy, handling complex skip patterns, and resolving inconsistencies between responses.
CITATION: Penne, M. A., Lessler, J. T., Bieler, G., & Caspar, R. (1998). Effects of experimental audio computer-assisted self-interviewing (ACASI) procedures on reported drug use in the NHSDA: Results from the 1997 CAI field experiment. In Proceedings of the 1998 Joint Statistical Meetings, American Statistical Association, Social Statistics Section, Dallas, TX (pp. 744–749). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: The 1997 computer-assisted interviewing (CAI) field experiment evaluated the impact on reported drug use of using alternative versions of an audio computer-assisted self-interviewing (ACASI) version of the National Household Survey on Drug Abuse (NHSDA).
METHODS: Alternative versions of the ACASI questionnaire were examined using a factorial design conducted during the fourth quarter of 1997. A subsample of the Quarter 4 national NHSDA, which used a combination of a paper-and-pencil interviewing (PAPI) version and self-administered questionnaire (SAQ) answer sheets, comprised the control group for the study. A fuller description of the design can be found in Lessler, Witt, and Caspar (1998) in this volume. In this paper, the authors examined reported drug use and compared the experimental ACASI factors to each other and to the control group. They also presented information on overall differences between ACASI and the control group.
RESULTS/CONCLUSIONS: The prevalence data indicate that ACASI will yield higher estimates of drug use, particularly for youths. The debriefing data indicate that this is due to the privacy-enhancing features of the method.
CITATION: U.S. General Accounting Office. (1998). Drug abuse treatment: Data limitations affect the accuracy of national and state estimates of need (GAO/HEHS-98-229, Report to Congressional Requesters, Health, Education, and Human Service Division). Washington, DC: U.S. Government Printing Office.
PURPOSE/OVERVIEW: The Federal Government annually provides approximately $3 billion for drug abuse prevention and treatment activities. However, determining the need for treatment services for the general population, as well as for specific subpopulations, may be problematic due to limitations in national and State data on treatment need. This report describes the Substance Abuse and Mental Health Services Administration’s (SAMHSA) efforts to estimate drug abuse treatment need on a national basis and obtain State estimates of drug abuse treatment need.
METHODS: The authors interviewed and obtained documents from officials in SAMHSA’s Center for Substance Abuse Treatment (CSAT), the Office of Applied Studies (OAS), and the Office of the Administrator. The authors also held discussions with experts in the substance abuse research community. They reviewed needs assessment information submitted by States. In addition, the authors attended a CSAT-sponsored workshop that included all States with current State Treatment Needs Assessment Program (STNAP) contracts in which States reported on their needs assessment studies.
RESULTS/CONCLUSIONS: Reliable assessments of treatment need are an essential component to accurately target treatment services. Although SAMHSA is improving its national estimates through the expansion of the National Household Survey on Drug Abuse (NHSDA) sample, the survey is still likely to underestimate treatment need. Also, STNAP’s goals to help States develop estimates of treatment need and improve State reporting of need data have not been fully accomplished. Even though States are required to provide estimates of treatment need as part of their block grant applications, not all States report this information, and some of the data reported are inaccurate. SAMHSA recognizes the need to increase State reporting and has set a target for increasing the number of States that provide the information. It also recognizes that the overall quality of the data reported is problematic.
CITATION: Wright, D. L., Aquilino, W. S., & Supple, A. J. (1998). A comparison of computer-assisted and paper-and-pencil self-administered questionnaires in a survey on smoking, alcohol, and drug use. Public Opinion Quarterly, 62(3), 331–353.
PURPOSE/OVERVIEW: This article assesses the impact of using computer-assisted self-interviewing (CASI) to collect self-report data on sensitive questions about drugs, alcohol, and mental health. The study also measures the interaction that computerization and respondent characteristics have on the survey responses.
METHODS: Using a national multistage area probability sample, 3,169 interviews were conducted with respondents aged 12 to 34. Approximately two thirds completed the computer self-administered questionnaire (SAQ), and the other one third completed a paper-and-pencil interviewing (PAPI) SAQ. The interviews used the drug and mental distress questions from the 1995 National Household Survey on Drug Abuse (NHSDA) questionnaire. Ordinary least squares regression was used to measure the effect of mode on prevalence estimates.
RESULTS/CONCLUSIONS: For all variables except for tobacco use, the computer SAQ produced higher estimates than the paper SAQ for adolescents. The greatest effects were seen on the most sensitive items. This supports the notion that adolescents are more familiar with and have a better understanding of computers than adults, which makes them more comfortable using the computer SAQ. There was a significant mode by mistrust interaction in older respondents for alcohol and drug use. Respondents who were mistrustful had higher estimates of drug and alcohol use in the PAPI SAQ than in the CASI SAQ.
CITATION: Wright, D., & Zhang, Z. (1998). Hierarchical models applied to the National Household Survey on Drug Abuse (NHSDA). In Proceedings of the 1998 Joint Statistical Meetings, American Statistical Association, Survey Methods Section, Dallas, TX (pp. 756–761). Alexandria, VA: American Statistical Association.
PURPOSE/OVERVIEW: A fairly comprehensive discussion of the methodology in hierarchical modeling was introduced in Hierarchical Linear Models: Applications and Data Analysis Methods (1992) by Anthony S. Bryk and Stephen W. Raudenbush. A significant part of that book focused on hierarchical structures in the field of education and the necessity of using an estimation methodology that reflects the structure of the data. A natural question is how that methodology might apply to the drug use field. Much of the drug use analysis research over the years has utilized simple logistic regression to estimate relationships between drug use and a variety of person-level variables. The authors wanted to explore how hierarchical models could be applied to the National Household Survey on Drug Abuse (NHSDA) data and what the consequences would be of ignoring the hierarchy. Since its inception, the NHSDA has been a multiphase stratified sample of primary sampling units (counties or groups of counties), segments (blocks or block groups), households, and individuals. The result of this complex nested design is that using traditional techniques of variance estimation that ignore the clustering of sample cases and the use of sample weights tends to overstate the significance of many findings. A number of statistical packages have been developed using Taylor series methods or replication methods, such as SUDAAN® (RTI) and WESVAR® (WESTAT), that take this structure into account. There was no widely used software for hierarchical mixed models, however, until hierarchical linear modeling (HLM).
METHODS: This paper explored some of the special circumstances in applying hierarchical models to NHSDA. The authors decided to use the data from six oversampled cities from the 1991 to 1993 NHSDAs. With a sample of approximately 2,500 individuals per city for each of the 3 years, there was a total sample of about 45,000 people. Each of the six cities (Washington, DC; Chicago; Miami; New York City; Denver; and Los Angeles) was selected with certainty, and within each city, segments were sampled at the first stage. Within segment, usually about 10 to 12 individuals aged 12 or older were selected. If the authors wanted to make inference to the collection of the six large cities as typical large cities, then they could consider four levels of hierarchy: city, segment, family, and person. Two issues arose immediately: Were there sufficient number of observations at each hierarchical level to support the analysis, and was it necessary to include all four levels of hierarchy in the estimation? By the latter, it is meant, “Was there sufficient variability at each stage to be necessary for inclusion?”
RESULTS/CONCLUSIONS: In the analysis, a significant portion of the variation (about 80 percent or more) was attributed to person-level variation. The person-level variation really included the variation between families as well. Because family-level variables, such as family drug use, parental attitudes about drug use, and family management practices and family conflict, had been identified as risk factors for adolescent drug use, it was possible that a significant amount of what had been labeled as person-level variation was really between-family variation. These components of variance needed to be estimated. The issue of small sample sizes at the family level in particular needed to be explored. In the 1997 and later NHSDAs, it was attempted to collect data on a random sample of pairs of individuals within a sample of households. Using Goldstein’s formula for the impact of this vis-a-vis ordinary least squares (OLS), the (n−1) would be equal to 2, and the inflation factor above and beyond simple random sampling would be (1+rho). For a large rho, this might make a considerable difference. If the level-2 sample sizes were larger, then even a small rho could make a difference. So, it was important to understand that any conclusions with respect to the use of HLM versus OLS applied only to the NHSDA and only with the current NHSDA sample sizes. The impact on other datasets or on the NHSDA with different sample sizes could be quite different. Although the variables that had been analyzed were interesting, there was a need to consider other drug-related variable scales and to extend the analysis to dichotomous data, such as past year use of marijuana or any illicit drug.
All material appearing in this report is in the public domain and may be reproduced or copied without permission from SAMHSA. Citation of the source is appreciated. However, this publication may not be reproduced or distributed for a fee without the specific, written authorization of the Office of Communications, SAMHSA, HHS.
Your browsing activity is empty.
Activity recording is turned off.
See more...
