Assessment of the computer-assisted instrument

CITATION: Caspar, R., & Penne, M. (2002). Assessment of the computer-assisted instrument. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768, pp. 53–84). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: The conversion of the National Household Survey on Drug Abuse (NHSDA) to computer-assisted interviewing (CAI) offered an opportunity to improve the quality of the data collected in the NHSDA in a number of ways. Some of these improvements were implemented easily and manifested themselves in more complete data (e.g., the ability to eliminate situations where questions were inadvertently left blank by the respondent). However, other improvements could only be realized through careful development and implementation of new procedures. Thorough testing was needed to determine whether these new procedures did, in fact, result in higher quality data.

METHODS: This chapter describes two significant revisions to how key NHSDA data items are collected and the effect of these revisions on the quality of the data obtained in the 1999 NHSDA. The first of these revisions was the addition of a methodology for resolving inconsistent or unusual answers provided by the respondent. This methodology was incorporated into the collection of a large number of the data items that are considered critical to the reporting needs of the NHSDA. The second revision dealt specifically with the way data on frequency of substance use over the past 12-month period was reported. This chapter also provides a review of several basic measures of data quality, including rates of “Don’t Know” and “Refused” responses, breakoff interviews, and the observational data provided by the interviewers at the conclusion of each interview. Where possible, these measures are compared between the CAI and paper-and-pencil interviewing (PAPI) NHSDA instruments as a means of assessing the effect of the move to CAI on data quality.

RESULTS/CONCLUSIONS: The results suggested that the move to CAI data collection has improved data quality, although in some cases the increase was fairly small because data quality for the PAPI NHSDA data was already quite high. Perhaps the most significant improvement to data quality came as a result of the inclusion of the inconsistent and unusual data checks (described in Section 4.1). This enhancement was a radical departure from the PAPI NHSDA and one that was possible only under the CAI mode of interview. The results presented here show that, although the CAI data did not suffer from a large amount of inconsistent or unusual data, the methodology was able to resolve a large number of these cases in a way that was both cost-effective and likely to enhance overall data quality for the items involved. Results of the change in the 12-month frequency of use item were somewhat difficult to interpret. The revised method for collecting these data resulted in higher reported frequencies, but whether this was due to the revision or simply the move from PAPI to CAI was impossible to determine. The distribution of responses and the fact that the mean frequency was higher under CAI than PAPI provided anecdotal support for the revised method of collecting these data, however. Finally, results from basic measures of data quality and interviewer debriefing items suggested that the CAI methodology reduced interview difficulties among respondents, helped to further enhance the degree of privacy, and appeared to contribute positively to item-level response rates.

Mode effects on substance use measures: Comparison of 1999 CAI and PAPI data

CITATION: Chromy, J., Davis, T., Packer, L., & Gfroerer, J. (2002). Mode effects on substance use measures: Comparison of 1999 CAI and PAPI data. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768, pp. 135–159). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: The shift from paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI) in the 1999 National Household Survey on Drug Abuse (NHSDA) was anticipated to have significant effects on the reporting of substance use by survey respondents. This expectation was based on several studies in the literature, as well as field testing done within the NHSDA project that showed respondents were more willing to report sensitive behaviors using audio computer-assisted interviewing than with self-administered paper questionnaires. Because of the great interest in analyzing trends in substance use prevalence, a critical component of the 1999 NHSDA redesign was the supplemental sample that employed the “old” PAPI NHSDA data collection methodology. The intent of this dual-sample design in 1999 was primarily to make it possible to continue to measure trends, using estimates from both before and after the redesign. However, the large dual-sample design of the 1999 NHSDA also can be viewed as an important survey research experiment assessing mode effects on the reporting of sensitive behaviors. With its dual-sample design, the 1999 NHSDA provided a large sample for assessing the impact of mode of interview. However, the intention of the 1999 NHSDA design was not to evaluate the mode effect, but to determine the impact of the overall change in method due to the redesign of the survey in 1999 on the time series of substance use statistics. The overall change involved many aspects of the survey design and estimation procedures in addition to mode, such as the sampling plan, the questionnaire, data editing, and imputation. Isolating the “pure” mode effect with these data was difficult and was complicated further by the unexpected impact of interviewer experience on substance use prevalence estimates (see Chapter 8 of the 2002 report). Nevertheless, the analyses presented in this chapter provide some important findings concerning mode effects, as well as on the comparability of pre-1999 NHSDA published estimates with estimates from the redesigned NHSDA. When studying the reporting of sensitive or illegal behaviors, conventional thinking has been that higher reporting is closer to the truth. This evaluation continued that approach, recognizing that this may not be true in every case.

METHODS: The authors compare the substance use prevalence estimates derived from the 1999 PAPI and CAI samples. In addition to providing NHSDA data users with information that will help them interpret NHSDA trends in substance use prevalence, the analysis also is of interest to survey researchers concerned with mode effects.

RESULTS/CONCLUSIONS: The results support previous research that shows higher reporting of sensitive behaviors with audio computer-assisted self-interviewing (ACASI) than with self-administered paper answer sheets. A total of 336 comparisons of unedited estimates from PAPI and CAI were made. Of these, 112 indicated significantly higher CAI estimates, while only 5 indicated significantly higher PAPI estimates. Higher CAI estimates were particularly evident for lifetime prevalence, the measures impacted least by questionnaire structure differences between CAI and PAPI. The analyses of edited and imputed NHSDA estimates showed mixed results for past year and past month use. Lifetime prevalence rates, which were minimally affected by editing and imputation, were generally higher with CAI than with PAPI. A total of 448 comparisons of edited and imputed estimates from PAPI and CAI were made. Of a total of 62 statistically significant differences in lifetime prevalence, 56 indicated a higher CAI estimate and 6 indicated a higher PAPI estimate. Results for past year and past month measures showed variation across substances and age groups and also were different in the full sample analysis than in the matched sample analysis. Out of 87 significant differences for past month or past year use, 41 indicated a higher CAI estimate and 46 indicated a higher PAPI estimate. For past month use of alcohol and cigarettes, the substances with the highest prevalence of use, the observed PAPI estimate was higher than the CAI estimate for every age group and for both the full and matched sample analyses, and 11 of the 16 comparisons were statistically significant. Marijuana use estimates also tended to be higher with PAPI, although most of these differences were not statistically significant. One clear conclusion is that the CAI mode of interviewing led to more internally consistent and complete data. Under PAPI, the need for editing and imputation to clarify recency of use was larger for most substances and provided a greater opportunity to influence estimates through the editing and imputation process. In summary, the CAI methodology produced more complete data with a lower requirement for editing and higher prevalence estimates when treating only unambiguous reports as positive indications of substance use.

Nonresponse in the 1999 NHSDA

CITATION: Eyerman, J., Odom, D., Wu, S., & Butler, D. (2002). Nonresponse in the 1999 NHSDA. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768, pp. 23–51). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: The redesign of the National Household Survey on Drug Abuse (NHSDA) in 1999 resulted in major changes in many aspects of the data collection procedures. This raised concerns that the response rates could be affected. In particular, the increased sample size, reduced clustering of sample segments, transition from a paper to a computerized screening instrument, and the transition from paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI) all had the potential to change the response rates. During the first quarter of 1999, an assessment of the progress in completing the fieldwork indicated a reduction in the response rates, relative to response rates achieved historically in the NHSDA. To address this problem, several management actions were implemented immediately. Although the response rates improved steadily throughout the remainder of the year, the result was significantly lower response rates for the 1999 NHSDA than for prior NHSDAs.

METHODS: Extensive analysis was undertaken in an attempt to understand the reasons for the drop in response rates and how it was related to each of the design changes. This chapter summarizes this analysis. It also discusses the management actions implemented during 1999 to improve the response rates and assesses the effectiveness of these actions.

RESULTS/CONCLUSIONS: The design changes between 1998 and 1999 corresponded with a large decrease in the response rates. A series of management efforts was taken to address the decrease, both in anticipation of the design changes and in reaction to unexpected results of the changes. In general, the efforts were successful. The extended analysis presented in this chapter summarizes the current understanding of the decline in the response rates in 1999. First, it appears that the previous understanding of the correlates of nonresponse was correct, but it does not completely explain the difference between 1998 and 1999. Second, management efforts taken during 1999 appear to have been successful in reducing the decline in the response rates, and this success carried over to 2000. Third, the computerized screening instrument reduced sampling bias by removing interviewer effects from the screening routine, and this had a small and negative impact on the response rates. Fourth, the transition from PAPI to CAI increased the response rates. Finally, much of the decline in 1999 can be attributed to changes in the composition of the field staff resulting from the large increase in sample size, most notably the reduced number of experienced field interviewers working on the project. However, this does not fully explain the decline in 1999.

Introduction. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues

CITATION: Gfroerer, J. (2002). Introduction. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768, pp. 1–8). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: In 1999, a major redesign of the National Household Survey on Drug Abuse (NHSDA) was implemented involving both the sample design and the data collection method of the survey. The strictly national design was changed to a much larger, State-based design to meet the needs of policymakers for estimates of substance use prevalence for each State. The data collection method was changed from a paper-and-pencil interviewing (PAPI) method to a computer-assisted interviewing (CAI) method, primarily to improve the quality of NHSDA estimates. This report has two purposes. First, it provides information on the impact of the redesign on the estimates produced from NHSDA. Researchers and other users of NHSDA data will find this information helpful in interpreting NHSDA estimates, particularly if they are interested in comparing data from the new design with data from the old design. The second purpose is to present research findings of interest to survey methodologists involved in designing and conducting surveys of all types, not just surveys of substance abuse.

METHODS: Implementation of these significant changes posed a number of difficult challenges involving a variety of methodological issues. These issues cover many aspects of the survey, including the management of fieldwork, the processing of data, and the reporting of results. This publication discusses several of the most critical issues encountered and describes how the research team conducting the survey addressed them.

RESULTS/CONCLUSIONS: Although these findings taken as a whole could be considered a case study in the redesign of a major ongoing survey, several of the chapters in this report present important research findings that are applicable to many types of surveys.

Redesigning an ongoing national household survey: Methodological issues

CITATION: Gfroerer, J., Eyerman, J., & Chromy, J. (Eds.). (2002). Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: In 1999, a major redesign of the National Household Survey on Drug Abuse (NHSDA) was implemented involving both the sample design and the data collection method of the survey. The data collection method was changed from a paper-and-pencil interviewing (PAPI) method to a computer-assisted interviewing (CAI) method, primarily to improve the quality of NHSDA estimates. Implementation of these significant changes posed a number of difficult challenges involving a variety of methodological issues. These issues cover many aspects of the survey, including the management of fieldwork, the processing of data, and the reporting of results.

METHODS: This publication discusses several of the most critical issues encountered and describes how the research team conducting the survey addressed them. This report has two purposes. First, it provides information on the impact of the redesign on the estimates produced from NHSDA. Researchers and other users of NHSDA data will find this information helpful in interpreting NHSDA estimates, particularly if they are interested in comparing data from the new design with data from the old design. The second purpose is to present research findings of interest to survey methodologists involved in designing and conducting surveys of all types, not just surveys of substance abuse. Although these findings taken as a whole could be considered a case study in the redesign of a major ongoing survey, several of the chapters in this report present important research findings that are applicable to many types of surveys.

RESULTS/CONCLUSIONS: N/A.

Impact of interviewer experience on respondent reports of substance use

CITATION: Hughes, A., Chromy, J., Giacoletti, K., & Odom, D. (2002). Impact of interviewer experience on respondent reports of substance use. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768, pp. 161–184). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: The redesign of the National Household Survey on Drug Abuse (NHSDA) in 1999 included a change in the primary mode of data collection from paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI). In addition, the sample design changed from a Nation-based design to a State-based one, and a supplemental sample was collected using PAPI to measure change between 1999 and earlier years. The overall sample size increased from 25,500 in 1998 to 80,515 in 1999, including the CAI and supplemental PAPI samples. Consequently, it was necessary to hire more interviewers than in previous years, which resulted in a higher proportion of inexperienced interviewers. New interviewing staff turnover also was high in 1999, requiring additional training of newly hired interviewers and contributing to the general inexperience of the interviewing staff for both the CAI and PAPI samples.

METHODS: This chapter describes the analysis that was done to understand and explain the relationship between the effects of changes in mode, sample design, and interviewer staffing.

RESULTS/CONCLUSIONS: The analysis presented in this chapter indicates that the uneven mix of experienced and inexperienced NHSDA field interviewers (FIs) in 1999 had some effect on estimated substance use rates for that year. Overall, the effect on 1999 CAI prevalence estimates was smaller in magnitude than the effect on 1999 PAPI rates, which was an indication that the CAI methods played a role in reducing the effects of FI experience on substance use rates. However, because the mechanism of these effects was unknown, it was determined that additional studies would be undertaken to increase an understanding of this phenomenon. In the meantime, analyses of interviewer effect as seen in this chapter were to continue to be presented in subsequent reports. These findings resulted in an added emphasis—in training and in the field—on encouraging experienced and new FIs to follow the interview protocol.

Development of editing rules for CAI substance use data

CITATION: Kroutil, L., & Myers, L. (2002). Development of editing rules for CAI substance use data. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768, pp. 85–109). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: A major change to the study protocol for the 1999 National Household Survey on Drug Abuse (NHSDA) was the shift from paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI). Although many of the substance use questions are similar in the two instruments, there are some differences. In addition, whereas the PAPI questionnaire required respondents to answer all questions in most sections, the CAI instrument makes extensive use of skip instructions. These significant differences in the nature of the data obtained in the new and old instruments necessitated the development of entirely new editing rules.

METHODS: This chapter discusses the development of the new editing rules for the NHSDA CAI data and presents the results of an investigation of alternative editing methods for CAI data. The analysis was based primarily on data from the first 6 months of data collection in 1999. The authors discuss data quality issues that were affected by conversion to CAI, present the general methodological approach used to define and test alternative rules for defining a usable case, discuss alternative editing rules, along with the final rule that was implemented, and present selected substance use measures to compare the impact of the new CAI editing procedures with the former PAPI procedures.

RESULTS/CONCLUSIONS: The authors find for some substances that there was little change in going from the raw CAI data provided by the respondents to the final imputed estimates. For example, the estimate of marijuana use in the past month changed from 4.6 percent in the raw data to 4.7 percent after editing and imputation. In comparison, editing made a greater contribution to estimates of past month marijuana use and cigarette use in the 1998 data. For past month marijuana use, the raw estimate in 1998 was 4.0 percent (weighted), the estimate after editing was 5.0 percent, and the final imputed estimate was 5.0 percent. Thus, the editing procedures that had been used in the NHSDA since 1994 increased the 1998 estimate of past month marijuana use by about 25 percent relative to the raw data; the additional impact of imputation on the final estimate of past month marijuana use was virtually nil. Differences in the impacts of the 1999 CAI editing and imputation procedures and those used in prior years were even more pronounced for less commonly used substances. Following imputation, a total of 70 CAI respondents were classified as past month heroin users, or a net increase of only 4 cases relative to the raw and a net increase of 6 relative to the edited. In comparison, the editing procedures in 1998 nearly doubled the number of respondents classified as being past month heroin users (17 respondents in the raw data and an additional 11 cases who were assigned to this category through editing). In all, these changes in the 1999 CAI editing procedures represent an improvement over the way missing or inconsistent data had been handled in that these issues are resolved primarily through statistical methods.

2001 National Household Survey on Drug Abuse: Incentive experiment combined quarter 1 and quarter 2 analysis

CITATION: Office of Applied Studies. (2002, July). 2001 National Household Survey on Drug Abuse: Incentive experiment combined quarter 1 and quarter 2 analysis (RTI 07190.388.100, prepared for the Office of Applied Studies, Substance Abuse and Mental Health Services Administration, by RTI under Contract No. 283-98-9008). Rockville, MD: Substance Abuse and Mental Health Services Administration.

PURPOSE/OVERVIEW: The purpose of this report was to summarize the results of the incentive experiment in the 2001 National Household Survey on Drug Abuse (NHSDA) and to evaluate the best treatment option for the use of monetary incentives in future NHSDAs. The NHSDA experienced a considerable decline in response rates in 1999 due in part to the transition from a national probability sample to a State probability sample designed to yield State-level estimates. A series of management adjustments were made to improve the response rates in 2000. In general, the adjustments were successful, and a recovery was made from the 1999 decline. However, the rates remained below the project target rate and the historical NHSDA average. An incentive given to respondents was considered as an option for addressing the downward trend in respondent cooperation. However, it has been noted that incentives may have a negative impact on areas of data quality other than unit response rates (Shettle & Mooney, 1999). Although it may lead to better response rates, it is possible that the additional costs may exceed the constraints of the project budget. In an effort to understand the risks and benefits associated with a respondent incentive, the NHSDA’s sponsor, the Substance Abuse and Mental Health Services Administration (SAMHSA), requested that the NHSDA’s contractor, Research Triangle Institute (RTI), conduct a special methodological field test in the form of an incentive experiment.

METHODS: The experiment was overlaid on the NHSDA main study data collection sample and scheduled during the first two quarters of 2001. A randomized, split-sample, experimental design was included with the main study data collection of the NHSDA to compare the impact of $20 and $40 incentive treatments with a $0 control group on measures of respondent cooperation, data quality, survey costs, and population substance use estimates. This report is the second of two. The first report describes the experimental design and the results from data collection in the first quarter of 2001 (Eyerman, Bowman, Odom, Vatalaro, & Chromy, 2001a). This second report provides combined findings for the full experiment for both quarters.

RESULTS/CONCLUSIONS: The results were very promising. The $20 and the $40 treatments produced significantly better interview response rates than the control for the combined results of both quarters of the experiment. This improvement led to a gain in overall response rates of about 10 points for each treatment. Furthermore, both the $20 and the $40 treatments more than paid for themselves, each resulting in a lower data collection cost per completed case, including the incentive, than the control. The incentives had a favorable impact on measures of respondent cooperation. Both treatments had significantly lower refusal rates than the control’s rate, and the $40 treatment had significantly lower noncontact rates than the control’s. Field interviewers reported that the incentives reduced the amount of effort required to complete a case and that the incentives influenced the respondent’s decision to cooperate. Perhaps most importantly, the incentives had little impact on the population estimates of past month alcohol, cigarette, or marijuana use. The prevalence rates for past month use of these substances by respondents in the treatment groups were not significantly different from those reported by those in the control. This suggests that incentives encourage greater participation by respondents, but do not change their self-reported substance use. Incentives may thus improve estimates by reducing nonresponse bias without increasing response bias. Taken together, the results clearly favor a $40 incentive for all individuals selected for the NHSDA.

Changes in NHSDA measures of substance use initiation

CITATION: Packer, L., Odom, D., Chromy, J., Davis, T., & Gfroerer, J. (2002). Changes in NHSDA measures of substance use initiation. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768, pp. 185–220). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: The National Household Survey on Drug Abuse (NHSDA) data are used to generate a number of annual estimates relating to the initiation of substance use. Using the responses to retrospective questions about age of first use, annual estimates are generated for the incidence rate of first substance use, for the number of initiates to substance use, and for the average age of first use. Estimates of new initiates and average age of first use are reported for all lifetime substance users aged 12 or older. These use initiation measures are important because they can capture the rapidity with which new substance users arise in specific population subgroups and can identify emerging patterns of substance use. The redesign of the NHSDA in 1999 introduced some changes in the questions about initiation, as well as the method of administration. In the presence of these changes, the overall data processing and estimation methodologies were reviewed and, in some cases, revised. (The revisions to the editing and imputation procedures are summarized in Chapters 5 and 6 of this 2002 report.) Limitations of the existing methodology for computing incidence rates were found. As a result, a new incidence rate methodology was developed. The definition of initiation of daily cigarette use was modified, and an adjustment to the program logic in the calculation for the incidence of first daily use of cigarettes also was made.

METHODS: This chapter is organized in three sections addressing the impact of methodological change on substance use initiation measures. Section 9.1 describes the old and new incidence rate estimation methods and evaluates its impact; the impact of the editing and imputing changes is evaluated in conjunction with the method impact. Section 9.2 focuses on the questionnaire wording and administration mode effects. Section 9.3 focuses on all the issues associated with initiation of first daily use of cigarettes.

RESULTS/CONCLUSIONS: Although the estimates for individual years were quite variable, the overall average impact of the new editing and imputation procedures was to increase incidence rates for both age groups (12 to 17 and 18 to 25) and to increase the estimated number of new initiates. The largest impacts were observed for pain relievers and other substances that use multiple gate questions before presenting the age-of-first-use question. Estimates of the average age of initiation did not appear to be consistently changed in either direction by the change in editing and imputation. The impact of the new method of incidence rate calculation also was studied. The number of new initiates occurring at age 17 was presumably quite high for almost all substances. The new incidence rate calculation rules treated respondents as 17 year olds right up to (but not including) their 18th birthday. The old rule classified respondents as 18 years old for the entire year in which their 18th birthday occurred. Thus, the new calculation method had the effect of increasing the estimates of time at risk and the number of initiates for 17 year olds, but because the number of initiates is high at age 17, the overall impact was greater on the numerator than the denominator. As a result, the incidence rates for youths aged 12 to 17 increased and the incidence rates for adults 18 to 25 usually decreased somewhat with the new method. Mode effects could not be cleanly isolated because of some accompanying changes in the question routing process and supplementary questions on date of first use for recent users that were implemented in conjunction with the implementation of computer-assisted interviewing (CAI). Within this limitation, comparable data from paper-and-pencil interviewing (PAPI) and CAI were studied. One somewhat surprising result was that the level of missing or inconsistent data actually increased with the introduction of CAI. However, this may have resulted because of the increased number of checks employed to identify inconsistent data in the post-survey processing. The increase in the proportion of missing age-at-first-use data may have been facilitated by the respondent’s option to answer “Don’t know” or “Refused” in CAI. A pattern of mode effects similar to that observed for reported lifetime substance use was found, with generally higher reporting of initiation in CAI than in PAPI.

Predictive mean neighborhood imputation for NHSDA substance use data

CITATION: Singh, A., Grau, E., & Folsom, R., Jr. (2002). Predictive mean neighborhood imputation for NHSDA substance use data. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768, pp. 111–133). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: In 1999, the instrument used to administer the National Household Survey on Drug Abuse (NHSDA) was changed from a paper-and-pencil interviewing (PAPI) format to a computer-assisted interviewing (CAI) format. In previous years, imputation of missing values in recency of use and frequency of use in the past 12 months was accomplished with an unweighted sequential hot-deck procedure. In the spirit of improving the quality of estimates from the redesigned NHSDA and as a result of fundamental differences between PAPI and CAI, there was a need to change the way missing data were edited and imputed. The implementation of the “flag and impute” editing rule, described in this 2002 report’s Chapter 5, and the desire to impute more variables required a new method that was rigorous, flexible, and preferably multivariate.

METHODS: This chapter presents a new imputation method with these characteristics, termed predictive mean neighborhoods (PMN), that was used to impute missing values in the NHSDA substance use variables. Following a discussion of background in Section 6.1, this chapter outlines the previously used hot-deck method, along with its limitations, in Section 6.2. The new method is described in general in Section 6.3, followed by details of the method in Section 6.4. Section 6.5 compares the method with other available methods and provides details concerning the motivation for employing a new method. In the concluding section (Section 6.6), the impact of imputation on substance use estimates is compared between PAPI and CAI.

RESULTS/CONCLUSIONS: The authors assess the relative impact of imputation for 1998 and 1999 (CAI) estimates of past month use, past year use, and lifetime use of all substances in the core section of the questionnaire. Because of numerous changes between the 1998 sample and the 1999 CAI sample, it would not be advisable to compare the final prevalence estimates (final percent) between the two samples. However, some comments can be made about the comparison of the “relative percent from imputes” between the two samples. In general, imputation had greater impact on the prevalence estimates in the CAI sample than on the estimates in the PAPI sample. With the implementation of the flag-and-impute editing rule in the CAI sample, where inconsistencies would be resolved by imputation, this result was not surprising. The exceptions to this rule were either due to differences in questionnaire format between PAPI and CAI or to attributes of the modules themselves.

1999–2001 National Household Survey on Drug Abuse: Changes in race and ethnicity questions

CITATION: Snodgrass, J. A., Grau, E. A., & Caspar, R. A. (2002, October 18). 1999–2001 National Household Survey on Drug Abuse: Changes in race and ethnicity questions (RTI 07190.488.010, prepared for Office of Applied Studies, Substance Abuse and Mental Health Services Administration, Contract No. 283-98-9008). Research Triangle Park, NC: RTI.

PURPOSE/OVERVIEW: Since the inception of the National Household Survey on Drug Abuse (NHSDA), renamed the National Survey on Drug Use and Health (NSDUH) as of 2002, questions have been included to determine the race and ethnicity of each respondent. Race and ethnicity are routinely used as part of the demographic breakdowns in the analyses and the various reports generated from the survey. From 1971 to 1998, the race and ethnicity questions underwent few changes. (See Appendix A of this 2002 report for the full list of race and ethnicity questions used for each NHSDA survey year from 1971 to 2001.) However, along with the switch from paper-and-pencil interviewing (PAPI) methods of questionnaire administration to computer-assisted interviewing (CAI) methods in 1999, the race and ethnicity categories were updated pursuant to new Office of Management and Budget (OMB) directives.

METHODS: This report details the revisions to the race and ethnicity questions. The report includes the history of the change, how the change affected the editing and imputation procedures, and how it changed the derivation of the race and ethnicity variables used in NHSDA analyses.

RESULTS/CONCLUSIONS: N/A.

Substance abuse among older adults in 2020: Projections using the life table approach and the National Household Survey on Drug Abuse

CITATION: Woodward, A. (2002, December). Substance abuse among older adults in 2020: Projections using the life table approach and the National Household Survey on Drug Abuse. In S. P. Korper & C. L. Council (Eds.), Substance use by older adults: Estimates of future impact on the treatment system (HHS Publication No. SMA 03-3763, Analytic Series A-21, pp. 95–105). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: One way of projecting substance use problems among older adults is to use a life table approach. The National Household Survey on Drug Abuse (NHSDA), a major data source on substance use and abuse among the U.S. civilian population aged 12 or older, could potentially be used in a life table approach.

METHODS: In the life table approach, cohorts are followed for a given period of time to determine their various outcomes. In the table, one cohort’s drug use is followed as the cohort ages. The NHSDA data are reviewed to see whether they can be used in this fashion to produce estimates for the groups who used illicit drugs or drank heavily or who were substance dependent.

RESULTS/CONCLUSION: A review of the NHSDA shows that, even with its increase in sample size in 1999, the survey currently does not provide sufficient detailed data to be used in a life table approach. The survey could be expanded, however, with selected questions added in a special supplement so that a life table or other more sophisticated approach could be used to make projections of substance use problems among older adults.

Summary of NHSDA design changes in 1999

CITATION: Wright, D., Barker, P., Gfroerer, J., & Piper, L. (2002). Summary of NHSDA design changes in 1999. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768, pp. 9–22). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

PURPOSE/OVERVIEW: An entirely new sample design and a state-of-the-art data collection methodology were implemented with the 1999 National Household Survey on Drug Abuse (NHSDA). The sample design changed from a national, stratified, multistage area probability sample to a 50-State design, with independent stratified, multistage area probability samples selected in each State. The sample size increased from about 25,500 interviews in 1998 to about 67,000 interviews in 1999. For the first time in NHSDA history, the 1999 survey administered the interview using computer-assisted interviewing (CAI) technology exclusively, including both computer-assisted personal interviewing (CAPI) and audio computer-assisted self-interviewing (ACASI). Because this new methodology was being implemented, an additional national sample was selected, and about 14,000 interviews administered using the previous paper-and-pencil interviewing (PAPI) methodology.

METHODS: Together, the PAPI sample and the CAI sample served three purposes. First, the PAPI samples for 1998 and 1999 provided a way to continue to measure the trend in substance use for that period. Second, with both representative samples for 1999, the effect of the change in data collection from PAPI to CAI could be measured without being confounded with the measurement of trends. Third, with a measurement of the impact of the switch to the CAI methodology, estimates for 1998 and earlier years could be adjusted to be comparable with CAI estimates for 1999 and later so that long-term trends in substance use could be estimated.

The CAI and PAPI samples for 1999 together resulted in 81,000 completed interviews. The 1999 NHSDA fully employed another technological innovation: use of a hand-held computer at each sample dwelling unit to conduct household screening and to select the sample person(s) for the interview. With this new design, technology, and markedly increased sample size, the structure of the data collection staff also had to be modified significantly for 1999. This chapter presents details of these changes.

RESULTS/CONCLUSIONS: N/A.