NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Carey TS, Crotty KA, Morrissey JP, et al. Future Research Needs for the Integration of Mental Health/Substance Abuse and Primary Care: Identification of Future Research Needs from Evidence Report/Technology Assessment No. 173 [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2010 Sep. (Future Research Needs Papers, No. 3.)

Cover of Future Research Needs for the Integration of Mental Health/Substance Abuse and Primary Care

Future Research Needs for the Integration of Mental Health/Substance Abuse and Primary Care: Identification of Future Research Needs from Evidence Report/Technology Assessment No. 173 [Internet].

Show details

Executive Summary

The Effective Health Care Program was initiated in 2005 to provide valid evidence about the comparative effectiveness of different medical interventions. The object is to help consumers, health care providers, and others in making informed choices among treatment alternatives. Through its comparative effectiveness reviews, the program supports systematic appraisals of existing scientific evidence regarding treatments for high-priority health conditions. It also promotes and generates new scientific evidence by identifying gaps in existing scientific evidence and supporting new research. The program puts special emphasis on translating findings into a variety of useful formats for different stakeholders, including consumers.

The full report and this summary are available at www.effectivehealthcare.ahrq.gov/reports/final.cfm.

Background

Half of the care for common mental disorders in the United States is delivered in general medical settings.1 Integrated mental health and general medical care models involve the systematic linkage of mental health and primary care providers and require communication or coordination between providers to meet both the mental and general health needs of the patient.

In 2008, the Agency for Healthcare Research and Quality (AHRQ), commissioned the University of Minnesota Evidence-based Practice Center (EPC) to conduct a systematic review of the literature evaluating the integration of mental health and substance abuse treatment with primary care. The review addressed six key questions (KQ); Table A lists a summary of findings, limitations, and future recommendations.

Table A. Summary of findings, limitations, and recommendations from the 2008 AHRQ review (modified using table 19 of the original report).

Table A

Summary of findings, limitations, and recommendations from the 2008 AHRQ review (modified using table 19 of the original report).

The report found substantial evidence for improved outcomes through integrated care, although most of the evidence was for treatment of depression in primary care settings. Studies reported positive results for symptom severity, treatment response, and achievement of remission when compared with usual care. The level of integration did not seem to be related to treatment outcomes. Most of the studies addressed the integration of mental health professionals into primary care; few examined the integration of primary care into mental health. The majority of the studies involved older patients. Some studies that found improved outcomes with integrated care have been largely composed of minority populations. The main barriers to a broader use of integrated care include programmatic costs, insurance coverage, and relationships with multiple payers. The U.S. Department of Veterans Affairs (VA) was shown to offer a good model of a sustained program. Key elements of successful models included active support at all levels of the organization and through specific funding.

The authors of the 2008 AHRQ review (Evidence Report/Technology Assessment No. 173) identified multiple research gaps and limitations (summarized in Table ES-1), including conditions other than depression or care integration in younger populations. Other gaps included research in rural areas, examination of the use of information technology (IT), and development of financial models. One of the largest gaps was on integrating medical care into mental health care for patients with serious and persistent mental illness.

In February 2010, AHRQ commissioned the RTI International–University of North Carolina at Chapel Hill Evidence-based Practice Center (RTI-UNC EPC) to work with stakeholders to develop a prioritized list of future research needs in this area that would inform researchers, funders, practitioners, advocacy groups, patients, and family members. A structured approach, including the AHRQ population, intervention, comparator, outcome, timeframe, setting (PICOTS) framework, to future research needs prioritization is new: this project, therefore, also served as a pilot for development and testing of methods to conduct such an evaluation. In the future, it is anticipated that all AHRQ-sponsored comparative effectiveness research systematic reviews will contain a documentation of future research needs.

Methods

We developed a comprehensive list of research gaps from the 2008 report and identified ongoing research projects through searches of relevant databases. We mapped each research project to its corresponding research gap. We conducted a scan of the peer-reviewed literature on integrated care published since 2008 and found that this continues to be a very active research area. We did not update the systematic review but used the literature scan to demonstrate the volume of research activity.

The 10-member stakeholder panel included representatives of advocacy groups, researchers, providers of care, federal government funders of research, and professional organizations. A variety of methods were employed to screen and select stakeholders. When screening stakeholders for participation, we evaluated a broad spectrum of organizations and interest groups, including federal agencies. We used moderated group discussion, email, and online prioritization to solicit input on multiple occasions. We reviewed conflict-of-interest forms. Had a substantial conflict of interest been discovered, the stakeholder would have been considered ineligible for participation on the panel and another stakeholder would have been approached in his or her place. To avoid any unfair advantage resulting from future requests for applications (RFAs) arising from this report, the stakeholders prioritized the final list of gaps individually and were not privy to the final rank order prior to publication of the report for public comment.

We used several methods to prioritize research gaps: a combination of teleconference, online Web meetings, and email. During an initial conference call, the stakeholder group recommended adding several gaps to the initial gaps list. We used an online priority-setting exercise for the initial prioritization. We presented the 10 stakeholder panelists with 40 non–rank-ordered research gaps and asked them to rate the gaps twice, using different methods. First, stakeholders were asked to use a 1–5 Likert scale of “importance” to indicate whether each gap was of low or high priority for future research, and second, we provided an online forced prioritization exercise in which each stakeholder was given a total of 20 “chits” of which a maximum of 5 could be assigned to any one gap. We held a second online meeting to review the initial findings and refine the list. The second round of prioritization also utilized an online exercise. Stakeholders were presented with a nonranked list of 20 gaps and provided a total of 12 chits to assign to gaps, 4 of which could be used for any one gap. The results from the second prioritization exercise formed the basis of the final prioritized list of further research needs. The final list of 13 research needs incorporated all gaps receiving two or more chits from stakeholders. We developed each future research need into a potential future study by specifying potential PICOTS, study design considerations, and, where relevant, power calculations.

We evaluated potential study design considerations for each of the prioritized future research needs against specific criteria such as stage of research, generalizability, feasibility, cost-effectiveness, and ethics. For instance, we suggested demonstration pilots for studies that require preliminary data on acceptability and feasibility before considering large studies or trials. We considered case studies where testing is necessary across multiple settings. When outcomes are available from claims data, we suggested secondary data analysis as a cost-effective method of using existing data to study identified outcomes. When controlled trials may be infeasible due to high costs and administrative difficulty, we suggested observational studies. We suggested randomized trials when practical and feasible for studies examining the efficacy of one treatment or system compared with another. We determined trial feasibility by conducting power analysis and examining the practicalities of clustered randomization, contamination, setting, and timeframe. A general discussion of sample size considerations in these practice-based studies was developed. Meta-analysis was considered when existing trials could be combined.

Results

The following is the final prioritized list of future research needs, derived from an initial list of 40. Research priorities are listed from 1 to 13. The first three topics had the same amount of enthusiasm by the stakeholders and share the position of top future research need. The research gap, as defined by the authors of the 2008 AHRQ report and modified and prioritized by the stakeholders, is presented initially, followed by a restatement as a research need incorporating the population and comparator, when appropriate. Key study design considerations are briefly described. The full report includes full descriptions of the PICOTS and additional study design considerations, which include advantages and disadvantages of approaches to addressing the research need. A rationale for each future research need is included in Appendix A.

Research Need #1. What are effective methods of integrating primary care into specialty mental health practice settings? Studies would include both mental and general health outcome(s) (e.g., obesity and depression)

Among adults with serious mental illness seen in specialty mental health settings, what are effective methods of integrating primary care components such as preventive interventions and chronic disease management, into their mental health care, compared with referral to primary care?

Study Design Considerations. We could envision several study designs to enhance understanding of integration of primary care into mental health.

  • Demonstration pilots to determine the acceptability and feasibility of this type of integrated care.
  • Use of claims data with propensity matched controls and difference-in-difference models as a low-cost approach to assessing patient-level outcomes.
  • Prospective observational studies with control for clinic and patient characteristics, although with the expected challenges of the need to control for case mix and practice environmental differences between settings.
  • Randomized trials of integration of primary care into specialty mental health practice will require randomization at the level of the practice, with analysis using clustered methods. Randomization at the level of the individual would likely be administratively infeasible. The primary unit of analysis in such studies would be at the patient level.

Research Need #2. Effectiveness of cross-cutting models/strategies for integration of mental health into primary care

Can the same personnel successfully provide integrated mental health services to primary care patients who may have any one of several mental health diagnoses, such as depression, anxiety disorder, or problem drinking, and will this lead to different outcomes when compared with separate programmatic initiatives?

Study Design Considerations.

  • Studies of the feasibility and fidelity of cross-cutting integrated care will be case studies. Are integrated care principles in a cross-cutting framework acceptable and feasible for practices, providers, and patients in several primary care settings?
  • Once feasibility is established, studies of the efficacy of the cross-cutting approach can employ randomized trials, with the practice site as the unit of randomization. Depending on the comparator used (usual care vs. diagnosis-specific care), the required sample size would be either moderate or very large (see power analysis, pp. ES-11, 21).

Research Need #3. Studies examining the use of information technology (IT) including text messaging, use of the Internet, and effective use of electronic health records for integrated mental health care and general medical health care

Among adults or children with mental health diagnoses seen in primary care, what is the effect of using IT to provide integrated mental health care, compared to integrated care without the use of health care IT, on clinical outcomes, costs, and patient and provider satisfaction?

Study Design Considerations. We could envision several study designs to enhance understanding of the use of information technology in the integration of mental health care into primary care.

  • Demonstration projects at a limited number of sites could be used to assess the feasibility of such interventions.
  • Randomized trials or observational studies with a contemporaneous control group. The practice would need to be the unit of randomization. Randomizing patients within a practice would likely not be possible due to administrative complexity and possible contamination of the control patients.
  • If the comparison arm is integrated care without health IT components, the effect on outcomes would likely be modest. This would necessitate a very large sample size, on the order of 12,000 patients.

Research Need #4. Studies examining sustainability of integrated care without external support, such as grant funding. Integrated care can be delivered with special grant funding but are there ways of supporting it following or in lieu of grant funding?

In primary care practices that successfully integrate mental health care, what practice and program characteristics are associated with program sustainability in the absence of grant funding?

Study Design Considerations. Case studies of both successes and failures to sustain integrated care.

  • Both qualitative and quantitative information could be used. Qualitative information might include interviews with practitioners, administrators, and payers. What led them to continue with the program after the end of the initial grant or contract? Which components were sustained and why? Quantitative information needed includes the financial aspects of the grant that the integrated care system started with, as well as documentation of any internal transfers that are occurring after the end of the grant.
  • If data were available, larger secondary analyses could be conducted of integrated care systems. An example might be examination of VA systems or state mental health systems if they had sufficient variability across practices.

Research Need #5. Studies examining the dissemination of successful models/strategies into community settings. How can efficacious interventions be incorporated into everyday practice in the face of weak incentives and competing medical priorities?

What factors facilitate the adoption of integrated mental health care models into community-based primary care practices serving adults or children with mental health diagnoses?

Study Design Considerations.

  • One approach would be to re-conduct randomized trials of integrated care in community settings including smaller practices, private practices, and rural practices. Such study replication would be expensive, and integrated care has already been demonstrated to be effective in multiple trials.
  • Studies examining the fidelity of the integrated care intervention would be qualitative and organizational.
  • Given that relatively little research has been conducted to date on integration of primary care into specialty mental health practice, studies examining the generalizability of those models may be premature pending additional efficacy studies.
  • Since the studies are largely qualitative and organizational, the main focus will be on lessons learned from implementation in community settings. What are the common adaptations necessary for successfully implementing integration practices in the community setting? Several case studies will be needed for each type of practice. Given the variety of settings and populations, at least 8–12 practices will need to be examined. Small or rural practices may need to adapt integrated care interventions to their circumstances, and documentation regarding how interventions change will be useful to the practitioner community.

Research Need #6. Studies examining effective models/strategies of integrated or “bundled” payment for integrated care. Are there effective ways of combining primary care reimbursement mechanisms with reimbursements for mental health care services?

Among primary care or specialty mental health practices employing integrated care services, what is the effect of bundled payment systems on the costs of services compared with nonbundled fee-for-service payment?

Study Design Considerations.

  • Randomization at the level of the practice would be ideal but may be organizationally difficult.
  • The most feasible study design will be an observational study comparing organizations that are using bundled payment with those that are not. Organizations that are early adopters of bundled payment may be systematically different from later adopters.
  • Description of types of payment could be based on figure 13 from the 2008 AHRQ review2 (Appendix B), although this framework may need to be modified as new payment systems are developed.

Research Need #7. Studies identifying the effectiveness of various components of integrated care, and determining the value added by each component individually and synergistically. What are the efficacious elements of integrated care?

Among patients with mental health diagnoses seen in primary care practices, what is the effectiveness of components of integrated care services when compared with each other or with programs incorporating multiple components of integrated care, in leading to improved mental health outcomes?

Study Design Considerations. A number of study designs could be considered to answer this question.

  • Randomized trials could be designed to test various components in a 4-cell matrix of integrated care but this would be expensive given the variety of approaches and individual components in question. The differences between the intervention and comparison groups are likely to be modest, necessitating large sample size, up to 80 practices and 12,000 patients (see power analysis). Descriptive case studies may be a more feasible option to identify successful and unsuccessful integration components. Both qualitative and quantitative information could be used. Qualitative information would include interviews with practitioners, patients, and administrative staff to discern which program elements were vital to success, considering cost and administrative complexity.
  • As with research need #5, the main focus will be on lessons learned from successful implementation. Sample sizes will not need to be large but different types of interventions will need to be examined.

Research Need #8. Cost-effectiveness of integrated models from the societal perspective

Among adults or children with mental health diagnoses who receive integrated care in either primary care or specialty mental health practices, how does the cost-effectiveness of care compare with that of nonintegrated care?

Study Design Considerations.

  • A cost-effectiveness analysis conducted from the societal perspective using the cost per quality-adjusted life year (QALY) as the outcome measure. Depression-free days (or its counterpart for other conditions) can be cross-walked to QALYs. Cost per QALY could be assessed using incremental cost-effectiveness ratios, the net benefits framework, cost-effectiveness planes, and acceptability curves. Clinical characteristics of the patients should extend beyond depression. Cost-effectiveness of the integrated care intervention may vary depending on the case mix and severity of the mental health problems addressed.
  • The “societal perspective” in this case may be close to the perspective of the insurer, but a true societal perspective will also include assessments of indirect costs, such as cost of time off work and disability payments.

Research Need #9. Studies examining the business case for integration. When a practice or system invests in integrated care staffing and services, what are the revenues generated, and what are the effects on downstream costs, such as hospitalization?

Among primary care or mental health specialty practices that implement integrated care programs for adults or children, what revenues are generated, and what costs are borne by the practice, including costs and revenues related to hospitalizations?

Study Design Considerations.

  • This research need requires observational studies. Given the variability in integrated care and the patient populations treated, adequate description of the intervention and patient case mix will be critical. The business case may be more readily achieved among patient populations who are more severely mentally ill, because they are the highest-cost patients.
  • The number of cases for study will likely need to be at least 8, in order to assess variability across practice size, patient case mix, and type of organization. If great variation in business case results is found, more case studies may be needed.

Research Need #10. Effectiveness of measurement-based integrated care for case identification, treatment, and monitoring, focusing on mental health conditions other than depression

Among adults with common mental health diagnoses other than depression seen in primary care practices using integrated mental health care, what is the additional benefit, if any, of measurement-based care (use of valid short instruments) with regard to the identification of mental health problems and mental health clinical outcomes?

Study Design Considerations.

  • These studies presuppose that short screening and measurement tools exist for diagnoses such as anxiety disorders, posttraumatic stress, and others.
  • Clinical outcomes for the patients would need to be assessed using valid instruments, but the treating clinician would need to remain masked to the study instrument outcomes in order to prevent contamination of the comparator group. Ethical and practical issues of such masking will be significant design issues.
  • Intervention would need to be at the level of the practice, not the provider, in order to avoid contamination of the comparator group.
  • Randomized trial design, with the randomization occurring by practice, would be optimal.

Research Need #11. Effectiveness of integrated care for patients with dual Medicaid and Medicare

Among adults with mental health diagnoses who are dually insured by Medicare and Medicaid and seen in primary care or mental health practices, what is the effectiveness of integrated care on clinical outcomes when compared with usual care (i.e., referral), and how does this relative effectiveness of integrated care compare to the relative effectiveness of integrated care for adults with mental health diagnoses who are not dually insured by Medicare and Medicaid?

Study Design Considerations.

  • Subgroup analysis of dual eligibles could be conducted within a larger trial or observational study. Ideally the population of dual eligibles would be defined in advance and intended as a subgroup question. The difficulty with such studies is having enough power to answer a secondary question.
  • A meta-analysis of subgroup data across existing trials has the advantage of providing adequate power by pooling data from different sources including small inadequately powered studies.
  • Secondary analysis of large datasets offers a potentially efficient and cost-effective method of studying integrated care for patients with dual Medicaid and Medicare eligibility. Datasets might include national comorbidity data, Substance Abuse and Mental Health Data Archive (SAMHDA), and linked Medicare and state Medicaid data sets.

Research Need #12. Effectiveness of integrated care in the presence of both general medical comorbidities such as diabetes or chronic pain, as well as mental health comorbidities, such as depression and anxiety

Among patients with serious mental health conditions and general medical and/or mental health comorbidities, what is the effectiveness of integrated care compared with usual practice on receipt of guideline-concordant care and on mental health and general medical patient outcomes?

Study Design Considerations.

  • One of several potential designs would be subgroup analysis within a larger trial or observational study, ideally powered to answer the secondary question.
  • A meta-analysis of subgroup data across several trials could answer this question and has the advantage of providing adequate power by pooling data from different sources including small studies. Whether the component studies could support such a meta-analysis would need to be assessed.
  • Secondary analysis of large datasets offers a potentially efficient and relatively low-cost method of studying integrated care for patients with dual comorbid mental and general medical conditions. Drawing information on medical conditions from an electronic medical record would also have efficiency advantages.
  • Randomized trials would require randomization at the level of the practice, with analysis using clustered methods. The primary unit of analysis in such studies would be at the level of the patient. Sample size would depend on the size of any practice effects and on the outcome measure used. Observational studies with control for clinic and patient characteristics are another option, but with the challenge of case mix adjustment.

Research Need #13. Effectiveness of the medical home as a model/strategy for integrated care

How does the effectiveness of the primary care medical home model compare with that of integrated care models for patients with mental health conditions in primary care and specialty mental health practices?

Study Design Considerations.

  • A scan of gray literature on the topic might be helpful because the concept of medical home is relatively new as a model for integrated care.
  • Surveys of current PCMH accredited practices could be conducted, comparing their activities with those of primary care practices using integrated mental health care principles. If the PCMH practices are not addressing mental health issues, then conducting a study comparing the outcomes would not be very productive. Cross-walking the principles of care management will be very useful, because similar practices may use different terminology.
  • Only after the first several steps are taken will it be wise to conduct cohort studies comparing the outcomes of patients with depression or other mental health diagnoses seen in PCMH with those of patients seen in integrated care practices that do not consider themselves PCMH. Randomization would very difficult given the complexity of PCMH accreditation.

Power analyses were conducted based on several hypothetical designs for cluster randomized trials that would fit one or more of the research needs outlined in this report. The objective was to determine the approximate numbers of practices and patients needed in order to have power of 0.80 to detect a clinically meaningful difference in outcomes with a two-tailed test at a significance level of 0.05. Various assumptions were used for the treatment effect size and the amount of variation between sites in the treatment effect. For a comparison of mental health outcomes in primary care practices, assuming an effect size of 0.3 and an intraclass correlation of 0.05, the required sample size would be about 500–700 patients and 40–80 practices, with at least 6–17 patients per practice (depending on the number of practices). For trials designed to test the effects of interactions between person-level characteristics and the treatment effect (as in research need #11 on dual eligibles), the required sample sizes may be larger than those reported above. The study team recommends that for any new trial being contemplated, the investigators conduct a separate, detailed power analysis specific to the proposed study.

Conclusions

This pilot engaged stakeholders, developed a process for prioritizing future research, and developed a framework for considering the most feasible study designs to employ for each identified research need. We identified 13 potential future research projects in the area of integrated care and developed study design options within the PICOTS framework. Because many of the future research need topics can be addressed through use of more than one study design, many more than 13 studies could be derived from this priority list. We have illustrated the advantages and disadvantages of approaches in the sections on study design considerations.

Our findings suggest that future research in the area of integrating mental health and primary care should first focus on (a) identifying methods of integrating primary care into specialty mental health settings; (b) identifying cross-cutting strategies for integration across multiple mental health diagnostic categories as opposed to a separate strategy for each diagnostic category; and (c) examining the use of information technology for integrating mental and general medical health care. Other important priorities for future consideration include examining the sustainability of successful integration models, identifying methods of disseminating integration programs into various settings, examining the business case for integration as well as methods of payment, assessing the cost-effectiveness of integration, and identifying key components of successful strategies. The importance of sustainability and economic justification for integrated care strategies was a theme throughout the prioritization process and in conference calls with the stakeholders. The literature and the stakeholders indicated that the efficacy of integrated care has been established; however, its ability to be sustained in everyday practice remains to be proved, and will in part depend on the level of incentives and support provided through payment system reform, as well as the ability of the practices to provide the care efficiently.

Although our final list of 13 research need statements was substantially reduced from the original list, when we mapped the research needs back to the 2008 analytic framework, the needs posed related to a range of elements of integrated care in mental health as described in the 2008 AHRQ report including screening, integration of providers, formation of teams, followup, and patient issues.

Our examination of sample size as one of the key study design considerations was useful. Some of the research gaps could be addressed relatively inexpensively through secondary data analysis or through case studies of 8–16 practices. Other questions, however, will require prospective data collection and large sample sizes. The need for very large sample sizes necessitates consideration as to whether the importance of the question is worth the large investment in research funding and effort.

Challenges included focusing stakeholders on the specific gaps and coming to agreement on an initial list of gaps. Stakeholders tended to redefine the questions asked in the report or sometimes posed questions that were outside the scope of the original systematic review. Stakeholders tended to define some gaps very broadly and others narrowly. This led to variability in the scope of the initial drafts of the future research needs. The decision to split or group research needs required much discussion. We felt that a relatively specific description of research needs was best; broad research aims or goals may be difficult to operationalize in the PICOTS framework. Nesting specific needs under broad headings may be an alternative approach but this presents technical challenges for online prioritization, since stakeholders may interpret such nested lists differently. At the end of the process, we still had a large number of identified priorities. Given this large number of potential projects, we did not develop any “value of information” analyses; this would have substantially lengthened the project and probably not have provided substantial new information.

The level of public and policy interest in this area will necessitate continued relatively rapid translation of these research gaps into studies and then to policy implementation.

PubReader format: click here to try

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (856K)
  • Disable Glossary Links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...