NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Institute of Medicine (US); Grossmann C, Powers B, McGinnis JM, editors. Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care: Workshop Series Summary. Washington (DC): National Academies Press (US); 2011.

Cover of Digital Infrastructure for the Learning Health System

Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care: Workshop Series Summary.

Show details

10Accelerating Progress


Throughout the workshop discussions—and most prominently at the third meeting in the series—participants identified several specific, cross-cutting action targets as priority elements for future work. These activities were presented as actionable next steps necessary to accelerate progress on the issues and domains outlined in Chapter 9. This chapter begins by presenting 10 priority action targets (summarized in Box 10-1) that were most often cited throughout discussions. These activities represent participants’ views on the necessary next steps to accelerate progress in four domains: stakeholder engagement, technical progress, infrastructure use, and governance. When discussing necessary follow-up activities, participants continually referenced the potential held by the next stages of the meaningful use guidelines for growing the digital health infrastructure. Participants’ views on key possibilities to be considered when developing and releasing stage 2 and 3 guidelines are summarized in Box 10-2 and elaborated on in this chapter. Finally, due to the cross-cutting nature of the priority action targets identified, discussions often focused on delineating specific stakeholder responsibilities and opportunities for action. This chapter concludes with a summary of participant views on the near-term steps that private and public stakeholders can take to accelerate progress on the follow-up areas identified.

Box Icon

BOX 10-1

Priority Action Targets Discussed. The case: Analyses to assess the potential returns on health and economic dimensions Involvement: Initiative on citizens, patients, and clinicians as active learning stakeholders

Box Icon

BOX 10-2

Meaningful Use and the Digital Learning Health System Infrastructure. Stage 1 of meaningful use established 14–15 (eligible hospitals or eligible professionals) required core functional components, focused on data capture and sharing, along with (more...)


The case: Analyses to assess the potential returns on health and economic dimensions. Because of the centrality of broad-based support to progress, and the “public good” nature of many of the activities, the need to demonstrate a value proposition or business case for participation by stakeholders in a digital learning health system was a topic of much discussion during the workshop series. This emphasis was reinforced by the approach taken by the President’s Council on Science and Technology report to encourage the development of a market around digital health information exchange. Support of methods that apply serious analytical rigor to these issues and generate both technical and policy suggestions were identified as being crucial to this effort. Researchers and organizations such as think tanks were discussed as likely being the best positioned to undertake the necessary analyses with support of a commissioning resource.

Involvement: Initiative on citizens, patients, and clinicians as active learning stakeholders. Many workshop discussions considered that stakeholder investment to be a necessary component of any successful strategy. Participants identified the need to redefine the roles of citizens, patients, and clinicians in a way that activates their participation in their own health, and the health of the population at large, through the facilitative properties of the digital infrastructure. It was noted that patient and clinician groups can play a crucial role in this effort by helping convey the value proposition and ensuring that the interests of their constituents are represented in the development and evolution of the system. Efforts that facilitate stakeholder participation—such as increased control of health information by patients and the use of patient-generated data in care plans and knowledge generating processes—were discussed as priority next steps in stakeholder engagement. Additionally, to attend to concerns around privacy, security, trust, and additional work burden, participants stressed the importance of honesty and transparency in facilitating support and understanding. Ultimately, discussions noted that demonstrating the value of a digital health infrastructure through the use of case studies that point to improved outcomes and efficiency was likely the most compelling strategy to appeal to stakeholders.


Functionality standards: Consensus on standards for core functionalities—care, quality, public health, and research. Progress on the technical standards necessary to support the core functionalities of the learning health system was continually referenced in workshop discussions. Participants focused on the standards necessary not only to improve, monitor, and guide care decisions but also to accelerate research, quality efforts, patient monitoring, and health surveillance. Related requirements include the ability to exchange information through the use of minimal standards (such as those to enable use of metadata-tagged information packets), query and analyze distributed repositories of data for research purposes, ensure care decision support, and enable quality improvement initiatives and public health surveillance and reporting. Discussions also touched on the need for the digital infrastructure to interface with next-generation systems including mobile health applications and the way in which these and other capacities could help engage patients and the public through improved information access. Participants also underscored the strategic importance of adhering to a minimal set of standards that support core functions but do not introduce unnecessary barriers to progress.

Interoperability: Stakeholder vehicle to accelerate exchange and interoperability specifications. System interoperability remains a major obstacle to realizing a digital learning health system. When applying the ultra-large-scale (ULS) system lens to this challenge, participants stressed the need to develop a parsimonious set of standards—such as those for metadata—to allow for practical interoperability and information exchange across systems. Noting that this issue lies in the realm of both technical capacity and governance structure, participants often compared this effort to the evolution and governance of the Internet. While the differences between the digital health infrastructure and the Internet were acknowledged, it was suggested that the establishment and work of the Internet Engineering Task Force might provide guidance for an industrial institution for the governance of interoperability-related standards. Additionally, leveraging and coordinating existing progress and ongoing efforts in the areas of standards development and facilitation were underscored as strategies to ensure activities progress as efficiently as possible.

ULS system test bed: Identify opportunities, implications, and test beds for ULS system approach. As discussions focused on the characterization of the health system as a complex sociotechnical ecosystem, analysis was suggested on how the ULS approach might be applied to the health system in both the short and long term. Mapping of a key ULS system report (Northrop et al., 2006) to the learning health system through a collaborative effort between software engineers, computer scientists, medical informaticians, and clinicians was offered as a starting point for this effort. Furthermore, performing a rigorous engineering systems analysis leading to a concept paper was suggested to clarify further the opportunities and implications for the ULS system approach. Integral to the ULS approach is the need to support rapid prototyping for continuous innovation. It was suggested that test beds for the development, assessment, and dissemination of these prototypes would be central to continual innovation. In this vein, several participants pointed to the opportunity presented by the creation of the Center for Medicare & Medicaid Innovation (CMMI). Certain communities of excellence already provide some capacity in this area, and participants often referenced ongoing activities at these institutions (see Appendix B).

Technical acceleration: Collaborative vehicle for computational scientists and HIT community. Much of the work in the development of a digital learning health system will necessitate interdisciplinary collaboration between academic, public, and private partners across the computer science, HIT, science, and engineering communities. Participants suggested establishing a collaborative forum where these efforts can be initiated and developed. This forum could catalyze the interdisciplinary research program necessary to develop the digital health infrastructure, and some participants suggested that funding for such a forum and its associated activities might best be served by collaborative efforts across relevant federal agencies (such as the National Institutes of Health (NIH) and the National Science Foundation (NSF)), relevant private sector partners, or both.


Quality measures: Consensus on embedded outcome-focused quality measures. Participants noted that the first step in determining the usefulness of data collected by the digital health infrastructure is to identify the necessary elements to collect. It was stated several times that in order to support the quality improvement and research activities required for a learning system, consensus around useful outcome-based measures is needed. Participants suggested that this would motivate vendors and users to incorporate these measures into their systems, driving seamless integration of quality measurement and reporting into the digital infrastructure. Work at the National Quality Forum, through the Office of the National Coordinator for Health Information Technology (ONC) HIT Policy Committee, and at the Centers for Medicare & Medicaid Services (CMS) has already begun addressing these needs.

Clinical research: Cooperative network to advance distributed research capacity and core measures. Discussions often highlighted the centrality of ongoing and continuous generation of knowledge from clinical data as a central feature of the learning health system. Efforts to do research on data held in distributed repositories, such as the HMO Research Network and the Food and Drug Administration’s (FDA’s) Mini-Sentinel program, were pointed to as important early-stage efforts in building systematic, larger scale capacity. Participants suggested that a multidisciplinary, cooperative network of the relevant stakeholders—principally computer scientists, clinical researchers, and data holders—could be a starting point in accelerating progress in this dimension. It was noted that this network would need to consider development of core datasets to facilitate research and quality efforts, fostering consensus on levels of consent and de-identification strategies necessary for effective re-use of data, development of methodologies for query-based and automated research and signal detection across distributed systems, development of standards for distributed queries across the system, implications for a ULS approach to existing and future distributed networks, and implications for distributed research from possible advances in data structure and packaging strategies for data interoperability and exchange across systems.

Identity resolution: Consortium to address patient identification across the system. One of the major barriers discussed for several key system functions—care appropriateness, continuity, quality assessment, and research—relates to the current inability to track and link individual patients with their associated information reliably across the health system. This poses a problem for issues around care coordination, including the goal of being able to make care decisions based on comprehensive health information, as well as the development of a useful knowledge generation engine that can incorporate all relevant information and deliver useful, accurate support. Privacy and system security are paramount, but participants noted that approaches are available to address these issues responsibly and the barrier appears to be one of cultural hesitancy rather than a lack of technical capability. Targeting this issue through a consortium approach was proposed as a way to provide the opportunity for stakeholder representation and engagement in an honest, transparent conversation about the component value issues involved.


Governance and coordination: Determination and implementation of governing principles, priorities, system specifications, and cooperative strategies. Workshop participants articulated the idea that governance principles and priorities for a learning health system will require breaking new ground both organizationally and functionally. Discussions identified the need to improve coordination among key stakeholders to accelerate progress in identifying and sharing lessons, examining commonalities, and exploiting opportunities for efficiencies. It was noted that broad agreement will need to be cooperatively marshaled to attend to principles and priorities that support learning system functionalities such as data integrity, policies for data use, human subjects research issues, and proprietary interests. In addition, discussions highlighted the role of governance in planning for and mitigating system failures, an inevitable occurrence in all systems, but one particularly well tolerated within the ULS system. Such failures would, of course, be opportunities for learning, but are potentially alarming in the context of health- and healthcare-associated information. An interdisciplinary consortium of computer scientists and health infomaticians, such as the one mentioned above, was suggested as a suitable place to engage this issue on a technical level. However, addressing system failures in the health system also has a deeply sociocultural component for which approaches that emphasize honesty and transparency with patients and the public were suggested. Education and outreach about this issue were identified as being crucial in preventing irreparable tears in the trust fabric necessary to support a digital learning health system. In this respect, participants noted the important contributions and potential of the HIT Policy Committee’s Governance Working Group. Discussions also underscored the potential advantages of establishing a novel nongovernmental or public–private venture to foster the necessary governance capacity in this country and to work with similar efforts internationally.


In line with these priorities, discussions often focused on the ongoing meaningful use requirement development process. Workshop participants discussed the “beyond meaningful use” issue as key to increasing the utility of digitally embedded clinical records in a learning health system. Specifically, since meaningful use is now such a well-established benchmark process, elements of particular importance to the development of a learning health system might not otherwise be addressed in the meaningful use process if they are not called out for explicit attention in the upcoming stages. Depicted in Box 10-2 is a brief description of the meaningful use stages, the current expected focus of the requirements for stages 2 and 3, and bullet points highlighting some key possibilities proposed by workshop participants.

Stage 2. Items that workshop participants felt were of particular importance in enhancing the impact that stage 2 of meaningful use could have on the progress of the digital learning health system cut across several dimensions. Flagged as especially key were actions to accelerate standards for semantic interoperability and exchange, as well as approaches for consistent identification of patients. In order to further the utility of EHRs in clinical research and population health, participants suggested core data elements for EHRs and seamless access to information from immunization registries. Reflecting the extensive discussion on the opportunity for using the digital infrastructure to better engage patients in their health care, participants suggested the addition of lay-interpretable language for patient-accessible information and incorporation of patient-generated data. Finally, discussions emphasized the need for clinical decision support to be seamlessly integrated into HIT systems to speed adoption.

Stage 3. Looking ahead to stage 3 of meaningful use, workshop participants suggested deepening the focus on requirements related to demonstrating semantic interoperability and exchange capacity among systems, the ability to access comprehensive patient records at the point of care, and seamless exchange of cases and alerts between clinicians and public health agencies. Additionally, participants suggested strategies for including additional types of data—including nonmedical, health-related data—as well as providing patients with an annotated editing ability over their own records.


Throughout each workshop, frequent reference was made to leadership responsibilities that fell naturally to individual stakeholders, or groups of stakeholders, to advance progress in developing the digital infrastructure for the learning health system. In many cases, this involved leveraging ongoing efforts or building upon them with an orientation toward a continuous learning system. Summarized below are some of those most often noted. These responsibilities are summarized in Appendix C.

Federal Government

Even though participants noted the decentralized manner in which localized innovation is likely to contribute to system progress, many of the central strategy elements and priority action targets discussed require strong leadership from federal agencies. Since a clear lead responsibility was given to ONC and the Secretary of the Department of Health and Human Services by the Health Information Technology for Economic and Clinical Health (HITECH) Act, ONC was noted as the natural leadership locus for activities needing coordination at the national level. Opportunities to build on the foundation laid by the HITECH requirements for work on standards, requirements, and certification criterion in meaningful use of EHRs include cooperation with other federal agencies in the development of a strategic plan for national HIT efforts; establishment of a governance mechanism for the Nationwide Health Information Network; accelerating, in cooperation with the National Institute for Standards and Technology, work on standards for exchange and interoperability; and work with the Federal Communications Commission, FDA, and CMS to identify standards and reconcile regulations to facilitate wireless transmission of medical information. Participants noted that, as the HITECH funds are used, the coordinating capacity of ONC will take on even greater importance, as coalitions will be needed to harmonize various key activities geared at developing the standards, policies, governance, and research projects necessary for effective progress toward a learning health system.

With respect to technical innovation, as the leading federal agency for funding computer science and engineering research, the NSF was noted as a logical locus to work with ONC and NIH in the development of test beds for the rapid deployment and evaluation of innovative technological approaches. This work would have the potential to transform the functionality and capacity of the digital health infrastructure, as well as to shepherd the establishment of collaborative vehicles for the ongoing partnerships between the HIT and computational science communities.

Similarly, it was noted that progress in the quality and knowledge generation dimensions of the digital platform will require leadership from federal health agencies. The Agency for Healthcare Research and Quality (AHRQ), working with ONC, professional societies, and groups such as the National Quality Forum and the National Committee for Quality Assurance, is a natural steward for initiatives that enhance the utility of the digital infrastructure for quality improvement and health services research.

The Centers for Disease Control and Prevention’s (CDC’s) focus on population health places it at the center of extending the scope of the digital infrastructure beyond health care. This carries implications for almost all elements of the system, but will be especially important for the support of public health processes and research as well as public engagement. To these ends, participants suggested developing templates and protocols for the integration of nonmedical population health and demographic information into the system.

As the nation’s largest healthcare financing organization, CMS currently serves as the principal vehicle for applying economic incentives and standards to accelerate application of the meaningful use requirements. Furthermore, much promise for future innovation in HIT to support a learning system resides in the CMMI, which provides an opportunity for testing innovative approaches suggested by workshop participants. These approaches include test beds for ULS-associated programs and new approaches to integrating clinical decision support with care coordination and delivery models.

On the research front, both NIH and NSF have mandates and networks to develop and demonstrate methods of improving the functionality of the digital infrastructure for health research applications. NIH, the Veterans Health Administration, the Department of Defense, FDA, and AHRQ all have active programs under way that can evolve into cooperative leadership efforts to expand the use of EHRs for research into the clinical effectiveness of health interventions.

To build support and engagement among patients and the general population, AHRQ, FDA, NIH, and ONC each has established links to patient communities that can serve as the building blocks for a collaborative initiative to better characterize and communicate the health and economic advantages of public involvement in a digital platform for health improvement.

Given this level of activity, and the number of central stakeholders, the importance of ONC’s coordination mandate was often underscored. Similarly emphasized was the need to cultivate strong counterpart capacity outside of government to partner in coordination and governance responsibilities.

State and Local Government Leadership

Given the regional emphasis of many of the ongoing efforts related to the digital learning health system—such as the establishment of regional health information exchanges—state and local governments and health departments have experience establishing governance structures and developing programs for engaging local stakeholders. As a result, participants noted, state and local bodies can function as resources and foundation stones for broader efforts. By collaborating with ONC, CMS, the Health Resources and Services Administration, and other federal initiatives, best practices and lessons learned can be leveraged from state and local efforts. Additionally, it was suggested that some of the more advanced local initiatives could serve as test beds for some of the innovative ULS-associated approaches suggested by participants.

Initiatives Outside Government

Outside of government, the entrepreneurial capacity of the commercial sector will certainly be a major driver of progress. Similarly, the full potential of the learning health system can only be achieved through the full engagement of patients and the public. Workshop discussants frequently underscored the roles of patient and clinician groups to facilitate dialogue between stakeholders and mediate public engagement. In particular, by using case studies to demonstrate the value of the digital infrastructure, participants felt these organizations could help develop the shared learning culture and trust necessary for the learning system to function. Many patient and clinician groups—such as the American College of Physicians, the American College of Cardiology, the Society of Thoracic Surgeons, and the National Partnership for Women and Families—are already involved in this type of work. Participants noted that these existing activities could be built upon to include issues of particular importance to the learning system approach.

Delivery systems, particularly those integrated across healthcare components, have been at the cutting edge of innovative EHR use, quality improvement, clinical data stewardship, patient engagement, quality initiatives, and distributed research efforts. Workshop conversations often pointed to these efforts, such as those at Kaiser Permanente and Geisinger Health System, suggesting that continued coordination between these delivery systems and relevant federal government agencies would be important in growing the digital health infrastructure.

As the stewards of the largest stores of clinical and transactional information outside of the federal government, insurers, payers, and product developers have an essential role to play in development of the digital infrastructure. Their use of transactional health data to assess utilization patterns, effectiveness, and efficiency is a foundational block on which strategies for broader knowledge generation can build. Furthermore, companies such as UnitedHealthcare have begun engaging the public in the use of data in health. These efforts often were cited during discussions as crucial first steps in establishing a learning culture.

Research is a fundamental aspect of the learning health system. Consequently, participants noted the fundamental role researchers have in developing the infrastructure necessary for continuous knowledge generation and application. Formation of multidisciplinary research communities was often cited as a critical step in accelerating many of the strategies discussed. Funding for these communities was noted as a clear opportunity for collaboration between NSF and NIH. Additionally, discussions highlighted that much work remains to be done in order to maximize the knowledge generation capabilities of the digital infrastructure, and that clinical research and product development communities have an essential role in building this capacity.

As much of the progress to date is a result of initiatives from many independent organizations, their continued efforts as facilitators and innovators were noted as crucial to accelerating progress. Reference was often made to the importance of these organizations as the foundational elements for coordination and governance leadership from outside government.

Finally, and ultimately of paramount importance, is the global perspective. As highlighted during workshop discussions and presentations (see Chapter 8), meeting the goals of a learning health system will inevitably require drawing upon resources and leadership of similar efforts throughout the world. Some of this activity has begun in the limited arena of infectious disease surveillance and monitoring, and offers a hint of the potential opportunities—and challenges—in developing a truly global clinical data utility for health progress.


  • Northrop L, Feiler PH, Pollak B, Pipitone D. Ultra-large-scale systems: The software challenge of the future. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University; 2006.
Copyright © 2011, National Academy of Sciences.
Bookshelf ID: NBK83573
PubReader format: click here to try


  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (2.7M)

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...