NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Institute of Medicine (US) Roundtable on Translating Genomic-Based Research for Health. Systems for Research and Evaluation for Translating Genome-Based Discoveries for Health: Workshop Summary. Washington (DC): National Academies Press (US); 2009.

Cover of Systems for Research and Evaluation for Translating Genome-Based Discoveries for Health

Systems for Research and Evaluation for Translating Genome-Based Discoveries for Health: Workshop Summary.

Show details

3Creating Evidence Systems

For the first panel session, speakers were asked to address four questions: (1) What are your goals for genetic research? (2) How do you decide what studies to pursue? (3) What barriers did you overcome, or do you still face, in your research? (4) What are the greatest challenges for translation of genomics research going forward?


Robert Davis, M.D., M.P.H.

Center for Health Research Southeast, Kaiser Permanente Georgia

The HMO Research Network (HMORN) is a consortium of 15 health maintenance organizations (HMOs) that collectively cover about 11 to 15 million health plan members. The goal of the network is to facilitate collaborative research aimed at improving health and health care. To that end, the Network recently formed a Pharmacogenomics Special Interest Group. Davis noted that over the past 10 years, there has been an emerging consensus on what the important issues are related to genetic testing and pharmacogenomics. One key issue is the concept of clinical utility. By the time a gene-based test is evaluated, the issues of clinical validity have generally been addressed, but not necessarily clinical utility. Clinical utility, Davis said, really means clinical outcomes. Davis cited several publications that discuss how to assess the impact of pharmacogenomics and evaluate the benefit and risk of new genome-based technology (Burke and Zimmern, 2004; Califf, 2004; Davis and Khoury, 2006; Grosse and Khoury, 2006; Khoury et al., 2008; Phillips, 2006).

An evidence-based framework to evaluate the clinical utility of new genetic tests and treatments is lacking in the current health care infrastructure. The goal of genome-based research is personalized delivery of therapeutics that account for the genetic variation of the patient. This is a long-term new direction in medicine that, Davis said, will play out over many years. Researchers have just begun to see how complicated the genome is. There is much to be learned about the role of polymorphisms, age-dependent changes, methylation, de novo mutations, or gene copies, for example.

Gene-based diagnostic tests are very powerful. They have distinctive risk/benefit profiles, and may have significant unintended effects. Historically, however, genetic tests have been held to a less stringent regulatory standard than pharmacogenetic drugs, which require evidence of improved clinical outcomes to receive Food and Drug Administration approval. Davis stressed that the default for gathering evidence on gene-based diagnostic tests and therapeutics should be a randomized controlled trial (RCT). If an RCT is not feasible, and many times it will not be due to lack of financial and human resources, then population-based observational studies should be conducted.

HMOs, such as Kaiser, evaluate new genetic technologies in similar fashion to what has been done previously for other types of technologies. The first step is to determine if there is good evidence, either from RCTs or observational data, that the technology improves outcomes. Based on a review of the evidence, for example, HMOs are now conducting gene testing for HER-2/neu status of breast cancer tumors. However, a decision about whether to conduct gene testing for polymorphisms involved in the metabolism of the anticoagulant warfarin is still under consideration, pending the results of an ongoing RCT. The second step is to determine whether the new technology improves outcomes in a cost-effective manner. There are no set criteria for what reasonable cost is, and cost is considered relative not only to money, but also to resources and time. An example of a new test that has been determined to be cost effective is the screening test for the presence of the HLA-B*5701 allele that has been shown to be associated with hypersensitivity to the antiretroviral drug abacavir. The results of an RCT (Mallal et al., 2008) showed that HLA-B*5701 screening had a negative predictive value of 100 percent, and a positive predictive value of 47.9 percent, and estimated that 1 out of every 25 to 30 Caucasians will be hypersensitive to abacavir, leading Kaiser to conclude that this test would be cost effective.

Collaborative Studies

The lack of data to support integrating new genetic tests and technologies into practice is a major challenge. In gathering this evidence, HMORN, like many research organizations, is primarily opportunistic. HMORN has formed joint informal collaborations with the Pharmacogenomic Research Network (PGRN), which is funded through the National Institute of General Medical Sciences, and with the Agency for Healthcare Research and Quality (AHRQ) Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) network. The goal of these collaborations is to bridge the divide between researchers and decision makers, and to collect the evidence needed to inform decisions on whether to adopt a gene-based test into practice. A number of studies are under way to examine genetic variation in response to metformin, statins, and asthma-related drugs (primarily beta agonists and steroids). An informal decision-making process is used to decide which drug classes to study. These drugs were selected for study because substantial morbidity and mortality are associated with diabetes, cardiovascular disease, and respiratory illness, especially in children, and treating these diseases is costly. The studies are feasible because there are a substantial number of exposed patients, and studies large enough to have statistical power can be conducted at a single site. Importantly, recent advances in science have made it possible to study the clinical impact of testing for these genetic polymorphisms in population-based settings.

For nearly 10 years, the PGRN has been focused on discovery of gene polymorphisms that influence the response to certain medications. HMORN is now conducting a case-control study to investigate the role of these gene polymorphisms in predicting response to drugs in routine clinical practice. If an association between polymorphisms and patients who do respond to drugs is found, then genetic status-dependent dosing and medication choice guidelines will need to be developed. To fully understand the impact these treatment decisions have, a randomized trial of gene-directed medication choice and dosing should be conducted. For metformin treatment of diabetes, for example, HMORN is conducting a case-control study of nonresponders to metformin versus responders as the controls. (In this case, metformin may interact with SNPs, or polymorphsisms, to affect the patient’s response to therapy.) If the study reveals a strong association between polymorphism and response, then following assessment of clinical validity, an RCT would be conducted to study a gene-guided choice of metformin or sulfanyureas administered to participants tested for polymorphisms, versus standard of care for the control group. A second example is a case-control study of polymorphisms that influence patient response to asthma medications. Nonresponders to steroids, albuterol, and montelukast are being compared to responders in the control group. Again, if the study reveals a strong association, following validation, an RCT would compare treatment with gene-directed choice of medication based on gene testing results to standard of care.


Davis described several barriers to gathering data for decision making, including the current research infrastructure, inadequate data systems, and mismatched incentives for licensure. First, there is no formal research infrastructure with adequate funding for outcome studies of new genomic technologies. As a result, outcome studies have been “bootstrapped” onto discovery projects, meaning that the HMORN has had to be creative in obtaining the necessary resources to be able to conduct these studies.

Second, data systems are at least one generation behind. Most ICD-9 (International Classification of Diseases, 9th Revision) diagnostic codes and CPT (Current Procedural Terminology) service codes are inadequate to the task of efficiently identifying patients who have had their genetic status tested, and what the test results were. As a result, it is generally not possible to assess whether a genetic test (e.g., HER-2/neu oncotype) is being done appropriately, or whether treatment (Herceptin in the HER-2/neu example) is being used appropriately. The available observational data are inadequate for studies of test effectiveness, in part because the exposure is unknown. Without up-to-date data systems, RCTs of new genetic tests must be conducted instead, but these will be impractical to do in many circumstances.

Finally, Davis said, the decision to integrate a licensed genetic test into practice hinges on the demonstration of clearly improved outcomes in large population-based settings. For some tests (e.g., determining onco-type or predicting variations in warfarin metabolism), RCTs may be feasible and justifiable. For others, however, clinical trials are not feasible. Observational data may suffice, but may only be available post licensure. Regardless, Davis said, funding agencies are unlikely to provide support for evaluation of a commercial product post licensure, and there is no regulatory incentive for companies to conduct RCTs or observational studies post licensure. Without fundamental changes, Davis predicted there will be repeated examples of underuse of potentially valuable technology. He cited the example of the Amplichip CYP450 genotype test to predict phenotypic variation in metabolism of certain drugs. Although clinical validity was studied, clinical utility was not, and many healthcare organizations are not using this technology.

Davis concluded by reiterating that genetic tests, similar to pharmaceutical products, should be required to show proof of clinical utility and improved outcomes as a condition for licensure. That, he said, is “going to require a fundamental sea change in the way we think about genetic tests.”


Sumitra Muralidhar, Ph.D.

Office of Research and Development, Veterans Health Administration

The U.S. Department of Veterans Affairs (VA) administers the largest health care system in the country, with 153 hospitals, 745 community-based outpatient clinics, and 245 veterans’ centers that provide readjustment and mental health counseling to returning veterans. In fiscal year 2007, the VA treated 5.5 million unique patients. The VA uses an electronic medical record system and has a stable patient population, allowing for long-term follow-up. Most VA medical centers are affiliated with academic institutions, and serve as major training hospitals for clinicians. The three main divisions of the VA are the Veterans Benefits Administration, the Veterans Health Administration (VHA), and the National Cemetery Administration. The VHA has two branches, Patient Care Services and the Office of Research and Development (ORD). ORD has four services: (1) the Biomedical Laboratory, (2) Clinical Science, (3) Rehabilitation Research, and (4) Health Service Research. Within clinical science there is a cooperative studies program that launches large-scale, multisite trials within the VA system.

The Genomic Medicine Program

In 2006, the Secretary for the VA formally launched the Genomic Medicine Program to examine the potential of emerging genomic technologies to optimize care for veterans. As a first step, Muralidhar explained, a 13-member Genomic Medicine Program Advisory Committee (GMPAC) was established to help lay the groundwork for the program. (As a federal advisory committee, the GMPAC is subject to the Federal Advisory Committee Act.) Members of the committee come from the public and private sectors and from academia, and include leaders in the fields of genetic research, medical genetics, genomic technology, health information technology, health care delivery policy, and program administration, as well as legal counsel. There is also representation from a Veterans Service Organization.

A primary goal of the Genomic Medicine Program is to try to enroll every veteran who walks into a VA hospital into the program. To succeed in this goal, a new physical and technological infrastructure needed to be built, incorporating health information technology, education for providers and patients, genetic counseling, and workforce development, as well as governance, policy, and ethics. This system would facilitate not only research, but also translation into patient care.


A significant challenge for the program has been that the VA is a very large, operationally decentralized system. Even though there is a centralized electronic medical record system, the VA is divided into 22 regional areas. Each operates independently on its own budget, with variability in infrastructure, operations, and capabilities across the system. Another challenge is the ability to incorporate emerging needs of genetic and genomic information within the existing information technology infrastructure. Keeping up with rapidly evolving genomic technologies is also a challenge. Budget constraints are a concern, and building one program can take resources from another. Ultimately, the program cannot work unless veterans are willing to participate.

Addressing the participation concerns first, in 2007 the VA launched a consultation project to assess veterans’ knowledge and attitudes about genomic medicine. This was facilitated through an interagency agreement with the National Human Genome Research Institute (NHGRI) and conducted under a cooperative agreement by the Genetics and Public Policy Center at Johns Hopkins University. The results of 10 focus groups in 5 locations across the country, and a follow-up survey of 931 participants, revealed overwhelming support among veterans for such a program. About 83 percent responded that the program should be undertaken, 71 percent said they would participate in the program if it was implemented, and 61 percent said they would be willing to go beyond basic participation. Examples included coming back for follow-up exams over time or allowing their medical records from non-VA health care to be added to the system (Kaufman et al., 2009). Interestingly, Muralidhar said, individual willingness to participate was associated with attitudes about research in general, attitudes about helping others and having a history of previous altruistic behavior, curiosity about genetics, and general satisfaction with the health care they were receiving at the VA.

Infrastructure Development

After assessing veterans’ willingness to participate, the next steps were to determine what was available within the VA system; if the program should build in-house capability within the VA, or leverage infrastructure available at the affiliated universities or through contracts with industry, or some of each; and what the research agenda should be. As described above, the Cooperative Studies Program conducts large multisite clinical trials within the VA system, providing an infrastructure on which the Genomic Medicine Program could be built. Four clinical trials coordinating centers across the country administer the trials: four Epidemiology Research and Informatics Centers, a health economics research center, a pharmacy coordinating center, and a central Institutional Review Board (IRB).

In addition, for the past 10 years or so, the VA has been banking samples from its clinical trials. A biorepository in Boston has about 30,000 blood samples and 6,000 DNA samples collected from various trials, and a capacity to bank 100,000 samples. The VA also has a DNA Coordinating Center in Palo Alto that links to the clinical information and patient data, and a tissue repository in Tucson that has a brain collection from amyotrophic lateral sclerosis (ALS) patients and tissue blocks. In 2008, the VA established a Pharmacogenomics Analysis Laboratory in Little Rock, which is now a Clinical Laboratory Improvements Amendments- (CLIA-) certified research genomics laboratory conducting large-scale genotyping. There is also a newly established Genomics Research Core at the VA medical center in San Antonio.

The information technology (IT) infrastructure also needed to be addressed. The VA has recently funded two IT projects, the Genomic Information System for Integrative Science (GenISIS) and the Veterans Informatics Information and Computing Infrastructure (VINCI). The GenISIS system is based in Boston along with the biorepository, the Clinical Trials Coordinating Center, and the Epidemiology Research and Informatics Center. Historically, research data, biological data, clinical data, and medical records have resided in separate compartments. Research is traditionally geared toward hypothesis testing, there is targeted data collection from individual studies, the data are used by a single “owner,” and the work is discipline driven. In contrast, the goal of GenISIS is to move toward a comprehensive data collection and retention system that facilitates hypothesis generation, data analysis, repurposing or reuse of data, and interdisciplinary interaction (Figure 3-1). GenISIS allows for secure gathering, integration, and analysis of patient information; discovery research through shared expertise; repurposing of data for secondary analysis; validation of genomic medicine findings; and integration of those findings into clinical medicine. Thus, the short-term goal for GenISIS is to create and support a knowledge base that would facilitate independent research projects and collaborative repurposing of data. The vision for GenISIS for the longer term is focused on patient care, integrating clinical care and research activities for improved patient outcomes. The objective of VINCI is to integrate existing databases across the VA and create a secure, high-performance computing environment for researchers to access data.

FIGURE 3-1. Integration of the components of the GenISIS system.


Integration of the components of the GenISIS system. SOURCE: Muralidhar, 2009.

Research Agenda

The VA research agenda is informed by the health care needs of veterans and, Muralidhar said, that approach would apply for genomics as well. The GMPAC meets three times each year and advises the VA on the various emerging technologies and tests that are available to move into the clinic. There are specific scientific advisory and working groups, such as groups focused on hereditary nonpolyposis colorectal cancer or endocrine tumors, that make recommendations on algorithms that the VA could use for screening and testing. There is also investigator-initiated research.

Genomics research projects include: a genome-wide associate study of ALS, using the VA registry containing more than 2,000 ALS patients; a study of the genetics of posttraumatic stress disorder (PTSD) and co-morbidities, including 5,000 returning Operation Iraqi Freedom and Operation Enduring Freedom veterans with PTSD; and a serious mental illness cohort, with plans under review to recruit 9,000 patients with schizophrenia and 9,000 with bipolar disorder and a 20,000-reference cohort. Future research areas of interest to the VA include diabetes and pharmacogenomics. The VA also funds investigator-initiated projects focused on the genetics and genomics of chronic diseases.

Moving Forward

The biggest challenge going forward, Muralidhar said, is launching an integrated system to facilitate genomics research, as well as translation of that research to clinical care of veterans, in a system as large as the VA. The VA must also develop governance and policy for various issues, such as access to samples and data. Interoperability with external health systems will also be a challenge. Many veterans who obtain health care at the VA obtain all their care primarily from the VA, but some veterans also receive care from outside the system, and it will be important for the VA to consider those data as well.

Several education initiatives are under way, including working with the National Coalition for Health Professional Education in Genetics to implement a web-based tool to provide continuing medical education accreditation and point-of-care materials for clinicians and other health professionals. The VA also interacts, discusses, and actively participates with various other genetics/genomics-focused organizations, including NHGRI, PGRN, the American Health Information Community, the federal working group on family history tool development, and the Institute of Medicine (IOM) Roundtable on Translating Genomic-Based Research for Health.


Marc S. Williams, M.D., F.A.A.P., F.A.C.M.G.

Intermountain Healthcare Clinical Genetics Institute

In the late 1800s, the Church of Jesus Christ of Latter-Day Saints (LDS) began opening hospitals and creating a health care system in the southwestern United States. In 1975, the church sold all of its health care properties to Intermountain Healthcare, a secular, not-for-profit entity. With more than 20 hospitals and more than 1,000 directly employed physicians caring for more than 1 million patients from Utah and southern Idaho every year, Intermountain Healthcare is now the largest health care system in Utah. It is also the only integrated health system in Utah, incorporating an insurance plan, outpatient and inpatient care, home care, pharmacy, hospice, and other services under one administrative roof.

Research Priorities

Intermountain Healthcare has been involved in research for quite some time. Intermountain began research into informatics in health care in the late 1950s. The Institute for Healthcare Delivery Research was established in 1986, focused on quality improvement in health care delivery. An academic medical faculty was established in the 1960s, providing for protected time to pursue academic activities even though Intermountain is not affiliated with an academic institution. There is also modest internal funding for research and programs through Intermountain’s Deseret Foundation.

Despite the long history of research at Intermountain, there was no overall vision for research until about 2 years ago, Williams said. The recently developed research mission statement calls for “excellence in clinical and translational research resulting in improved clinical care within the Intermountain Healthcare system.” The vision for research at Inter-mountain is to improve patient care and well-being for many; encourage expertise; effectively communicate accomplishments; be financially responsible; and ensure that research is effectively resourced, optimally efficient, and complies with all applicable rules and regulations. Research priorities include retaining focus in areas of traditional strengths (e.g., cardiovascular, pulmonary/critical care, and informatics); supporting clinicians who have good research ideas, regardless of therapeutic area; using research to better support clinical program goals and objectives; and establishing genetics and genomics as a research strength across all specialties.

The rationale for including genomics as a research priority, Williams said, was that genomics will impact care across many clinical areas in the future. Also, Intermountain’s information system positions the organization to be able to make important contributions to research in genomics. But, Williams noted, Intermountain recognizes that it cannot succeed alone. Intermountain needs to combine its unique assets with partners in the academic, commercial, and public health sectors. In this regard, Intermountain recently completed a master research agreement with the University of Utah. The VINCI program described by Muralidhar involves the bioinformatics faculty at the University of Utah, many of whom are Intermountain Healthcare employees.

Genomics Research

Genomics research at Intermountain is ongoing within existing specialty areas. Cardiovascular medicine, for example, has a biorepository of more than 16,000 samples obtained at the time of catheterization, and has created a genealogy resource modeled after the Utah Population Database. This allows them to construct a genealogy for a given patient, look for other members of that family with similar diagnoses of interest, and conduct targeted recruiting of participants for discovery studies. Cardiovascular medicine also has a small molecular laboratory dedicated to genome discovery research. The group has conducted pharmacogenomics-based research, such as a prospective controlled trial looking at pharmacogenomic dosing for warfarin (Anderson et al., 2007). In pulmonary/critical care, there has been a lot of interest in primary pulmonary hypertension associated with the BMPR2 gene, and in maternal–fetal medicine, there are ongoing studies of genetic factors for premature birth, in partnership with the University of Utah.

To establish the Clinical Genetics Institute, thought leaders at Inter-mountain convinced the overall leadership that if genetic medicine was not done properly, there would be a significant risk to the system. They proposed that a central core of experts working across the entire system be established. Strategic planning commenced in 2002, hiring began in 2004, and the Institute began operations in January 2005. The primary objective of the Institute is to move evidence-based genetic medicine into clinical practice. Meeting this objective will require novel mechanisms, Williams said, and the Institute is leveraging expertise in informatics and health care delivery research as it moves forward with implementation. The Institute is also committed to working with providers to understand their needs and workflow.

Research efforts focus on the ability to define and measure outcomes of interventions. The institute will communicate research results to a broad audience, and hopes to build processes that will work not only at Inter-mountain, but could potentially be disseminated to other organizations.

Although there are currently only three staff at the Clinical Genetics Institute, their range of expertise spans genetics, health care delivery, quality improvement, informatics, and technology assessment. There is a clear internal vision of program goals, and strong support from some individuals in the larger system. On the negative side, the Institute has no discretionary resources beyond its personnel; large capital projects within the organization are decreasing the resource pool for all researchers across the system; and as noted earlier, there has been no shared institutional vision until recently.

Because of the limited availability of resources, a key component of the Institute’s research strategy is partnerships. The Institute seeks to identify quick wins and targets of opportunity. Research is aligned with clinical efforts wherever possible, and methods are consistent with the Intermountain core values.

Current Research Activities

Williams highlighted several recent and ongoing genome-based research activities at Intermountain. One effort involved developing a rapid ACCE1 model for technology assessment of emerging genomic tests, reducing the assessment time from 12 to 18 months following the standard ACCE structure, to several months using the rapid protocol (Gudgeon et al., 2007). Family history is another area of interest, Williams said, because it captures data that genomics cannot, such as shared environment and exposures. There are no published papers, he noted, on how primary care physicians use the family history data they collect. As a result, Intermountain is preparing a paper on this topic. There is also a family history tool for the patient portal in development, and Intermountain will study how best to move information from a patient portal environment (which would be somewhat analogous to a personal health record) across the firewall into the electronic health record.

Another topic of research is the economics of genetic services. The pharmacogenomic warfarin dosing study described earlier also collected actual cost data from all of the patients randomized into the trial. Epidemiologic research is also under way using Intermountain clinical data, in combination with the Utah Population Database and the National Children’s Study.

Several informatics research projects are under way. Intermountain has created point-of-care education resources in its electronic health record, allowing care providers to click on an information button and link directly to genetics reference information for the patient’s condition, including gene testing. As discussed by Davis above, current coding systems are inadequate in terms of genetics, and Intermountain is working to develop an appropriate infrastructure for coding and messaging of cytogenetic results.

Intermountain also has a partnership with researchers at Harvard to study electronic communication of genetic test results. Intermountain is also conducting health services research, looking at, for example, patient satisfaction with traditional clinical genetic services, identification of genetic diseases using the Clinical Data Repository, and implementation of a tumor-based screening for Lynch syndrome.


From an internal perspective, developing a unified vision of genomic research has been a primary task. Different research entities within Inter-mountain are at varied levels of maturity regarding genetics and genomics. Adequate resources are a significant issue, including not just funding, but also personnel and laboratory facilities. Identification and establishment of equitable partnerships between Intermountain and other outside entities is challenging. There is also a tension between Intermountain’s primary mission of clinical care and the relevance of research to that mission.

Externally, the vision and funding of translational research remains a challenge. Less than 3 percent of federal dollars are allocated to research that is beyond basic discovery. As a nontraditional research environment, Intermountain faces extra challenges in the competition for awards. Inter-mountain is working to define the role of health care delivery research, which is more of a “real-world” scenario, versus a tightly controlled, hypothesis-based research model. One criticism that Intermountain has received is that, due to the unique resources available at Intermountain, results of its research may not translate to other institutions or systems. The current environment, including health care delivery and reform efforts and economics, impacts Intermountain’s initiatives as well.

The Future

Williams closed noting that he sees several reasons to be optimistic about the future. The recent Bush administration had an interest in personalized medicine and the implementation of electronic health records, and this focus appears likely to continue under the Obama administration. Funds are now available through the Centers for Disease Control and Prevention National Office of Public Health Genomics and AHRQ to support health services research that aligns with the Intermountain strategy. There is also the potential that more traditional sources of funding, such as the National Institutes of Health (NIH), will shift toward real-world clinical applications of genomics research. Clinical Translational Science Awards at the University of Utah emphasize partnerships between academic medical centers and private entities, and there is more interest in general about public–private partnerships to broker information.


Wylie Burke, M.D., Ph.D.


Transforming Genomics: Perceptions and Practices

Burke opened the discussion session by asking the panelists to comment on the phrase “sea change,” as Davis said in his presentation there is the need for “a sea change in the way we think about genetic tests.”

Davis responded that three sea changes could be very helpful. The first, and perhaps most important, he said, relates to how new technology is evaluated. While it is inconceivable that a drug would come to market based on clinical validity, that is what happens for technologies such as MammoPrint and AmpliChip. When technology products are released, Davis said, studies of how they impact health outcomes should be conducted. The federal government is hesitant to fund outcome studies of technologies that have been developed by industry because they could potentially be used for marketing. A second sea change involves IRBs, which, much like clinical data systems, are a generation behind. IRBs still hold the opinion that patients don’t want personalized medicine, that it is very risky, and that people are primarily concerned about privacy. Risk and privacy are valid concerns, Davis said, but we need to move away from viewing these studies as extraordinarily high-risk ventures, and think of them as part and parcel of the 21st-century medical enterprise. The third change needed involves funding. Davis cited recent funding announcements for studies of gene–environment interactions that do not pay for any specimen collection, only seeking to fund studies to be done using existing infrastructure or biobanks.

Williams said one thing that needs to change is that insurance companies are the de facto regulators of gene-based medicine. A second issue is that funding favors RCTs, and has not been supportive of real-world clinical trials and health services research. It takes years for something that is known to be effective to be put into practice, and unfortunately, it also takes years for something that is found to be ineffective to be removed from practice (unless there is a lawsuit, in which case removal from clinical practice can occur overnight). The third area where change is needed is coding. He cited a study done on Hereditary Hemorrhagic Telangiectasia (HHT) and juvenile polyposis (Williams and Wood, 2009), and the potential to use the Intermountain Clinical Data Repository to identify patients who may have undiagnosed HHT. Unfortunately, there is only an ICD-9 code for polyps, with no differentiation for an adenomatous polyp or a juvenile polyp. That limitation in coding nearly ended the study, Williams said, but the group was able to capture the information from the pathology system. There also are no specific codes for any genetic tests that are in regular use. Updated coding systems are necessary to be able to mine data from information systems at the level required for genetic studies. Williams also noted that most economic models in use are based on public or national health system implementation, and called for the development of economic analyses that can be done at the level of the health care delivery system.

Muralidhar supported Williams’ point about regulation. She said that at a recent Personalized Medicine Coalition meeting, participants raised the need for a separate agency to evaluate the effectiveness of emerging technologies. She added that a change in education is going to be necessary as well.

Teutsch said the process for insurance coverage is often a one-way stream. Once interventions are covered, “they’re in,” and if coverage is denied, “they’re out.” There is rarely the chance to revisit a coverage decision to determine if the intervention is being used effectively. Changing to a process of incremental implementation would allow for learning along the way. Generally, however, “coverage with evidence development” has only been applied for major, very expensive technologies.

A participant commented that the diagnostic tests used in cardiovascular medicine were adopted decades ago and became the standard of care, and now it is very difficult to study them to see whether they really have an impact on patient outcomes. The same paradigm may be occurring with genomics, he said, but the questions now being asked suggest to him that a sea change in thinking regarding technology assessment is beginning to occur. There is also a sea change occurring regarding attitudes toward funding of biomedical research. The current stimulus package includes an additional $10 billion in funding for NIH over the next 2 years, as well as $1.1 billion for comparative effectiveness research, specifically focusing on technologies already available to clinicians and for which efficacy has not been studied.

Database Issues

A participant asked Williams if the population of Utah is still as genetically homogeneous as it was when used in cohort studies, and how any changes in homogeneity would influence the Intermountain database. Williams responded that a recent study concluded that the heterogeneity within the Caucasian population in Utah is essentially indistinguishable from that of the United States and Northern Europe. African Americans are generally underrepresented in the Utah population, but Utah is not completely homogeneous. There has been an increase in the Hispanic population. Utah also has a unique population of South Pacific Islanders, most likely as a result of the LDS Church’s missionary efforts in Samoa, Tonga, and other island locales, and there is a Native American population that is representative of their founding groups within the larger population. From the perspective of the Genomewide Association Studies, however, the current population mixture is not going to be a significant factor.

A question was raised about the basic assumptions underlying the development of infrastructures and systems. A striking discovery, the participant said, is how many of the common polymorphisms associated with diseases identified through Genomewide Association Studies are actually just echoes of a much more detailed private polymorphism mix. Can the infrastructure that is being developed handle assessment of a single mutation in a family causing a disease? In addition, how much does in silico (i.e., computer-simulated) evidence count? How is environmental information going to be incorporated? We are not collecting any of the relevant information on the social environment, the built-in environment, or diet, she said.

Williams said that one way in silico modeling is useful in clinical practice is when an existing genetic test uncovers a “variant of unknown significance.” From an individual counseling perspective, that type of information is extremely helpful. Standardization across testing laboratories for how to address new variants, such as additional tests to be run, and creating databases of mutations would be useful. In silico modeling is also helpful in terms of targeting direction or prioritization. To address environmental influences, Williams reiterated that Intermountain focuses on family history, and already has empiric data about several common diseases.

Teutsch said a major challenge for genomics is determining where to expend resources. The focus of the workshop is how to gather data, but another challenge is how to bridge genomics and personalized health data with public health and population health information. Otherwise, there could be a potentially costly one-on-one clinical approach that deals with individual risks, which may only be modest on a population basis.

Williams continued the point, asking which would have a greater impact on asthma: research on polymorphisms that predict beta agonist response, or environmental research to decrease the amount of particulates in the air? Most would argue that improving air quality would have orders-of-magnitude greater impact. But it is a much harder problem to solve.

The panel was asked how research initiatives would change if, or when, a widely available, affordable human genome with sequence-searching capabilities was available. Williams responded that it would completely change the paradigm of genetic testing. At a given price point, and at a given level of analytic validity, it does not make sense to pay a company thousands of dollars to search a specific genetic test if you could search the whole genome for $1,000, and then build database queries against those particular sequences. It would lower many of the barriers related to sample collection and storage, and enhance access to information. It would, however, raise many questions about who would have access, and under what circumstances.

Medical Education and Practice

One participant commented that applicants to medical schools know how to conduct current technological procedures (e.g., gene splicing), but don’t necessarily know why they are doing it. Williams responded that the percentage of doctors really interested in understanding why they are being told to conduct a specific test is relatively low. They are interested in managing their patients better, and have approached Clinical Decision Support to help them do that. For those who are interested, Intermountain’s Clinical Decision Support System provides the ability to drill down through Intermountain’s clinical guidelines, national clinical guidelines, and the basic literature, simply by successive mouse clicks within the electronic health record.

A participant noted that there may be upcoming revisions to the medical boards, combining parts one and two of the boards into a single exam encompassing both basic and clinical science. The participant said this transition time could be a window of opportunity to insert genetics back into the curriculum.

Another participant said that clinicians are often aware that a test exists and will request it, leaving pathologists caught in the middle between quality oversight and the lack of knowledge about the clinical outcomes of genome-based tests.

A concern was raised by a participant about professional societies promulgating guidelines that he said have no evidence basis. Fifteen years ago, for example, there was a burden of proof required before routine prenatal cystic fibrosis screening was adopted as a guideline. More recently, however, the American College of Medical Genetics recommended the adoption of spinal muscular atrophy screening for all U.S. couples, and he questioned where the feasibility studies were. How many millions of dollars worth of tests will be done before someone accepts the burden of proof and demonstrates whether there is clinical utility or not? Organizations need to make sure that recommendations are evidence based.

Davis added that RCTs simply cannot be done for all of these tests, or even the majority, but that does not preclude evaluation using other data sources. Vaccines, for example, are released and safety in large populations is followed for 5 years. These paradigms could be adopted for evaluating the clinical utility and safety of new genetic technologies.

Williams said professional organizations have a responsibility to scan the horizon, understand what the public is pushing for, and determine at what point they need to intervene. He noted that for newborn screening, there are inconsistencies from state to state regarding which diseases are included in the screen. Williams said that professional societies need not refrain from taking any action until the data reaches a certain evidentiary bar, but they do have a responsibility to be absolutely transparent and explicit in terms of the evidence used to reach a decision.

A participant noted that there are concerns that by the time an outcomes study of a new technology is completed and disseminated, the technology is outdated and newer ones may already be in use.

Williams recalled that when he graduated from medical school, it was estimated that medical knowledge would double every 30 years. The doubling time of medical knowledge is now 7 years and decreasing. The whole continuum of education, from undergraduate, to medical education, to residency training, to practice, needs to be evaluated with an eye toward implementing rapid change as evidence develops.

Williams pointed out that issues surrounding reimbursement were not discussed. Reimbursement follows policies, not necessarily evidence. Barriers created by reimbursement practices are going to have a tremendous impact in terms of moving genetic tests into the clinic, especially if a test is ultimately defined as preventive.

Research Participation

A participant from industry noted that although panelists discussed the need for more RCTs and observational trials, the need for funding for sample collection, and problems in coding, biobanking, and other operational issues, these are the lesser problems from the industry perspective. Industry conducts RCTs and some observational trials, and adding the genetics component to them is a marginal cost. Companies are generally well funded, do not have to rely on the ICD-9 codes per se, and have good sample banking. The biggest obstacle, he said, is patient participation. A company may intend to collect DNA from 100 percent of individuals who participate in a subset of Phase I, and all Phase II-and-beyond clinical trials, but the participation rate is very low, and enrollment is challenging due to the imposition of a variety of obstacles and constraints by IRBs and Ethical Review Boards. A large trial must work across many of these review boards, which have different rules depending on the country in which they operate. What can be done to better facilitate enrollment and encourage patients to participate?

Teutsch noted that the Secretary’s Advisory Committee has this constellation of issues on their agenda, and understands that Health Insurance Portability and Accountability Act (HIPAA) and IRB regulations need to be kept up to date with current ethical and legal needs and standards. The committee plans to consider what could be done with those systems to facilitate research, while still protecting the rights and privileges of the individuals.

A participant drew attention to the recently released IOM report Beyond the HIPAA Privacy Rule: Enhancing Privacy, Improving Health Through Research (IOM, 2009). The committee, she said, called for an entirely new framework to address privacy issues in research. She also noted that the committee offered practical suggestions for changes that could be made based on interpretation of regulations, without necessarily drafting new laws.

Williams commented that many of the issues being discussed involve personal values as well as medical value. Genomic medicine, or personalized medicine, provides a real opportunity to learn from incorporating a shared medical decision-making model, ensuring that providers are not only delivering the best medical care, but providing care that patients highly value.

Data Sharing

An audience member questioned if the data in the various repositories was proprietary, or whether any researcher could, for example, use the VA data. She also wondered if the move towards comparative effectiveness research and electronic medical records would provide an opportunity to better leverage the information across all of these different systems. Could handwritten data in charts and pathology reports be entered into the electronic system, so that it could be used more easily to supplement the claims data?

Williams responded that researchers are welcome to use Intermountain’s data in collaboration with Intermountain researchers. He also noted that in Utah, they have formed a genomic medicine workgroup that includes representatives from Intermountain, the University of Utah, Utah State University, the Salt Lake City VA Hospital, and a number of private groups. The group is in the early stages, but is looking to foster collaboration and find venues to disseminate information. Relating to information systems, he said, a project called FURTHeR (Federated Utah Research Translational Health e-Repository), which is being run out of University of Utah Biomedical Informatics, is examining ways to combine University of Utah health care data, Intermountain Healthcare data, and Salt Lake VA health care data into a larger dataset. The project first needs to address issues such as rules that govern use, deidentification, and security. Another issue is the lack of standardization across systems. Most aspects that are standardized do not relate to the types of information that are needed for genomics. There needs to be investment in the development of standards that can be incorporated into the next-generation information systems.

Muralidhar said that at the VA, the GenISIS and VINCI programs are working to electronically capture data from case report forms and various other handwritten materials. They are also considering ways to give researchers Internet-based access to the VA data.



ACCE is discussed by Teutsch in Chapter 2.

Copyright © 2009, National Academy of Sciences.
Bookshelf ID: NBK32311


  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (882K)

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...