NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Gliklich RE, Dreyer NA, Leavy MB, editors. Registries for Evaluating Patient Outcomes: A User's Guide [Internet]. 3rd edition. Rockville (MD): Agency for Healthcare Research and Quality (US); 2014 Apr.

Cover of Registries for Evaluating Patient Outcomes

Registries for Evaluating Patient Outcomes: A User's Guide [Internet]. 3rd edition.

Show details

1Patient Registries

1. Introduction

The purpose of this document is to serve as a guide for the design and use of patient registries for scientific, clinical, and health policy purposes. Properly designed and executed, patient registries can provide a real-world view of clinical practice, patient outcomes, safety, and comparative effectiveness. This user's guide primarily focuses on practical design and operational issues, evaluation principles, and best practices. Where topics are well covered in other materials, references and/or links are provided. The goal of this document is to provide stakeholders in both the public and private sectors with information they can use to guide the design and implementation of patient registries, the analysis and interpretation of data from patient registries, and the evaluation of the quality of a registry or one of its components. Where useful, case examples have been incorporated to illustrate particular points or challenges.

The term registry1 is defined both as the act of recording or registering and as the record or entry itself. Therefore, “registries” can refer to both programs that collect and store data and the records that are so created.

The term patient registry is generally used to distinguish registries focused on health information from other record sets, but there is no consistent definition in current use. E.M. Brooke, in a 1974 publication of the World Health Organization, further delineated registries in health information systems as “a file of documents containing uniform information about individual persons, collected in a systematic and comprehensive way, in order to serve a predetermined purpose.” 2

The National Committee on Vital and Health Statistics3 describes registries used for a broad range of purposes in public health and medicine as “an organized system for the collection, storage, retrieval, analysis, and dissemination of information on individual persons who have either a particular disease, a condition (e.g., a risk factor) that predisposes [them] to the occurrence of a health-related event, or prior exposure to substances (or circumstances) known or suspected to cause adverse health effects.”

Other terms also used to refer to patient registries include clinical registries, clinical data registries, disease registries, and outcomes registries.4, 5

This user's guide focuses on patient registries that are used for evaluating patient outcomes. It is not intended to address several other types of or uses for registries (although many of the principles may be applicable), such as geographically based population registries (not based on a disease, condition, or exposure); registries created for public health reporting without tracking outcomes (e.g., vaccine registries); or listing registries that are used solely to identify patients with particular diseases in clinical practices but are not used for evaluating outcomes. This user's guide is also not intended to address the wide range of studies that use secondary analyses of data collected for other purposes.

Many of these other types of registries are included in the Registry of Patient Registries (RoPR) effort. RoPR is a central listing of patient registries established in 2012 by the Agency for Healthcare Research and Quality (AHRQ) in collaboration with the National Library of Medicine.6 It is designed to improve transparency and reduce redundancy in registry-based research. Inclusion of all types of patient registries is important to achieve RoPR's goals, and the system therefore defines patient registries broadly.

In contrast to RoPR, this user's guide focuses on the subset of patient registries used for evaluating patient outcomes, defined as follows:

  • A patient registry is an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or policy purposes.
  • The patient registry database describes a file (or files) derived from the registry.

Based on these definitions, the user's guide focuses on patient registries in which the following are true (although exceptions may apply):

  • The data are collected in a naturalistic manner, such that the management of patients is determined by the caregiver and patient together and not by the registry protocol.
  • The registry is designed to fulfill specific purposes, and these purposes are defined before collecting and analyzing the data. In other words, the data collection is purpose driven rather than the purpose being data driven (meaning limited to or derived from what is already available in an existing data set).
  • The registry captures data elements with specific and consistent data definitions.
  • The data are collected in a uniform manner for every patient. This consideration refers to both the types of data and the frequency of their collection.
  • The data collected include data derived from and reflective of the clinical status of the patient (e.g., history, examination, laboratory test, or patient-reported data). Registries include the types of data that clinicians would use for the diagnosis and management of patients.
  • At least one element of registry data collection is active, meaning that some data are collected specifically for the purpose of the registry (usually collected from the patient or clinician) rather than inferred from sources that are collected for another purpose (administrative, billing, pharmacy databases, etc.). This definition does not exclude situations where registry data collection is a specific, but not the exclusive, reason data are being collected, such as might be envisioned with future uses of electronic health records, as described in Chapter 15. This definition also does not exclude the incorporation of other data sources. Registries can be enriched by linkage with extant databases (e.g., to determine deaths and other outcomes or to assess pharmacy use or resource utilization), as discussed in Chapter 6.

Data from patient registries are generally used for studies that address the purpose for which the registry was created. In some respects, such as the collection of detailed clinical and longitudinal followup data, studies derived from the patient registries described in this user's guide resemble traditional observational cohort studies. Beyond traditional cohort studies, however, some registry-based studies may be more flexible in that the scope and focus of the data collection activity of the registry may be adapted over time to address additional needs. For example, new studies, such as cluster-randomized studies or case-control studies, may be nested within an ongoing registry, and the database derived from the registry may be used to support secondary studies, such as studies that link the registry database with other data sources to explore new questions.

2. Current Uses for Patient Registries

A patient registry can be a powerful tool to observe the course of disease; to understand variations in treatment and outcomes; to examine factors that influence prognosis and quality of life; to describe care patterns, including appropriateness of care and disparities in the delivery of care; to assess effectiveness; to monitor safety and harm; and to measure quality of care. Through functionalities such as feedback of data, registries are also being used to study quality improvement.7

Different stakeholders perceive and may benefit from the value of registries in different ways. For example, for a clinician, registries can collect data about disease presentation and outcomes on large numbers of patients rapidly, thereby producing a real-world picture of disease, current treatment practices, and outcomes. For a physician organization, a registry might provide data that can be used to assess the degree to which clinicians are managing a disease in accordance with evidence-based guidelines, to focus attention on specific aspects of a particular disease that might otherwise be overlooked, or to provide data for clinicians to compare themselves with their peers.8 For patients and patient advocacy organizations, a registry may increase understanding of the natural history of a disease, contribute to the development of treatment guidelines, or facilitate research on treatment.9, 10 From a payer's perspective, registries can provide detailed information from large numbers of patients on how procedures, devices, or pharmaceuticals are actually used and on their effectiveness in different populations. This information may be useful for determining coverage policies.11 For a drug or device manufacturer, a registry-based study might demonstrate the performance of a product in the real world, meet a postmarketing commitment or requirement,12 develop hypotheses, or identify patient populations that will be useful for product development, clinical trials design, and patient recruitment. The U.S. Food and Drug Administration (FDA) has noted that “through the creation of registries, a sponsor can evaluate safety signals identified from spontaneous case reports, literature reports, or other sources, and evaluate the factors that affect the risk of adverse outcomes such as dose, timing of exposure, or patient characteristics.”13

The use of patient registries varies by priority condition, with cancer and cardiovascular disease having a large number of registries and areas such as developmental delays or dementia, far fewer. Overall, the use of patient registries appears to be active and growing. For example, a review of ClinicalTrials.gov in the area of cancer reveals over 270 large (more than 2,000 patients) observational studies that would meet the criteria for a patient registry. Of these studies, 4 have more than 100,000 patients, and 27 have more than 10,000. In some cases, the drivers for these registries have been Federal stakeholders. For example, since 2005, the FDA Center for Devices and Radiological Health has called for some 160 postapproval studies, many of which use new or existing registries to study the real-world effectiveness of specific devices in community practice.14 The establishment of RoPR provides a new resource for tracking registry development and use by condition, purpose, type, and multiple other factors.6

2.1. Evaluating Patient Outcomes

Studies from patient registries and randomized controlled trials (RCTs) have important and complementary roles in evaluating patient outcomes.15 Ideally, patient registries collect data in a comprehensive manner (with few excluded patients) and therefore produce outcome results that may be generalizable to a wide range of patients. They also evaluate care as it is actually provided, because care is not assigned, determined, or even recommended by a protocol. As a result, the outcomes reported may be more representative of what is achieved in real-world practice. Patient registries also offer the ability to evaluate patient outcomes when clinical trials are not practical (e.g., very rare diseases), and they may be the only option when clinical trials are not ethically acceptable. They are a powerful tool when RCTs are difficult to conduct, such as in surgery or when very long-term outcomes are desired.

RCTs are controlled experiments designed to test hypotheses that can ultimately be applied to real-world care. Because RCTs are often conducted under strict constraints, with detailed inclusion and exclusion criteria (and the need for subjects who are willing to be randomized), they are sometimes limited in their generalizability. If RCTs are not generalizable to the populations to which the information will be applied, they may not be sufficiently informative for decisionmaking. Conversely, patient registries that observe real-world clinical practice may collect all of the information needed to assess patient outcomes in a generalizable way, but interpreting this information correctly requires analytic methodology geared to address the potential sources of bias that challenge observational studies. Interpreting patient registry data also requires checks of internal validity and sometimes the use of external data sources to validate key assumptions (such as comparing the key characteristics of registry participants with external sources in order to demonstrate the comparability of registry participants with the ultimate reference population). Patient registries, RCTs, other study designs, and other data sources should all be considered tools in the toolbox for evidence development, each with its own advantages and limitations.16

2.2. Hierarchies of Evidence

One question that arises in a discussion of this type is where to place studies derived from patient registries within the hierarchies of evidence that are frequently used in developing guidelines or decisionmaking. While the definition of patient registry used in this user's guide is intentionally broad, the parameters of quality described in Chapter 25 are intended to help the user evaluate and identify registries that are sufficiently rigorous observational studies for use as evidence in decisionmaking. Many registries are, or include, high-quality studies of cohorts designed to address a specific problem and hypothesis. Still, even the most rigorously conducted registries, like prospective observational studies, are traditionally placed in a subordinate position to RCTs in some commonly used hierarchies, although equal to RCTs in others.17-19 Debate continues in the evidence community regarding these traditional methods of grading levels of evidence, their underlying assumptions, their shortcomings in assessing certain types of evidence (e.g., benefit vs. harm), and their interscale consistency in evaluating the same evidence.16, 20, 21

The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group has proposed a more robust approach that addresses some of the decisionmaking issues described in this user's guide. As noted by the GRADE collaborators:

[R]andomised trials are not always feasible and, in some instances, observational studies may provide better evidence, as is generally the case for rare adverse effects. Moreover, the results of randomised trials may not always be applicable—for example, if the participants are highly selected and motivated relative to the population of interest. It is therefore essential to consider study quality, the consistency of results across studies, and the directness of the evidence, as well as the appropriateness of the study design.22

AHRQ has also developed a guidance system for grading the strength of evidence that recommends a careful assessment of the potential value of observational studies. The guidance, which is designed to support the systematic reviews conducted by the Evidence-based Practice Center (EPC) program, is conceptually similar to the GRADE system.23 When using the AHRQ approach, reviewers typically give evidence from observational studies a low starting grade and evidence from RCTs a high starting grade. These initial grades can then be raised or lowered depending on the strength of the five required evidence domains (study limitations, directness, consistency, precision, and reporting bias).24 For example, the reviewers may find that observational studies are particularly relevant for some systematic review questions. The report notes:

EPCs may act on the judgment that, for certain outcomes such as harms, observational studies have less risk of bias than do RCTs or that the available RCTs have a substantial risk of bias. In such instances, the EPC may move up the initial grade for strength of evidence based on observational studies to moderate or move down the initial rating based on RCTs to moderate.23

Reviewers may also raise or lower evidence grades based on a secondary set of domains (dose-response association, existence of confounding that would diminish an observed effect, and strength of association). These secondary domains supplement the required domains and are used when relevant to the systematic review question. The report explains that the secondary domains “may increase strength of evidence and are especially relevant for observational studies where one may begin with a lower overall strength of evidence grade based on study limitations.”23

As the methods for grading evidence for different purposes continue to evolve, this user's guide can serve as a guide to help such evaluators understand study quality and identify well-designed registries. Beyond the evidence hierarchy debate, users of evidence understand the value of registries for providing complementary information that can extend the results of clinical trials to populations not studied in those trials, for demonstrating the real-world effects of treatments outside of the research setting and potentially in large subsets of affected patients, and for providing long-term followup when such data are not available from clinical trials.

2.3. Defining Patient Outcomes

The focus of this user's guide is the use of registries to evaluate patient outcomes. An outcome may be thought of as an end result of a particular health care practice or intervention. According to AHRQ, end results include effects that people experience and care about.25 The National Cancer Institute further clarifies that “final” endpoints are those that matter to decisionmakers: patients, providers, private payers, government agencies, accrediting organizations, or society.26, 27 Examples of these outcomes include biomedical outcomes, such as survival and disease-free survival, health-related quality of life, satisfaction with care, and economic burden.28 Although final endpoints are ultimately what matter, it is sometimes more practical when creating registries to collect intermediate outcomes (such as whether processes or guidelines were followed) and clinical outcomes (such as whether a tumor regressed or recurred) that predict success in improving final endpoints.

In Crossing the Quality Chasm,29 the Institute of Medicine (IOM) describes the six guiding aims of health care as providing care that is safe, effective, efficient, patient-centered, timely, and equitable. (The last three aims focus on the delivery and quality of care.) While these aims are not outcomes per se, they generally describe the dimensions of results that matter to decisionmakers in the use of a health care product or service: Is it safe? Does it produce greater benefit than harm? Is it clinically effective? Does it produce the desired effect in real-world practice? Does the right patient receive the right therapy or service at the right time? Is it cost effective or efficient? Does it produce the desired effect at a reasonable cost relative to other potential expenditures? Is it patient oriented, timely, and equitable? Most of the patient outcomes that registries evaluate reflect one or more of the IOM guiding aims. For example, a patient presenting with an ischemic stroke to an emergency room has a finite window of opportunity to receive a thrombolytic drug, and the patient outcome, whether or not the patient achieves full recovery, is dependent not only on the product's dissolving the clot but also on the timeliness of its delivery.30, 31

2.4. Purposes of Registries

As discussed throughout this user's guide, registries should be designed and evaluated with respect to their intended purpose(s). Registry purposes can be broadly described in terms of patient outcomes. While there are a number of potential purposes for registries, this handbook primarily discusses four major purposes: (1) describing the natural history of disease, (2) determining clinical and/or cost-effectiveness, (3) assessing safety or harm, and (4) measuring or improving quality of care. Other purposes of patient registries mentioned but not discussed in detail in this user's guide are for public health surveillance and disease control. An extensive body of literature from the last half century of experience with cancer and other disease surveillance registries is available.

2.4.1. Describing Natural History of Disease

Registries may be established to evaluate the natural history of a disease, meaning its characteristics, management, and outcomes with and/or without treatment. The natural history may be variable across different groups or geographic regions, and it often changes over time. In many cases, the natural histories of diseases are not well described. Furthermore, the natural histories of diseases may change after the introduction of certain therapies. As an example, patients with rare diseases, such as the lysosomal storage diseases, who did not previously survive to their 20s, may now be entering their fourth and fifth decades of life, and this uncharted natural history is being first described through a registry.32 The role of registries in rare diseases is explored in Chapter 20.

2.4.2. Determining Effectiveness

Registries may be developed to determine clinical effectiveness or cost-effectiveness in real-world clinical practice. Multiple studies have demonstrated disparities between the results of clinical trials and results in actual clinical practice.33, 34 Furthermore, efficacy in a clinical trial for a well-defined population may not be generalizable to other populations or subgroups of interest. As an example, many important heart failure trials have focused on a predominantly white male population with a mean age of approximately 60 years, whereas actual heart failure patients are older, more diverse, and have a higher mortality rate than the patients in these trials.35 Similarly, underrepresentation of older patients has been reported in clinical trials of 15 different types of cancer (e.g., studies with only 25 percent of patients age 65 years and over, while the expected rate is greater than 60 percent).36 Data from registries have been used to fill these gaps for decisionmakers. For example, the FDA used the American Academy of Ophthalmology's intraocular lens registry to expand the label for intraocular lenses to younger patients.37 Registries may also be particularly useful for tracking effectiveness outcomes for a longer period than is typically feasible with clinical trials. For example, some growth hormone registries have tracked children well into adulthood.

In addition to clinical effectiveness, registries can be used to assess cost-effectiveness. Registries can be designed to collect cost data and effectiveness data for use in modeling cost-effectiveness.38 Cost-effectiveness is a means to describe the comparative value of a health care product or service in terms of its ability to achieve a desired outcome for a given unit of resources.39 A cost-effectiveness analysis examines the incremental benefit of a particular intervention and the costs associated with achieving that benefit. Cost-effectiveness studies compare costs with clinical outcomes measured in units such as life expectancy or disease-free periods. Cost-utility studies compare costs with outcomes adjusted for quality of life (utility), such as quality-adjusted life years (QALYs). Utilities allow comparisons to be made across conditions because the measurement is not disease specific.40 It should be noted that for both clinical effectiveness and cost-effectiveness, differences between treatments are indirect and must be inferred from data analysis, simulation modeling, or some mixture.

With improvement in methodologies for using observational research for comparative effectiveness research (CER), including better methods for managing bias and better understanding of the limitations,41 there is both increasing interest and investment in registries for CER across a number of stakeholders. Reports from the IOM and the Congressional Budget Office in 2007 cited the importance of patient registries in developing comparative effectiveness evidence.42, 43 The Federal Coordinating Council for Comparative Effectiveness Research, in its Report to the President and the Congress (June 30, 2009), defined CER as “the conduct and synthesis of research comparing benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in ‘real world’ settings.”44 The report specifically identifies patient registries as a core component of CER data infrastructure.

More recently, the newly formed Patient-Centered Outcomes Research Institute (PCORI) has identified registries as an important potential source of data to support patient-centered outcomes research (PCOR). PCOR “assesses the benefits and harms of preventive, diagnostic, therapeutic, palliative, or health delivery system interventions to inform decisionmaking, highlighting comparisons and outcomes that matter to people; is inclusive of an individual's preferences, autonomy and needs, focusing on outcomes that people notice and care about such as survival, function, symptoms, and health related quality of life; incorporates a wide variety of settings and diversity of participants to address individual differences and barriers to implementation and dissemination; and investigates (or may investigate) optimizing outcomes while addressing burden to individuals, availability of services, technology, and personnel, and other stakeholder perspectives.”45

Similar to their function in CER, registries are expected play an important role in this new area of research in part because of their ability to provide information on ‘real-world’ settings and broad patient populations. PCORI included minimum standards for the use of registries for PCOR in the Methodology Report.46 While some registries are designed explicitly to examine questions of comparative effectiveness or patient-centered outcomes research, many others are designed for different objectives yet still collect data that are useful for these analyses. Registries that were not explicitly designed for CER or PCOR may need to be augmented or linked to other data sources—for example, to obtain long-term outcomes data in the case of an in-hospital registry using linkage to claims data to evaluate blood pressure medications.47

2.4.3. Measuring or Monitoring Safety and Harm

Registries may be created to assess safety versus harm. Safety here refers to the concept of being free from danger or hazard. One goal of registries in this context may be to quantify risk or to attribute it properly. Broadly speaking, patient registries can serve as an active surveillance system for the occurrence of unexpected or harmful events for products and services. Such events may range from patient complaints about minor side effects to severe adverse events such as fatal drug reactions or patient falls in the hospital.

Patient registries offer multiple advantages for active surveillance. First, the current practice of spontaneous reporting of adverse events relies on a nonsystematic recognition of an adverse event by a clinician and the clinician's active effort to make a report to manufacturers and health authorities. Second, these events are generally reported without a denominator (i.e., the exposed or treated population), and therefore an incidence rate is difficult to determine. Because patient registries can provide systematic data on adverse events and the incidence of these events, they are being used with increasing frequency in the areas of health care products and services. The role of registries in monitoring product safety is discussed in more detail in Chapter 19.

2.4.4. Measuring Quality

Registries may be created to measure quality of care. The IOM defines quality as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.” Quality-focused registries are being used increasingly to assess differences between providers or patient populations based on performance measures that compare treatments provided or outcomes achieved with “gold standards” (e.g., evidence-based guidelines) or comparative benchmarks for specific health outcomes (e.g., risk-adjusted survival or infection rates). Such programs may be used to identify disparities in access to care, demonstrate opportunities for improvement, establish differentials for payment by third parties, or provide transparency through public reporting. There are multiple examples of such differences in treatment and outcomes of patients in a range of disease areas.48-53 Quality improvement registries are described further in Chapter 22.

2.4.5. Multiple Purposes

Many registries will be developed to serve more than one of these purposes. Registries developed for one purpose may also be modified to serve additional purposes as the research, practice, or policy environment changes. While registries often serve more than one purpose, their original or primary purpose generally guides their design and, as a result, more care is needed in evaluating results for secondary or additional purposes.

3. Taxonomy for Patient Registries

Even limited to the definitions described above, the breadth of studies that might be included as patient registries is large. Patients in a registry are typically selected based on a particular disease, condition (e.g., a risk factor), or exposure. This user's guide uses these common selection criteria to develop a taxonomy or classification based on how the populations for registries are defined. Three general categories with multiple subcategories and combinations account for the majority of registries that are developed for evaluating patient outcomes. These categories include observational studies in which the patient has had an exposure to a product or service, or has a particular disease or condition, or various combinations thereof.

3.1. Product Registries

In the case of a product registry, the patient is exposed to a health care product, such as a drug or a device. The exposure may be brief, as in a single dose of a pharmaceutical product, or extended, as in an implanted device or chronic usage of a medication.

Device registries may include all, or a subset, of patients who receive the device. A registry for all patients who receive an implantable cardioverter defibrillator, a registry of patients with hip prostheses, or a registry of patients who wear contact lenses are all examples of device registries. Biopharmaceutical product registries similarly have several archetypes, which may include all, or subsets, of patients who receive the biopharmaceutical product. For example, the British Society for Rheumatology established a national registry of patients on biologic therapy.54 Again, the duration of exposure may range from a single event to a lifetime of use. Eligibility for the registry includes the requirement that the patient received the product or class of products (e.g., COX-2 inhibitors). In some cases, public health authorities mandate such registries to ensure safe use of medications. Examples include registries for thalidomide, clozapine, and isotretinoin.

Pregnancy registries represent a separate class of biopharmaceutical product registries that focus on possible exposures during pregnancy and the neonatal consequences. The FDA has a specific guidance focused on pregnancy exposure registries.55 This guidance uses the term “pregnancy exposure registry” to refer to “a prospective observational study that actively collects information on medical product exposure during pregnancy and associated pregnancy outcomes.” Pregnancy registries are discussed in more detail in Chapter 21.

3.2. Health Services Registries

In the context of evaluating patient outcomes, another type of exposure that can be used to define registries is exposure to a health care service. Health care services that may be used to define inclusion in a registry include individual clinical encounters, such as office visits or hospitalizations, procedures, or full episodes of care. Examples include registries enrolling patients undergoing a procedure (e.g., carotid endarterectomy, appendectomy, or primary coronary intervention) or admitted to a hospital for a particular diagnosis (e.g., community-acquired pneumonia). In these registries, one purpose of the registry is to evaluate the health care service with respect to the outcomes. Health care service registries are sometimes used to evaluate the processes and outcomes of care for quality measurement purposes (e.g., Get With The Guidelines® of the American Heart Association, National Surgical Quality Improvement Program of the Department of Veterans Affairs and the American College of Surgeons).

3.3. Disease or Condition Registries

Disease or condition registries use the state of a particular disease or condition as the inclusion criterion. In disease or condition registries, the patient may always have the disease (e.g., a rare disease such as cystic fibrosis or Pompe disease, or a chronic illness such as heart failure, diabetes, or end-stage renal disease) or may have the disease or condition for a more limited period of time (e.g., infectious diseases, some cancers, obesity). These registries typically enroll the patient at the time of a routine health care service, although patients also can be enrolled through voluntary self-identification processes that do not depend on utilization of health care services (such as Internet recruiting of volunteers). In other disease registries, the patient has an underlying disease or condition, such as atherosclerotic disease, but is enrolled only at the time of an acute event or exacerbation, such as hospitalization for a myocardial infarction or ischemic stroke.

3.4. Combinations

Complicating this classification approach is the reality that these categories can be overlapping in many registries. For example, a patient with ischemic heart disease may have an acute myocardial infarction and undergo a primary coronary intervention with placement of a drug-eluting stent and postintervention management with clopidogrel. This patient could be enrolled in an ischemic heart disease registry tracking all patients with this disease over time, a myocardial infarction registry that is collecting data on patients who present to hospitals with acute myocardial infarction (cross-sectional data collection), a primary coronary intervention registry that includes management with and without devices, a coronary artery stent registry limited to ischemic heart disease patients, or a clopidogrel product registry that includes patients undergoing primary coronary interventions.

3.5. Duration of Observation

The duration of the observational period for a registry is also a useful descriptor. Observation periods may be limited to a single episode of care (e.g., a hospital discharge registry for diverticulitis), or they may extend for as long as the lifetime of patients with a chronic disease (e.g., cystic fibrosis or Pompe disease) or patients receiving a novel therapy (e.g., gene therapy). The period of observation or followup depends on the outcomes of interest.

3.6. From Registry Purpose to Design

As will be discussed extensively in this document, the purpose of the registry defines the registry focus (e.g., product vs. disease) and therefore the registry type. A registry created for the purpose of evaluating outcomes of patients receiving a particular coronary artery stent might be designed as a single product registry if, for example, the purpose is to systematically collect adverse event information on the first 10,000 patients receiving the product. However, the registry might alternatively be designed as a health care service registry for primary coronary intervention if a purpose is to collect comparative effectiveness or safety data on other treatments or products within the same registry.

4. Patient Registries and Policy Purposes

In addition to the growth of patient registries for scientific and clinical purposes, registries are receiving increased attention for their potential role in policymaking or decisionmaking.56 As stated earlier, registries may offer a view of real-world health care that is typically inaccessible from clinical trials or other data sources and may provide information on the generalizability of the data from clinical trials to populations not studied in those trials.

The utility of registry data for decisionmaking is related to three factors: the stakeholders, the primary scientific question, and the context. The stakeholders are those associated with the disease or procedure that may be affected from a patient, provider, payer, regulator, or other perspective. The primary scientific question for a registry may relate to effectiveness, safety, or practice patterns. The context includes the scientific context (e.g., previous randomized trials and modeling efforts that help to more precisely define the primary scientific question), as well as the political, regulatory, funding, and other issues that provide the practical parameters around which the registry is developed. In identifying the value of information from registries, it is essential to look at the data with specific reference to the purpose and focus of the registry.

From a policy perspective, there are several scenarios in which the decision to develop a registry may arise. One possible scenario is as follows. An item or service is considered for use. Stakeholders in the decision collaboratively define “adequate data in support of the decision at hand.” Here, “adequate data” refers to information of sufficient relevance and quality to permit an informed decision. An evidence development strategy is selected from one of many potential strategies (RCT, practical clinical trial, registry, etc.) based on the quality of the evidence provided by each design, as well as the burden of data collection and the cost that is imposed. This tradeoff of the quality of evidence versus cost of data collection for each possible design is termed the “value of information” exercise (Figure 1–1). Registries should be preferred in those circumstances where they provide sufficiently high-quality information for decisionmaking at a sufficiently low cost (relative to other “acceptable” designs).

The figure uses a series of boxes connected by arrows to depict the “value of information” exercise for deciding when to develop a registry. Each box represents a step in the process. In the first step, the item/service is presented for consideration. In the second step, the clinical/policy question(s) are formulated. In the third step, “adequate data” is defined. In the fourth step, the current evidence is reviewed. If the data are deemed adequate for decision-making, the decision is made. If the data are not deemed adequate for decision-making, a list of alternative designs is constructed. Next, the “value of information” exercise is performed. Lastly, the evidence development strategy is implemented.

Figure 1–1

Deciding when to develop a registry: The “value of information” exercise.

One set of policy determinations that may be informed by a patient registry centers on the area of payment for items or services. For example, the Centers for Medicare & Medicaid Services (CMS) issued Guidance on National Coverage Determinations With Data Collection as a Condition of Coverage in 2006. That original guidance document (which has undergone subsequent revisions, including an additional draft guidance published in 201257) provided several examples of how data collected in a registry might be used in the context of coverage determinations. As described in the Guidance:

[T]he purpose of CED [Coverage with Evidence Development] is to generate data on the utilization and impact of the item or service evaluated in the NCD [National Coverage Determination], so that Medicare can (a) document the appropriateness of use of that item or service in Medicare beneficiaries under current coverage; (b) consider future changes in coverage for the item or service; (c) generate clinical information that will improve the evidence base on which providers base their recommendations to Medicare beneficiaries regarding the item or service.56

The Guidance provided insight into when registry data may be useful to policymakers. These purposes range from demonstrating that a particular item or service was provided appropriately to patients meeting specific characteristics, to collecting new information that is not available from existing clinical trials. CED based on registries may be especially relevant when current data do not address relevant outcomes for beneficiaries, off-label or unanticipated uses, important patient subgroups, or operator experience or other qualifications. Registry-based studies may also be important when an existing treatment is being reconsidered. (An RCT may not be possible under such circumstances.) Registry-based studies are also being used increasingly in fulfillment of postmarketing commitments and requirements.

In many countries, policy determinations on payment rely on cost-effectiveness and cost-utility data and therefore can be informed by registries as well as clinical trials.58 These data are used and reviewed in a variety of ways. In some countries, there may be a threshold above which a payer is willing to pay for an improvement in patient outcomes.59 In these scenarios—particularly for rare diseases, when it can be difficult to gather clinical effectiveness data together with quality-of-life data in a utility format—the establishment of disease-specific data registries has been recommended to facilitate the process of technology assessment and improving patient care.60 In fact, the use of new or existing registries to assess health technology or risk-sharing arrangements is growing in such countries as the United Kingdom, France, Germany, and Australia, and in conditions ranging from bariatric surgery to stroke care.61-66

Consider the clinical question of carotid endarterectomy surgery for patients with a high degree of stenosis of the carotid artery. Randomized trials, using highly selected patients and surgeons, indicate a benefit of surgery over medical management in the prevention of stroke. However, that benefit may be exquisitely sensitive to the surgical complication rates; a relatively small increase in the rate of surgical complications is enough to make medical management the preferred strategy instead. In addition, the studies of surgical performance in a variety of hospitals may suggest substantial variation in surgical mortality and morbidity for this procedure. In such a case, a registry to evaluate treatment outcomes, adjusted by hospital and surgeon, might be considered to support a policy decision as to when the procedure should be reimbursed (e.g., only when performed in medical centers resembling those in the various randomized trials, or only by surgeons or facilities with an acceptably low rate of complications).67

5. Global Registries

As many stakeholders have international interests in diseases, conditions, and health care products and services, it is not surprising that interest in global patient registries is growing. While some of the specific legal and regulatory discussions in this user's guide are intended for and limited to the United States, most of the concepts and specifics are more broadly applicable to similar activities worldwide. Chapter 7 (ethics, data ownership, and privacy), Chapter 9 (protection of registry data), and Chapter 12 (adverse event detection, processing, and reporting) are perhaps the most limited in their applicability outside the United States. There may be additional considerations in data element selection and patient-reported outcome measure selection (Chapters 4 and 5) stemming from differences ranging from medical training to use of local remedies; the types of data sources that are available outside the United States (Chapter 6); the requirements for informed consent (Chapter 8); the issues surrounding clinician and patient recruitment and retention in different health systems and cultures (Chapter 10); specific data collection and management options and complexities (Chapter 11), ranging from available technologies to languages; and specific requirements for mandated pregnancy registries (Chapter 21).

6. Summary

A patient registry is an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure and that serves predetermined scientific, clinical, or policy purpose(s). Studies derived from well-designed and well-performed patient registries can provide a real-world view of clinical practice, patient outcomes, safety, and clinical, comparative, and cost-effectiveness, and can serve a number of evidence development and decisionmaking purposes. In the chapters that follow, this user's guide presents practical design and operational issues, evaluation principles, and good registry practices.

References for Chapter 1

1.
Webster's English Dictionary. [August 12, 2012]. http://www​.m-w.com.
2.
Brooke EM. The current and future use of registers in health information systems. Geneva: World Health Organization; 1974. Publication No. 8.
3.
National Committee on Vital and Health Statistics. Frequently Asked Questions About Medical and Public Health Registries. [August 14, 2012]. http://ncvhs​.hhs.gov/9701138b.htm.
4.
Dokholyan RS, Muhlbaier LH, Falletta JM, et al. Regulatory and ethical considerations for linking clinical and administrative databases. Am Heart J. 2009 Jun;157(6):971–82. [PubMed: 19464406]
5.
Hammill BG, Hernandez AF, Peterson ED, et al. Linking inpatient clinical registry data to Medicare claims data using indirect identifiers. Am Heart J. 2009 Jun;157(6):995–1000. [PMC free article: PMC2732025] [PubMed: 19464409]
6.
Developing a Registry of Patient Registries (RoPR). Project Abstract. Agency for Healthcare Research and Quality; [August 14, 2012]. http://www​.effectivehealthcare​.ahrq.gov/index​.cfm/search-for-guides-reviews-and-reports​/?productid=690&pageaction=displayproduct.
7.
LaBresh KA, Gliklich R, Liljestrand J, et al. Using “get with the guidelines” to improve cardiovascular secondary prevention. Jt Comm J Qual Saf. 2003 Oct;29(10):539–50. [PubMed: 14567263]
8.
Kennedy L, Craig AM. Global registries for measuring pharmacoeconomic and quality-of-life outcomes: focus on design and data collection, analysis and interpretation. Pharmacoeconomics. 2004;22(9):551–68. [PubMed: 15209525]
9.
Charrow J, Esplin JA, Gribble TJ, et al. Gaucher disease: recommendations on diagnosis, evaluation, and monitoring. Arch Intern Med. 1998 Sep 14;158(16):1754–60. [PubMed: 9738604]
10.
Bushby K, Lynn S, Straub T, et al. Collaborating to bring new therapies to the patient--the TREAT-NMD model. Acta Myol. 2009 Jul;28(1):12–5. [PMC free article: PMC2859629] [PubMed: 19772190]
11.
Dhruva SS, Phurrough SE, Salive ME, et al. CMS's landmark decision on CT colonography--examining the relevant data. N Engl J Med. 2009 Jun 25;360(26):2699–701. [PubMed: 19474419]
12.
Postmarketing studies and clinical trials – implementation of Section 505(o) of the Federal Food, Drug and Cosmetic Act. FDA Guidance for Industry. Draft guidance. Jul, 2009. [August 14, 2012]. http://www​.fda.gov/downloads​/Drugs/GuidanceComplianceRegulatoryInformation​/Guidances/UCM172001.pdf.
13.
U.S. Food and Drug Administration. Guidance for Industry. Good pharmacovigilance practices and pharmacoepidemiologic assessment. Mar, 2005. [August 14, 2012]. http://www​.fda.gov/downloads​/RegulatoryInformation​/Guidances/UCM126834.pdf.
14.
15.
Dreyer NA, Garner S. Registries for robust evidence. JAMA. 2009 Aug 19;302(7):790–1. [PubMed: 19690313]
16.
Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000 Jun 22;342(25):1887–92. [PMC free article: PMC1557642] [PubMed: 10861325]
17.
Guyatt GH, Sackett DL, Sinclair JC, et al. Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. JAMA. 1995 Dec 13;274(22):1800–4. [PubMed: 7500513]
18.
Agency for Healthcare Research and Quality. Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews. Version 1.0. Rockville, MD: [August 14, 2012]. [Draft posted Oct. 2007] http:​//effectivehealthcare​.ahrq.gov/repFiles​/2007_10DraftMethodsGuide.pdf.
19.
Vandenbroucke JP. Observational research, randomised trials, and two views of medical science. PLoS Med. 2008 Mar 11;5(3):e67. [PMC free article: PMC2265762] [PubMed: 18336067]
20.
Atkins D, Eccles M, Flottorp S, et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004 Dec 22;4(1):38. [PMC free article: PMC545647] [PubMed: 15615589]
21.
Rawlins M. De Testimonio: on the evidence for decisions about the use of therapeutic interventions. Clin Med. 2008 Dec;8(6):579–88. [PMC free article: PMC4954394] [PubMed: 19149278]
22.
Atkins D, Best D, Briss PA, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004 Jun 19;328(7454):1490. [PMC free article: PMC428525] [PubMed: 15205295]
23.
Grading the Strength of a Body of Evidence When Assessing Health Care Interventions – AHRQ and the Effective Health Care Program: An Update. [October 12, 2012]. Draft released for public comment. http://www​.effectivehealthcare​.ahrq.gov/ehc​/products/457/1163​/GradingTheStrengthofEvidence​_DraftMethodsChapter_20120625​.pdf.
24.
Viswanathan M, Ansari MT, Berkman ND, Chang S, Hartling L, McPheeters LM, Santaguida PL, Shamliyan T, Singh K, Tsertsvadze A, Treadwell JR. Agency for Healthcare Research and Quality Methods Guide for Comparative Effectiveness Reviews. Rockville, MD: Agency for Healthcare Research and Quality; Mar, 2012. [October 12, 2012]. Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. AHRQ Publication No. 12-EHC047-EF. http://www​.effectivehealthcare​.ahrq.gov/ehc​/products/322/998​/MethodsGuideforCERs​_Viswanathan_IndividualStudies.pdf.
25.
Clancy CM, Eisenberg JM. Outcomes research: measuring the end results of health care. Science. 1998 Oct 9;282(5387):245–6. [PubMed: 9841388]
26.
Lipscomb J, Snyder CF. The Outcomes of Cancer Outcomes Research: focusing on the National Cancer Institute's quality-of-care initiative. Med Care. 2002 Jun;40(6 Suppl):III3–10. [PubMed: 12064755]
27.
National Cancer Institute. The Emerging Field of Outcomes Research. [August 14, 2012]. http://outcomes​.cancer​.gov/aboutresearch/index.html.
28.
Lipscomb J, Donaldson MS, Hiatt RA. Cancer outcomes research and the arenas of application. J Natl Cancer Inst Monogr. 2004;(33):1–7. [PubMed: 15504917]
29.
Institute of Medicine; Committee on Quality of Heallth Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington DC: National Academy Press, Institute of Medicine; 2001.
30.
Schwamm LH, LaBresh KA, Pan WQ, et al. Get With The Guidelines – Stroke produces sustainable improvements in hospital-based acute stroke care. Stroke. 2006;37(2):624.
31.
Schwamm LH, LaBresh KA, Pan WQ, et al. Get With The Guidelines – Stroke improves the rate of “defect-free” acute stroke care. Stroke. 2006;37(2):659.
32.
Barranger JA, O'Rourke E. Lessons learned from the development of enzyme therapy for Gaucher disease. J Inherit Metab Dis. 2001;24 Suppl 2:89–96. discussion 87-8. [PubMed: 11758684]
33.
Wennberg DE, Lucas FL, Birkmeyer JD, et al. Variation in carotid endarterectomy mortality in the Medicare population: trial hospitals, volume, and patient characteristics. JAMA. 1998 Apr 22-29;279(16):1278–81. [PubMed: 9565008]
34.
MacIntyre K, Capewell S, Stewart S, et al. Evidence of improving prognosis in heart failure: trends in case fatality in 66 547 patients hospitalized between 1986 and 1995. Circulation. 2000 Sep 5;102(10):1126–31. [PubMed: 10973841]
35.
Konstam MA. Progress in heart failure Management? Lessons from the real world. Circulation. 2000 Sep 5;102(10):1076–8. [PubMed: 10973832]
36.
Hutchins LF, Unger JM, Crowley JJ, et al. Underrepresentation of patients 65 years of age or older in cancer-treatment trials. N Engl J Med. 1999 Dec 30;341(27):2061–7. [PubMed: 10615079]
37.
Brown SL, Bright RA, Tavris DR, editors. Medical device epidemiology and surveillance. John Wiley & Sons, Ltd.; 2007. Opthalmic devices and clinical epidemiology. chapter 28.
38.
Lipscomb J, Yabroff KR, Brown ML, et al. Health care costing: data, methods, current applications. Med Care. 2009 Jul;47(7 Suppl 1):S1–6. [PubMed: 19536004]
39.
Eichler HG, Kong SX, Gerth WC, et al. Use of cost-effectiveness analysis in health-care resource allocation decision-making: how are cost-effectiveness thresholds expected to emerge? Value Health. 2004 Sep-Oct;7(5):518–28. [PubMed: 15367247]
40.
Palmer AJ. Health economics--what the nephrologist should know. Nephrol Dial Transplant. 2005 Jun;20(6):1038–41. [PubMed: 15840678]
41.
Good ReseArch for Comparative Effectiveness. [August 14, 2012]. http://www​.graceprinciples.org.
42.
Institute of Medicine. Learning what works best: the Nation's need for evidence on comparative effectiveness in health care. Washington DC: National Academy Press; 2007. pp. 1–80.
43.
Congressional Budget Office. Research on the Comparative Effectiveness of Medical Treatments: Issues and Options for an Expanded Federal Role. 2007. [August 14, 2012]. http://www​.cbo.gov/sites​/default/files/cbofiles​/ftpdocs/88xx​/doc8891/12-18-comparativeeffectiveness​.pdf.
44.
Federal Coordinating Council for Comparative Effectiveness Research. Report to the President and the Congress. U.S. Department of Health and Human Services; Jun 30, 2009. [August 14, 2012]. http://www​.hhs.gov/recovery​/programs/cer/cerannualrpt.pdf.
45.
Patient-Centered Outcomes Research Institute. Patient-Centered Outcomes Research. [August 28, 2012]. http://www​.pcori.org/what-we-do/pcor/
46.
Public comment draft report of the Patient-Centered Outcomes Research Institute (PCORI) Methodology Committee presented on July 23, 2012, and revised thereafter. [August 28, 2012]. http://pcori​.org/assets​/MethodologyReport-Comment.pdf.
47.
Agency for Healthcare Research and Quality. Bridging Knowledge Gaps in the Comparative Effectiveness of ACE Inhibitors and ARBs. Draft abstract. 2008. [August 14, 2012]. http://www​.effectivehealthcare​.ahrq.gov/index​.cfm/search-for-guides-reviews-and-reports​/?productid=296&pageaction=displayproduct.
48.
Hodgson DC, Fuchs CS, Ayanian JZ. Impact of patient and provider characteristics on the treatment and outcomes of colorectal cancer. J Natl Cancer Inst. 2001 Apr 4;93(7):501–15. [PubMed: 11287444]
49.
Reeves MJ, Fonarow GC, Zhao X, et al. Quality of care in women with ischemic stroke in the GWTG program. Stroke. 2009 Apr;40(4):1127–33. [PubMed: 19211482]
50.
Fonarow GC, Abraham WT, Albert NM, et al. Influence of a performance-improvement initiative on quality of care for patients hospitalized with heart failure: results of the Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients With Heart Failure (OPTIMIZE-HF). Arch Intern Med. 2007 Jul 23;167(14):1493–502. [PubMed: 17646603]
51.
Greene FL, Gilkerson S, Tedder P, et al. The role of the hospital registry in achieving outcome benchmarks in cancer care. J Surg Oncol. 2009 Jun 15;99(8):497–9. [PubMed: 19466739]
52.
Schweikert B, Hunger M, Meisinger C, et al. Quality of life several years after myocardial infarction: comparing the MONICA/KORA registry to the general population. Eur Heart J. 2009 Feb;30(4):436–43. [PubMed: 19019995]
53.
Lane K, Kempf A, Magno C, et al. Regional differences in the use of sentinel lymph node biopsy for melanoma: a potential quality measure. Am Surg. 2008 Oct;74(10):981–4. [PubMed: 18942627]
54.
Griffiths I, Silman A, Symmons D, et al. BSR Biologics Registry. Rheumatology (Oxford). 2004 Dec;43(12):1463–4. [PubMed: 15328420]
55.
U.S. Food and Drug Administration. Guidance for Industry: Establishing Pregnancy Exposure Registries. [August 15, 2012]. http://www​.fda.gov/downloads​/Drugs/GuidanceComplianceRegulatoryInformation​/Guidances/ucm071639.pdf.
56.
Centers for Medicare & Medicaid Services. Guidance for the Public, Industry, and CMS Staff: National Coverage Determinations with Data Collection as a Condition of Coverage: Coverage with Evidence Development. Jul 12, 2006.
57.
Centers for Medicare & Medicaid Services. Draft Guidance for the Public, Industry, and CMS Staff: Coverage with Evidence Development in the context of coverage decisions. [February 20, 2013]. http://www​.cms.gov/medicare-coverage-database​/details/medicare-coverage-document-details​.aspx?MCDId=23.
58.
Connock M, Burls A, Frew E, et al. The clinical effectiveness and cost-effectiveness of enzyme replacement therapy for Gaucher's disease: a systematic review. Health Technol Assess. 2006 Jul;10(24):iii–iv. ix–136. [PubMed: 16796930]
59.
Devlin N, Parkin D. Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Econ. 2004 May;13(5):437–52. [PubMed: 15127424]
60.
Connock M, Juarez-Garcia A, Frew E, et al. A systematic review of the clinical effectiveness and cost-effectiveness of enzyme replacement therapies for Fabry's disease and mucopolysaccharidosis type 1. Health Technol Assess. 2006 Jun;10(20):iii–iv. ix–113. [PubMed: 16729919]
61.
Chalkidou K, Tunis S, Lopert R, et al. Comparative effectiveness research and evidence-based health policy: experience from four countries. Milbank Q. 2009 Jun;87(2):339–67. [PMC free article: PMC2881450] [PubMed: 19523121]
62.
National Institute for Health and Clinical Excellence (UK). Final appraisal determination: alteplase for the treatment of acute ischaemic stroke. 2007. [August 14, 2012]. http://www​.nice.org.uk​/nicemedia/live/11617/33971/33971.pdf.
63.
Graves SE, Davidson D, Ingerson L, et al. The Australian Orthopaedic Association National Joint Replacement Registry. Med J Aust. 2004 Mar 1;180(5 Suppl):S31–4. [PubMed: 14984361]
64.
Owen A, Spinks J, Meehan A, et al. A new model to evaluate the long-term cost effectiveness of orphan and highly specialised drugs following listing on the Australian Pharmaceutical Benefits Scheme: the Bosentan Patient Registry. J Med Econ. 2008;11(2):235–43. [PubMed: 19450082]
65.
Haute Autorité de Santé Heart surgery with or without extracorporeal circulation: role of the second surgeon. 2008. [August 14, 2012]. http://www​.has-sante​.fr/portail/upload/docs​/application/pdf/abstract​_heart_surgery.pdf.
66.
Haute Autorité de Santé Interview du docteur Jean-François Thébaut, cardiologue libéral et Président du Conseil National Professionnel de Cardiologie. [August 14, 2012]. [French.]. http://www​.has-sante​.fr/portail/jcms/c_777504​/interview-du-docteur-jean-francois-thebautcardiologue-liberal-et-president-du-conseil-nationalprofessionnel-de-cardiologie?portal​=c_63456.
67.
Matchar DB, Oddone EZ, McCrory DC, et al. Influence of projected complication rates on estimated appropriate use rates for carotid endarterectomy. Appropriateness Project Investigators. Academic Medical Center Consortium. Health Serv Res. 1997 Aug;32(3):325–42. [PMC free article: PMC1070194] [PubMed: 9240284]

Views

  • PubReader
  • Print View
  • Cite this Page

Related information

  • PMC
    PubMed Central citations
  • PubMed
    Links to PubMed

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...