NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Gliklich RE, Dreyer NA, editors. Registries for Evaluating Patient Outcomes: A User's Guide. 2nd edition. Rockville (MD): Agency for Healthcare Research and Quality (US); 2010 Sep.

Cover of Registries for Evaluating Patient Outcomes: A User's Guide

Registries for Evaluating Patient Outcomes: A User's Guide. 2nd edition.

Show details


This project was performed under a contract from the Agency for Healthcare Research and Quality (AHRQ) in collaboration with the Centers for Medicare & Medicaid Services (CMS) through the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Network of AHRQ’s Effective Health Care (EHC) Program. The purpose of the project was to update and expand Registries for Evaluating Patient Outcomes: A User’s Guide, published in 2007. The 2007 user’s guide was developed as a reference for establishing, maintaining, and evaluating the success of registries created to collect data about patient outcomes. The purpose of this revised and expanded second edition is to incorporate information on new methodological or technological advances into the existing chapters and to add new chapters to address emerging topics in registry science.

Both the 2007 version and this second edition were created with support from a large group of stakeholders. Following award of the initial project on September 29, 2005, we created a draft outline for the document, which was posted for public comment on AHRQ’s Effective Health Care Web site ( from January through March 2006. During that same period, we worked with AHRQ to create a process for selecting contributors and reviewers. We broadly solicited recommendations from a range of stakeholders, including government agencies, industry groups, medical professional societies, and other experts in the field; conducted a review of the pertinent literature; and contacted the initial list of contributors to confirm their interest and area of expertise and to seek further recommendations. Through that process and in collaboration with AHRQ and CMS, we arrived at a set of contributors and reviewers based on subject/content expertise, practical experience, and interest and availability, with balanced representation from key stakeholder groups for nearly all chapters. In addition, a request for submission of real-world case examples that could be used in the user’s guide to illustrate issues and challenges in implementing registries was posted on the Effective Health Care Web site. The primary selection criteria for these examples concerned their utility in illustrating a practical challenge and its resolution.

An initial meeting of contributors was convened in February 2006. A second meeting including contributors and chapter reviewers was held in June 2006, following creation of an initial draft document and focused review by the reviewers. The collaborative efforts of contributors, reviewers, and editors resulted in a draft document that was posted for public comment on the Effective Health Care Web site in October and November 2006. In all, 39 contributors and 35 individual reviewers participated in the creation of the first document, which was released in April 2007 and has been published online and in print.

In August 2008, the user’s guide update project was awarded. The project involved revising the existing chapters and case examples, creating new content to address four topics, and soliciting new case examples. From September to November 2008, we worked with AHRQ to select contributors and content reviewers for the new user’s guide. We followed a process similar to that used in the creation of the original user’s guide to arrive at a set of contributors and reviewers with subject matter expertise and a broad range of perspectives. The contributors drafted white papers on four topics: use of registries in product safety assessment, when to stop a registry, interfacing registries and electronic health records, and linking registry data. The white papers were reviewed and discussed at a meeting in April 2009. The papers were then posted for public comment in August and September 2009. After the papers were revised in response to public comments, the final papers were included in the expanded user’s guide.

During the same timeframe, we contacted the authors and reviewers of the 2007 version of the user’s guide. We asked authors and reviewers to update the existing chapters to address any new methodological, technological, or legal topics. The revised chapters were circulated for review and discussed at a meeting in July 2009. We also posted a new call for case examples on the Effective Health Care Web site in June 2009. The primary selection criteria for the new examples concerned their utility in illustrating issues and challenges related to the new topics addressed in the white papers. In addition, we contacted authors of the original case examples to obtain updated information on the registries.

For both the 2007 version and this second edition, the contributors and reviewers participated as individuals and not necessarily as representatives of their organizations. We are grateful to all those who contributed to both documents, and who reviewed them and shared their comments.

To begin the discussion of registries, we would like to clarify some distinctions between registries and clinical trials. Although this subject is further discussed in Chapter 1, we offer here the following distinctions from a high-level perspective. The clinical trial is an experiment in which an active intervention intended to change a human subject’s outcome is implemented, generally through a randomization procedure that takes decisionmaking away from the practitioner. The research protocol describes inclusion and exclusion criteria that are used to select the patients who will participate as human subjects, focusing the experiment on a homogeneous group. Human subjects and clinical researchers agree to adhere to a strict schedule of visits and to conduct protocol-specific tests and measurements.

In contrast, registries use an observational study design that does not specify treatments or require any therapies intended to change patient outcomes (except as specific treatments or therapies may be inclusion criteria). There are generally few inclusion and exclusion criteria in an effort to study a broad range of patients to make the results more generalizable. Patients are typically observed as they present for care, and the data collected generally reflect whatever tests and measurements a provider customarily uses.

Patient registries represent a useful tool for a number of purposes. Their ideal use and their role in evidence development, design, operations, and evaluation resemble but differ from clinical trials in a number of substantive ways, and therefore they should not be evaluated with the same constructs. This user’s guide presents what the contributors and reviewers consider good registry practices. Many registries today may not meet even the basic practices described. On the whole, registry science is in an active state of development. This second edition of the user’s guide is an important step in developing the field.

This book is divided into three sections: Creating, Operating, and Evaluating Registries. The first two sections provide basic information on key areas of registry development and operations, highlighting the spectrum of practices in each of these areas and their potential strengths and weaknesses. Section I, “Creating Registries,” includes eight chapters. “Patient Registries” defines and characterizes types of registries, their purposes, and uses, and describes their place within the scope of this document. “Planning a Registry” focuses on the recommended steps in planning a registry, from determining if a registry is the right option, to describing goals and objectives, to determining when a registry may end. “Registry Design” examines the specifics of designing a registry once the goals and objectives are known. “Use of Registries in Product Safety Assessment” describes the utility and challenges of designing a registry to assess safety. “Data Elements for Registries” provides a scientific and practical approach to selecting data elements. “Data Sources for Registries” addresses how existing data sources (administrative, pharmacy, other registries, etc.) may be used to enhance the value of patient registries. “Linking Registry Data: Technical and Legal Considerations” discusses the technical and legal issues surrounding the linkage of registry data with other data sources. “Principles of Registry Ethics, Data Ownership, and Privacy” reviews several key legal and ethical issues that should be considered in creating or operating a registry.

Section II, “Operating Registries,” provides a practical guide to the day-to-day operational issues and decisions for producing and interpreting high-quality registries. “Recruiting and Retaining Participants in the Registry” describes strategies for recruiting and retaining providers and patients. “Data Collection and Quality Assurance” reviews key areas of data collection, cleaning, storing, and quality assurance for registries. “Interfacing Registries With Electronic Health Records” describes the current state of electronic health record (EHR) integration technology and maps out potential options for developing interfaces between registries and EHRs. “Adverse Event Detection, Processing, and Reporting” examines relevant practical and regulatory issues. “Analysis and Interpretation of Registry Data To Evaluate Outcomes” addresses key considerations in analyzing and interpreting registry data.

Interspersed throughout the first two sections of the user’s guide are case examples. As discussed above, the choice of examples was limited to those submitted for consideration during the 2007 and 2009 public submission periods. The purpose of their inclusion is solely to illustrate specific points in the text from real-world examples, regardless of whether the source of the example is within the scope of the user’s guide as described in Chapter 1. Inclusion of a case example is not intended as an endorsement of the quality of the particular registry, nor do the case examples necessarily present registries that meet all the criteria described in Chapter 14 as basic elements of good practice. Rather, case examples are introduced to provide the reader with a richer description of the issue or question being addressed in the text. In some cases, we have no independent information on the registry other than what has been provided by the contributor.

Section III is “Evaluating Registries.” This final chapter on quality assessment summarizes key points from the earlier chapters in a manner that can be used to review the structure, data, or interpretations of patient registries. It describes good registry practice in terms of “basic elements” and “potential enhancements.” This information might be used by a person developing a registry, or by a reviewer or user of registry data or interpretations derived from registries.

Richard E. Gliklich

Nancy A. Dreyer

Senior Editors


Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...