2.1.1. Creating a Project Charter

The creation of a project charter is often a useful starting point in planning a transition. A project charter typically includes the following information:

• Overview of the transition
• Purpose/justification for the transition
• Goals and objectives of the transition
• Business case for the transition (if applicable)
• Identification of major stakeholders
• Assumptions and constraints (organizational, environmental, and external)
• Potential risks
• Milestones/deliverables or high-level timeline
• Budget
• Transition team members
• References to source documents, if applicable (e.g., new clinical practice guidelines)

2.1.2. Forming a Transition Team

The next step is to assemble the transition team, which will be responsible for planning and implementing the registry transition. It is important to include key stakeholders and to think broadly about the talent and expertise needed to accomplish a successful transition. In general, the transition team should include the following members:

• Sponsor/funding organization representative: Ensures that the team has the resources necessary to complete the project and keeps the sponsor apprised of any issues that may affect the timeline or budget for the transition.
• Project manager: Accountable for all aspects of the transition, including timely escalation of issues for resolution.
• Clinical expert: Provides guidance on changes that affect the clinical content of the registry (e.g., changes in purpose and data collection), and provides input on data migration, as needed.
• Epidemiologist/biostatistician: Provides guidance on changes that affect the study design and analysis plans (e.g., changes in purpose, data collection, data management, and data migration).
• Data management expert: Provides guidance on changes that affect data collection, data storage, or data quality assurance.
• Legal/ethical expert: Provides guidance on how changes affect the legal and ethical construct of the registry (e.g., contract with funding source(s), contracts with participating sites, contracts with vendors, and data sharing agreements), and identifies any ethical issues (e.g., need for institutional review board review, changes to informed consent documents, need to re-consent participants).
• Other representatives: Depending on the nature of the transition, other representatives may be included on the transition team, such as (1) a principal investigator or study coordinator from a participating site with experience entering data into the registry to provide guidance on feasibility and burden of data entry, (2) a technical expert to help guide a transition to a new technology platform, (3) a patient advocate to provide the patient's perspective, and/or (4) representatives of other databases that are linked to the registry (e.g., a related registry or substudy).

Once the transition team has been assembled, it is critical to achieve consensus on the rationale and the overarching goal(s) for the registry transition. Open discussion at this stage may identify unanticipated barriers, which can be addressed proactively in the transition planning. Gaining the full support of the transition team will increase the likelihood of a successful registry transition.

2.1.3. Developing a Project Plan

The next step for the transition team is to develop a detailed project plan encompassing timeline and budget. The transition project plan should be thoughtful, complete, and realistic. As with all projects of this magnitude and complexity, disagreement among stakeholders over scope, cost overruns, and time delays may occur. These predictable issues should be anticipated as much as possible, and risk mitigation strategies considered. The project plan should also consider other sources of risk specific to the transition (e.g., unexpected issues with technology compatibility, delays in obtaining institutional review board approval, and disputes related to ownership issues). Chapter 2 provides more information on project planning considerations.

The project plan should also address staffing issues. The transition may require new expertise and skills that alter staffing requirements. Training existing employees or hiring appropriately skilled personnel may be necessary. Planning for additional workload on the registry staff during the actual transition is also an important consideration, as they may be operating and supporting the existing registry while working on the transition to the modified registry.

Other issues that should be considered in transition planning relate to governance, ethical concerns, legal matters, data collection, and technology. These issues are discussed in more detail in the following sections.

2.1.4. Governance Issues

Nearly all registry transitions will require an internal and external governance structure to manage and approve changes, whether the transition relates to the scientific objectives of the registry, technology changes, or data access. The transition team is one important component of the governance structure. Chapter 2, Section 2.6 reviews the governance considerations for the planning of a new registry, many of which are relevant to a registry transition. Some additional considerations are addressed below.

• Scientific advisory board governance during the transition: Many registries have scientific advisory boards that oversee the conduct of the registry. These boards may also play a role in governance during a registry transition and provide external perspective for the considerations and future objectives for a registry transition. Membership of the scientific advisory board should be reviewed to ensure the key stakeholders that are involved in the transition are represented. During the registry transition, the scientific advisory board can also act as an advocate of change by publicly supporting the transition and helping to engage and motivate clinicians at the participating centers. External stakeholders, such as patient advocacy groups and regulatory agencies/health authorities, may also be informed of the transition and, depending on the goals of the transition, potentially enlisted as additional public advocates for the registry transition.
• Governance of data access: Registry transitions will also require revisiting the registry's data access policies and procedures. If a data access committee is already in place, the committee should be charged with (1) determining how changes in the registry will affect the policies and procedures for accessing data, and (2) reviewing the operational plan for executing analysis plans with respect to the registry transition. Furthermore, if the transition involves a change in registry stakeholders, the procedures for conducting analyses and developing publications should be re-examined. New stakeholders may need to be involved in the prioritization of analysis plans, conduct of analyses, and/or the review of scientific abstracts and manuscripts.

2.1.5. Ethical and Legal Issues

The major ethical and legal issues for registries focus on data privacy, patient confidentiality, and ownership of and access to the data. These issues, covered comprehensively in Chapter 7, should also be carefully considered during a registry transition. It is important to note that interpretations of the pertinent laws and regulations are numerous and varied, leading to inconsistent application among institutions, which may affect multicenter registries.¹ Hence, input from legal counsel and regulatory authorities should be sought when planning a registry transition. Some common legal or ethical concerns that may arise during registry transitions are reviewed here.

An early step in the registry transition planning process is consideration of the need for institutional review board (IRB)/ethics committee (EC) review. If the purpose of the registry is unchanged and no new data are being collected, IRB/EC review may not be necessary—subject to ethical guidelines and the requirements of the individual IRBs/ECs. However, IRB/EC review would likely be required in certain transitions; for example, if new data will be collected through contact with patients, if the new data that will be collected includes identifiable personal information, or if the data will be used in a different manner than previously communicated to patients (45 CFR §46.102(f)).

Especially when the purpose of the registry has changed, a registry transition may involve extending the followup period of the initial cohort. In these circumstances, recontacting patients or using their identifiers may be necessary to collect the longer-term data. For example, a cardiac assist device registry may have been established initially to determine perioperative safety. However, new safety concerns associated with longer term implantation may prompt a change in the purpose of the registry. Medical records, death indices, and patient interviews may be required in order to collect the longer-term followup data. This new data collection effort would likely require IRB/EC review.

Consideration should also be given to whether any changes will be required in the informed consent process (e.g., obtaining revised consents from existing subjects, obtaining new consents for registries that do not currently have such consents). If consent was obtained for registry participation initially, re-consenting may be needed, especially when the registry transition will result in (1) longer or otherwise different followup than what was originally agreed to by patients, (2) direct contact with patients to obtain new data, (3) collection of biological samples or linkage of existing specimens to registry data, (4) the use of data from deceased participants, or (5) linkage of the participant's data to other databases. If the planned registry modifications involve patients for whom the feasibility of obtaining consent would require unreasonable burden or situations where the consenting process would potentially introduce an unacceptable level of bias,^2-4 discussions with local IRBs/ECs should be undertaken to see if the consent can be waived. Chapter 8 discusses these issues in more detail.

2.1.6. Data Collection

A major component of the registry transition project plan should be a thorough evaluation of current and future data collection needs. The project plan should allocate time for epidemiologists and clinical experts to jointly review the current registry case report form (CRF). It is of paramount importance that the relevance of the current set of data elements is reviewed, in light of what is known about new hypotheses to be tested. During this review, some data elements may deemed irrelevant and may not be required moving forward. When considering the collection of additional covariates and outcomes, particular attention must be given to balancing the scientific relevance of the new data elements with the logistical burden on participating centers.

Additional considerations may arise if a registry transition involves one of the following specific circumstances:

• Collection of Biological Samples: Biobanks, defined as facilities that store biological material (e.g., serum, genomic material, pathology specimens) from humans, are increasingly popular additions to registries.⁵ The addition of a biobank raises many logistical issues, which are outside the scope of this chapter. However, it should be noted that the addition of a biobank will likely require changes in the informed consent. Some biobanks have used general consents to cover future analyses of the biological material and integration into the registry, but there is significant concern about these broad consent documents. Some commentaries on this issue have suggested that such broad consents are more appropriate when limited to a specific disease entity, thereby allowing for studies examining diagnosis, mechanisms of disease, risk factors, and treatment outcomes.^6-8 Chapter 8 discusses these issues in more detail.
• Pediatric Registries: If a registry enrolls pediatric participants and the registry transition involves extending the followup period, consideration should be given to whether participants need to give consent when they reach an eligible age. This is particularly important for those registries that plan to add a biobank or link to other databases as part of the transition process. There is considerable debate regarding the ethics of parents enrolling their children in research studies. More discussion on this topic can be found in Chapter 7. It is also important to note that for all registries, the right to withdraw is inherent;^{8, 9} see Chapter 8.
• National to International Registry: Some registry transitions may extend the geographic scope of a registry. For example, a U.S.-based registry may add participating sites in Europe. When the registry scope extends beyond national borders, additional ethical and legal concerns must be addressed. Each country may have different legislation and restrictions for the collection and processing of subject information and its use for research. Adequate time and additional resources to investigate these requirements should be factored into the project plan. Moreover, if Federal funds are used in the registry transition, additional steps may be involved in the expansion of the registry. In particular, some registries may be collecting data on vulnerable international populations for which additional privacy protection safeguards may be necessary. Federal guidelines for performing international research should be consulted as part of the planning process.

2.1.7. Data Ownership and Licensing

A number of scenarios exist in which ownership of registry materials must be delineated, including the ownership of the interface, platform, infrastructure, and data. During a registry transition, particularly one involving a change in stakeholders, a careful review of agreements or contracts should be performed to determine if modifications are needed. In some cases, the registry transition may involve moving data from one platform to another. Hence, data ownership may need to be clarified. For example, a professional organization may determine that the vendor maintaining its registry is performing below expectations and may select a new vendor to house and run the registry. Depending on the terms of the prior agreements, it may or may not be possible to import the historical data into the new vendor's system.

Registry data are often collected using electronic or paper CRFs that may have intellectual property protections, including copyright, trademark, and patent. If continued use of these forms is planned, measures should be taken to ensure that the appropriate permissions for use are still applicable when the registry transitions.

2.1.8. Data Access

In addition to data ownership, ongoing data access is an important consideration. The new and ongoing registry stakeholders should consider whether the previous stakeholders should have access to the previously collected data as well as to the data collected in the future. Federal and academic stakeholders may need to execute technology transfer agreements (e.g., material transfer agreements) or other contractual agreements in order to access the data.

2.1.9. Changes in Funding

Registry transitions may also include changes in funding. For example, a registry that was initially funded through a government grant may be transitioned to a professional association or industry partner. When funding sources change, the role of the funding entities should be clearly delineated to ensure that there is no real or perceived threat to privacy or data confidentiality.

In some cases, a change in funding may require contract modifications in anticipation of potential conflicts between the new stakeholders and the remaining stakeholders. For example, industry may elect to partially fund a registry that is also receiving Federal funding from a regulatory agency. Contracts may need to be modified to clearly delineate how each set of funds will be spent. The new Chapter 24 on public-private partnerships provides more information on these issues. As with all contracts involving Federal funds, attention should be given to regulations governing their appropriate use. Additionally, changes in funding may alter the locus of legal rights and obligations. It is important to have unambiguous conversations with stakeholders and associated contractual agreements that clearly delineate the rights of the funding entities.

When data are transferred from one owner/sponsor to another, the liability associated with the protection of subjects' information should be clarified. Consideration should be given to indemnification clauses in data transfer agreements. Oftentimes, the data transfer agreements detail that the new sponsor of the registry will accept all liability for use of the data previously collected by the transferring sponsor. The data transfer agreement should also contain a clause that the new sponsor agrees to use the data properly. In these circumstances, the liability would be assumed by the new sponsor if a breach of information occurred whereby subject-level information was relayed to an outside party. If the new sponsor is a Federal entity, however, there are regulations that prohibit the Federal Government from indemnifying others (e.g., Anti-Deficiency Act).

2.1.10. Contracts With Vendors

Issues may arise with vendors (including inadequate performance of duties, loss of financial solvency, or escalating cost of renewing the contract), necessitating a transition to a new vendor. In light of these potential outcomes, it is necessary to draft contracts that consider these scenarios and contain provisions to address them. For example, if a registry is being transitioned to a new, fledgling company, consideration should be given to establishing an escrow account for the registry. This account would cover the cost of ensuring that the data remain accessible to the sponsoring body. Moreover, it would prevent the registry from being part of the estate if the company is unable to meet its contractual obligations. Establishing the escrow account would increase the cost of the initiative for the sponsor, the vendor, or both, and should be considered when planning the transition. In addition, contracts should contain explicit clauses that guarantee the transmission of data to a new vendor when the contract expires or if the vendor defaults on the contract.

2.1.11. Technology Considerations

A registry designed to collect long-term followup data will inevitably undergo technology changes. Platforms for electronic data capture (EDC) may be upgraded (for example, through version updates within a system), or the registry sponsor may select a different third-party vendor to host the EDC system. Upgrading the EDC system and technology platform may enable more frequent data entry from participating centers, rather than annual or semi-annual data reporting under previous technology environments. Such changes have implications for training plans for participating centers (see Section 2.2.2, Training). Technology considerations relevant to linkage of a registry to an electronic health (medical) records (EHR/EMR) database or other database and for collection of patient-reported outcomes are covered in Chapters 15 through 18. In transitioning to a new registry technology platform, it is important to clearly define software requirements in order to avoid design flaws, which are costly to correct after project completion. Soliciting input from various stakeholders (e.g., data entry personnel, clinical experts, data analysts) may be helpful to validate the proposed design of the new registry. The proposed design should be presented to them in an easy-to-understand format (e.g., a prototype) rather than a detailed requirements document, which may be more difficult to comprehend. Setting aside time for user-acceptance testing or pilot testing may also be useful to identify issues before the transition is complete.

One of the earliest and most important decisions in transitioning to a new technology platform is whether to develop the platform in-house or use an external vendor. Each approach has advantages and disadvantages. The in-house approach requires personnel with the appropriate expertise and the infrastructure to support such a project. Development tools widely used by software companies should be employed, if possible, to mitigate the risk of experiencing shortages of qualified personnel for ongoing support and maintenance of the application. Organizations that do not have the internal resources and expertise to develop a registry application in-house usually turn to external vendors. Selecting a registry vendor is an important strategic decision for an organization, particularly for sponsors who anticipate operating the registry for many years. Some factors that should be considered in selecting a registry vendor are outlined in Table 14–1.

Table 14–1

Considerations in selecting a registry vendor.

Once a vendor has been selected and the features of their technology platform are known, it is important to assess the hardware, software, and browser configurations at the participating sites, as these may affect performance of the registry application. It is also important to ensure that the participating sites have access to the optimal configurations on which the application has been tested and validated. Requesting a technology contact person at each of the participating sites may be helpful to facilitate working through these issues during the transition.

Another technology consideration is transitioning personnel involved in data entry at participating sites from an existing registry to the new registry. This requires an analysis of security levels in order to transfer users to the appropriate permission level in the revised registry. In some cases, users can be transferred electronically from the existing to the new registry application, but in other cases, they must be added manually. The transition team must develop a plan for accomplishing the transfer that minimizes the effort at the participating sites, but ensures that only valid users can access the registry at the appropriate permission level. Of note, a registry transition provides an opportunity to assess the activity level of users at the participating sites and their ongoing need to access the registry.

A final technology consideration pertinent to a transition relates to the closeout of an existing registry. Generally, the closeout should be scheduled well after the anticipated launch of the new registry, as timelines on such complex projects are often delayed. The existing registry may also be useful in validating successful data migration into the new registry.

2.2.1. Communication

Communicating with all stakeholders is critical during a registry transition. The transition team should develop a communication plan that defines who is responsible for communicating what and to whom. The frequency and mode of communication should be established with a particular sensitivity to key stakeholders. Since the registry transition will likely disrupt workflow at the participating sites, communicating the rationale for the change, the timeline, and the impact on users is important. Any change in expectations or incentives for participation should be fully explained. It is important to anticipate and respond to questions and concerns from participating sites, knowing that change can lead to stress and anxiety. In most circumstances, the communication plan will focus on retaining participating sites through the transition. However, a registry transition provides an opportunity to evaluate participating centers to decide whether all of them should be retained. A transition may also be an ideal time to recruit additional sites.

2.2.2. Training

The development and implementation of a robust training program prior to the registry transition will facilitate the rollout of the revised registry and improve the quality of data collected. Training needs will vary according to the scope of the registry transition. For example, a technological change that affects the user interface, functionality, and/or organization of the data elements will likely require extensive training, whereas a transition related to a change in purpose with minimal impact on data entry should require less training. When developing a training program, the key elements of adult learning theory¹⁰ should be kept in mind, and several questions should be addressed:

• Who is the intended audience? Determining the audience will have a significant impact on the design and implementation of the training program. For example, internal staff training will differ from that of external registry participants, and the training program for clinicians will likely differ from that designed for data entry personnel.
• What are the learning objectives? The learning objectives should drive the development of the curriculum. What do the people involved in the registry need to know to be successful during and after the transition? The focus should be on what will change and why, and what the impact of the changes on registry participants will be.
• What information is needed to meet the learning objectives? High-level overviews and detailed documents are useful to help participants with varying levels of interaction with the registry understand the changes. The creation of a reference guide that clearly describes what changes were made and why each change was made will be extremely helpful to some registry participants.
• What are the best mechanisms for disseminating the information? People respond differently to various learning environments and techniques. Depending on the size of the registry, training may be offered in various ways, some of which are described below:

  –

  Conference calls can be effective for smaller groups and allow for open discussion.

  –

  Webinars can be useful when larger groups are involved and the training activity includes visual presentation.

  –

  Face-to-face meetings are frequently effective since the learner is less likely to be distracted.

  –

  One-on-one training sessions are usually well received, since the training can be customized to the individual learner. However, this approach is costly.

  –

  User's guides, manuals, FAQs, and other documents can be posted on a Web site, or hard-copy materials can be distributed to participants.

• What is the best approach to ensure that learning has occurred? It is important to confirm that the training program is successful, in order to avoid issues with retention and data quality after the transition launch. Learning assessment approaches include tests (e.g., the completion of a sample data collection form or other task), surveys, and direct feedback. Feedback from the learning assessments should be incorporated into the training program, as needed. Pilot testing may also be useful for refining and strengthening the training program before launch.

2.2.3. Supporting Participants Through the Registry Launch

In addition to a robust training program, sufficient personnel and resources should be assigned to respond to input and inquiries from registry participants following the launch of the revised registry. Accessibility of the support team is very important during this critical period of the transition. Planning for the registry launch should delineate how users can submit questions or concerns (e.g., by email or by calling a support desk), who will be the first responders, and how complex issues will be escalated for further evaluation. Many straightforward questions (e.g., problems logging on) can be resolved quickly and efficiently. However, it is important to carefully assess all input from participants since they may uncover problems with the revised registry that have been missed during testing. Such problems may require immediate attention not only from support personnel, but also from the developers of the registry application. At some defined point in time (e.g., 1 to 3 months after launch), a broader analysis of all of the questions and comments from participants may be helpful in prioritizing any further changes to the registry.

2.2.4. Data Management

Technological changes may require changes to the database/data warehouse used to store the registry data. Database or data warehouse transfers are complex processes that involve a number of steps, including creating a new database layout, mapping the legacy data to the new database layout, and transferring the data with rigorous quality controls to ensure that the transfer is successful. Database transfers also need to be conducted in accordance with any regional IRB or EC approvals to ensure that the privacy of any patient-level data is maintained. The size and complexity of the registry as well as the extent of the changes in the CRFs will determine the complexity of the data mapping process. The data fields known to users of the registry might be collected in different contexts (e.g., with added specificity or new dependencies between data elements on CRFs), and these differences must be considered in the data mapping process. Relatively small changes in the wording of a question on the CRF, or creating an additional category on an existing item (e.g., expanding categories of ethnicities) may introduce ambiguities in the mapping of the existing data set to the new environment. In other instances, significant changes to the definition of an outcome variable will typically require review and adjudication of prior cases to establish longitudinal consistency across the data set (for further detail, see Section 2.3.3. below). For these reasons, input and evaluation of the impact of the migration on future registry outcome analyses from subject matter experts, including epidemiologists and clinical experts, along with documentation of decision rules that were established during the epidemiological and clinical review, will be needed in the data mapping process. The effort and expense involved in the data migration is often underestimated and adequate time must be allocated during the project planning and in establishing timelines. Despite careful attention to detail, this activity often becomes an iterative process, with data mapping, data importation, and quality control checks that lead to corrections in the data mapping, re-importation of the data, et cetera.

Many practical issues should be considered when transferring a database. First, it is important to document the rationale for adding, modifying, or deleting data fields, so that this information can be communicated to stakeholders and registry participants. Second, careful consideration should be given to the future impact of changes. Certain changes may make it difficult to link prior data sets with the new data sets. For example, adopting a new, broader definition may mean that data can only be linked in one direction, as shown in Table 14–2.

Table 14–2

Impact of definition changes on data linkage.

When making changes to the data structure, the following questions should be considered:

• Will existing queries (i.e., questions raised by a data manager and issued back to the participating centers regarding a data entry issue) need to be rewritten for the new data set?
• Will existing reports (e.g., percent of patients with a laboratory value above a certain number) need to be revised for the new dataset?
• Will more server space be needed to house the data?
• How can the impact of the changes on the processes affected by the new data structures be minimized?

It is also important to determine what metadata (e.g., long name, short name, data type/data format, and permissible values) are important to capture for each field and how the transition will affect the metadata.

2.3.1. Changes in Cohort Definition

A registry transition may involve a change in the inclusion or exclusion criteria for patient participation, thus shifting the definition of the study cohort. These changes can occur under a number of scenarios, such as if the registry moves from a disease-based cohort (i.e., no inclusion criteria for receiving a particular treatment) to focusing on a cohort of patients with the disease who receive a specific therapy or class of therapies (i.e., inclusion criteria now requires patients to be receiving a treatment). Cohort definitions may also change based on geography (e.g., if a registry transitions from a national to a global catchment area). This introduces the possibility of geographic differences in disease severity or treatment patterns, which may require thorough documentation of baseline clinical status in order to stratify or perform covariate adjustment, if necessary.

Other changes in the cohort definition may occur if the registry transitions from having broad participation by centers to a limited set of centers (e.g., physicians who are associated with large specialty care clinics). A registry transition that results in such a change in the cohort definition has the potential to introduce selection bias into the registry by focusing the enrollment and ongoing followup of subjects on a potentially more severely affected group of patients. As enrollment and followup occur, epidemiologists should be actively involved to assess whether selection bias has been introduced. Comparisons of demographic and baseline clinical features of subjects before and after the transition may be sufficient to assess the degree of bias introduced and to understand which factors or variables can be considered for stratification or covariate adjustment. Advanced methodologies such as comorbidity indices or propensity score analyses may be necessary to adequately adjust for the changes in the cohort over time.

2.3.2. Introducing New Data Elements

As scientific advances improve the understanding of a particular disease or new treatments become available, new hypotheses will likely be formed. In order to test new hypotheses, it may be necessary to add data elements and/or refine the definition of existing data elements. When adding new data elements, common data elements and validated instruments should be used when possible. (See Chapter 4.) it may be necessary to validate the new data elements, through source document verification of the original medical records, laboratory tests, or diagnostic reports. Results of source document verification may show there are discrepancies in the accuracy of new data elements being captured. For example, investigators interested in collecting data on heart failure as an outcome may find variation in how the definition of heart failure is applied across contributing centers. While the refinement of definitions for data elements occurs, analyses on the outcome variables may still take place. However, methods of quantitative sensitivity analysis may be necessary to understand the degree to which misclassification of variables may introduce bias into the analytic results. Results of source document verification efforts can be used as inputs into quantitative sensitivity analysis to directly estimate the sensitivity and specificity of the outcome variable.

2.3.3. Impact on Existing Cases of an Outcome

A registry transition may lead to redefining an outcome in order to increase sensitivity and specificity. For example, a registry that has been collecting data about the onset of Parkinson's disease as an outcome measure may transition to more stringent inclusion and exclusion criteria. Although this may result in increased validity of the outcome, the statistical power of the analyses from the registry may be compromised, as there will likely be fewer patients meeting the case definition going forward. Patients who have already been identified in the registry as cases may require re-evaluation (and possibly re-adjudication) to determine if their clinical scenario fulfills the revised selection criteria.

Figure 14–1 shows the potential impact of a change in an outcome (e.g., case definition of Parkinson's disease) following a registry transition. Note that the smaller cohort size following the registry transition may reduce statistical power and cases that met original case definition may require re-evaluation.

Figure 14–1

Potential impact of a change in outcome.

2.3.4. Impact of Patient-Reported Outcomes

Registries frequently include patient-reported outcomes, such as those reported on the SF-36^® health survey, or activities of daily living. It is important to note and characterize whether the type of patients who are reporting self-assessments to the registry and their disease severity or outcome status are changing over time because of the transition. If such instruments are introduced during a registry transition, patients may begin to preferentially recall events, which can lead to a bias in the outcomes. In addition, if the registry transitions from a purely disease-based registry to a therapy- or product-based registry, patients who become aware of this change may begin to report their health status more or less favorably.

Table 14–3 illustrates the possible consequences of transitioning from a disease-based registry to a focus on patients with the disease exposed to a particular therapy. Prior to the transition, the risk of the outcome among exposed and unexposed patients was similar. Following the transition, there are more exposed patients, and, for the purposes of illustrating the impact of bias, assume awareness of the registry transition results in exposed patients preferentially reporting onset of a particular outcome. Because of this preferential report, the risk is approximately 25 percent greater among the exposed as compared with before the transition. The apparent risk ratio is now 1.46 comparing exposed to unexposed.

Table 14–3

Possible consequences of a change in registry focus.

2.3.5. Comparative Effectiveness Analysis

A registry may transition from a disease-based cohort to one that is focused on specific treatment(s) in order to establish comparative effectiveness studies between multiple treatments. A greater emphasis on baseline covariate data may be required in this situation, and epidemiologists should be involved to identify the key variables that would account for differences in disease severity among the treatment groups in order to mitigate biases such as confounding by indication. Epidemiologists must also be involved in planning the statistical analysis, which may require matching or other multivariate statistical techniques.

2.3.6. Biostatistics and Statistical Power

Statistical power must be considered in registry transitions that lead to changes in the size of the cohort and/or the extent of followup. For example, a transition that focuses the registry on a smaller number of participating centers may diminish the number of new enrollees, but provide the benefit of an extended length of followup. The transition may eventually provide a greater number of exposed patients who develop the outcome(s) of interest. Biostatisticians should be involved in assessing the impact of changes in cohort accrual on statistical precision of the analyses. Previously specified hypotheses of interest may no longer be testable from the standpoint of statistical power. Alternatively, consideration of statistical power for newly specified hypotheses following the transition may provide an assessment of the extent of enrollment and followup required for robust future analyses.

3.1.1. Stopping an Experiment

The principles regarding rules for stopping a study mostly stem from the need to consider stopping an experiment. Because experiments differ from registries in crucial ways, it is important to distinguish between the issues involved in stopping an experimental study and in stopping a nonexperimental study. In an experiment, the patient's treatment is determined by the study protocol, which typically involves random assignment to a treatment regimen. In a nonexperimental study, patients are treated according to the treatment protocol devised by their own clinicians, typically uninfluenced by the study. In a randomized trial of a new therapeutic agent or a field trial for a vaccine, the size of the study population is ordinarily set in the study protocol, based on assumptions about the expected or hypothesized results and the study size needed to reach a reasonable scientific conclusion. Ordinarily this planned study size is based on power calculations, which require as input the criteria for statistical significance, the effect size anticipated, the baseline occurrence rate of the study outcome, and the relative size of the study arms. Because of inherent problems in relying on statistical significance for inference,^{11, 12} the study size preferably will be planned around estimation of effect and the desired level of precision. In a study intended to provide some reassurance about the safety of an agent, the study size may be planned to provide a specific probability that the upper confidence bound of a conventional confidence interval measuring an adverse effect would be less than some specified value, given a postulated value for the effect itself (such as no effect). In the latter situation, if no effect is anticipated, a power calculation is not only unreasonable but is not even possible, whereas planning a study on the basis of precision of estimation is always possible and always reasonable.

Stopping an experiment earlier than planned is an important decision typically made by an advisory group, such as a data safety and monitoring board, which is constituted to monitor study results and make decisions about early stopping. In a biomedical experiment, the investigator has a greater ethical obligation than in a nonexperimental study to safeguard the well-being of study participants. This is because the investigator is administering an intervention to study participants that is expected to affect the probability that study participants will experience one or more specific health outcomes.

Equipoise is a widely accepted (but, unfortunately, not universally accepted) ethical precept regarding human biomedical experimentation.¹³ Equipoise requires that at the outset of the study, the investigator has a neutral outlook regarding which of the study groups would fare better. A strict interpretation of equipoise requires each of the study investigators to be in a state of equipoise. An alternative view, referred to as “clinical equipoise,” is that equipoise can be achieved at the group level, with the enthusiasm of some investigators for the prospects of the study intervention being balanced by the skepticism of others.¹⁴ Whichever interpretation of equipoise is adopted, most investigators agree that if equipoise becomes untenable as study results accumulate, the study should be stopped to avoid depriving some study participants of a potential benefit relative to what other participants receive.

For an advisory board to decide to stop a study early, there must be solid evidence of a difference between the groups before the planned study endpoint is reached. Such stopping decisions are usually based on ethical concerns, as scientific considerations would seldom dictate an early stop to a study that had been planned to reach a specific size. Advisory boards must base stopping decisions on analyses of accumulating study data, which are usually formally presented at regular meetings of the review board. Statistical concerns have been raised about biases that can arise from repeated analyses of accumulating data.¹⁵ To offset these concerns, many experiments are planned with only a limited number of interim analyses, and the interpretation of study results takes into account the number of interim analyses.

3.1.2. Stopping a Fixed-Length Nonexperimental Study

Like experiments, most nonexperimental studies also have a fixed time for their conduct and a planned size that reflects goals analogous to those in experimental studies. Nevertheless, the ethical concerns that motivate stopping an experiment before its planned completion do not have a direct counterpart in nonexperimental studies. Nonexperimental studies have ethical concerns, but they relate to issues such as data privacy, intrusive questioning, or excessive inducements for participation rather than to concerns about intervention in the lives of the participants. Although it is theoretically reasonable that an investigator could choose to stop a nonexperimental study for ethical reasons, those reasons would presumably relate to ethical problems that were discovered in the course of the study but were unrecognized at the outset rather than to an early conclusion regarding the study goal. For example, the investigator in a nonexperimental study could learn from an interim analysis that the association between the exposure and the outcome under study was much stronger than anticipated. Unlike the experimental setting, however, the investigator in a nonexperimental study is not administering the exposure to any of the study subjects and thus has no responsibility to the study subjects regarding their exposure.

The discovery of an ethical problem during the conduct of a nonexperimental study is therefore possible but extremely rare. Because the findings from an interim analysis should not lead to discontinuation of a nonexperimental study, there is little motivation to conduct interim analyses for nonexperimental studies that have been planned with a fixed size and period of execution. If there is some considerable time value to the findings, such as to inform regulatory action, it might be worthwhile to conduct an interim analysis to get an early appraisal of study findings. Unless there is an appropriate outlet for releasing interim findings, however, it is possible that early findings will not circulate beyond the circle of investigators. In most circumstances, such analyses are hard to justify in light of the fact that they are based on a smaller amount of data than was judged appropriate when the study was planned; thus the originally planned analysis based on all the collected data will still need to be conducted. Unless there is a clear public health case to publicize interim results, journal policies that require that data to be published have not been previously published may inhibit any release of preliminary findings to news media or to journals.

3.1.3. Stopping an Open-Ended Study

Although patient registries may be undertaken with a fixed length or size, or both, based on study goals relating to specific safety or efficacy hypotheses, many such studies are begun as open-ended enterprises without a planned stopping point. For example, patient registries without specific hypotheses may be undertaken to monitor the safety of patients receiving a novel therapy. The Antiepileptic Drug Pregnancy Registry, established in 1997, is an example of an open-ended registry that focuses on a set of specific endpoints (congenital malformations) among a subset of patients (pregnant women) taking a class of medications (antiepileptic drugs).¹⁶ It has no fixed stopping point.

Measuring the frequency of rare endpoints demands large study sizes. Therefore, a monitoring system that includes rare endpoints may have to run for a long while before the accumulated data will be informative for low-frequency events. On the other hand, the lower the frequency of an adverse event, even one with serious consequences, the smaller is the public health problem that a relative excess of such events would represent.

Traditional surveillance systems are intended to continue indefinitely because they are intended to monitor changes in event frequency over time. For example, surveillance systems for epidemic infectious diseases provide early warning about outbreaks and help direct efforts to contain such outbreaks. In contrast, a patient registry is not a true surveillance system, since most are not intended to provide an early warning of a change in outcome frequency. Rather, most patient registries are intended to compile data on outcomes associated with novel treatments, to supplement the sparse data usually available at the time that these treatments are considered for approval by regulatory agencies. For example, a regulatory agency might mandate a patient registry as a condition of approval to supplement safety information that was submitted during the application process.

How long should such a registry continue? Although it is not possible to supply a general answer to this question, there is little reason to support a registry continuing indefinitely unless there is a suspicion that the treatments or treatment effects will change over time. Otherwise, the time should come when the number of patients studied suffices to answer the questions that motivated the registry. The Acyclovir Pregnancy Registry, which began in 1984, was stopped in 1999. Its advisory committee concluded: “The [Acyclovir Pregnancy] Registry findings to date do not show an increase in the number of birth defects identified among the prospective reports [of exposures to acyclovir] when compared with those expected in the general population. In addition, there is no pattern of defects among prospective or retrospective acyclovir reports. These findings should provide some assurance in counseling women following prenatal exposure [to acyclovir].”¹⁷ The consensus was that additional information would not add materially to the information that already had been collected, and thus the registry was closed down.

To avoid uncertainty about the fate of an open-ended study, it would be sensible to formulate a specific goal that permits a satisfactory conclusion to data collection. Such a goal might be, for example, the observation of a minimum number of specific adverse events of some type. Even better would be to plan to continue data collection until the upper bound of a confidence interval for the rate or risk of the key outcome falls below some threshold or until the lower bound falls above a threshold. Analogous stopping guidelines could be formulated for registry studies that are designed with a built-in comparison group.