Chapter  159:  Collection and Use of Cancer Family History in Primary Care

Primary Care Physicians and Cancer Risk Assessment and Management

Primary care providers (PCPs) have always used family history information as a core tool for their practice,⁵ well before the arrival of the “genomics age”. However, the increasing emphasis on identifying and managing genetic susceptibility, and the question of what might now be considered an “adequate” family history for this purpose, presents real challenges for PCPs.⁶ While a genetics specialist may be able, indeed advised, to devote substantial time to eliciting and confirming family history data (on the order of several hours)^{65, 78, 79} family physicians, internists, and other non-genetics providers may have only minutes. Other barriers to more than a “minimal” approach include unfavorable reimbursement policies, pressure from colleagues and patients to focus on other aspects of care, perceived lack of skills, and lack of confidence.^{3, 80} Conversely, family physicians and other PCPs may be able to capture family history data over time, and are well placed to keep such information up to date.

The use of family history information to make preventive and clinical management decisions also depends on the adequacy of providers' knowledge, skills and confidence; this is extremely challenging in a field where the knowledge base is rapidly evolving. To complement more general educational interventions, there is a strong case for the development of effective tools, designed for use in primary care settings, which permit providers to translate an individual's family history data into meaningful risk stratification, with linkage to evidence-based guidance on appropriate preventive and clinical management interventions. Thus, the translation of family history information into improved health outcomes depends on the availability and integrated use of effective interventions for data capture, risk assessment, and clinical intervention.

Accuracy of Family History Reporting

In order for family history to be of value in clinical decision making, patients must possess, and PCPs must be able to ascertain, accurate family health information. Assessing accuracy requires a clear idea of an appropriate gold standard—what patients “should” know, and what clinicians “should” be able to obtain. In simple terms, an “accurate” family history could be considered to be one which is sensitive (disease in relatives is correctly identified) and specific (lack of disease in relatives is correctly identified). Work in the field of psychiatry has suggested three gold standards for studies of family history taking: an “ideal” standard, based on comprehensive data obtained from the relatives, hospital and physician records and/or disease registers;⁸¹–⁸³ a “best estimate diagnosis” (BED) standard,⁸⁴ based on best available data from death certificates and medical records;^{65, 85, 86} and a “pragmatic BED”, based on the family history obtainable in a detailed interview conducted by a trained clinical genetics professional. Our consultation with the key stakeholders in this review has indicated that an appropriate practical gold standard for evaluating accuracy would be information obtained directly from relatives' medical records, cancer registries, and/or death certificates. Such information should be used both to confirm reported cases of cancer in the family, and to confirm absence of a cancer diagnosis in relatives who were reported not to have cancer.⁸⁷

Collection of Family History in Primary Care

There is no single agreed-upon approach to guide primary care practitioners in taking a genetic family history within office consultations (which are often brief). Family history taking can be conducted as part of a disease specific approach which aims to identify risk of selected single gene disorders (e.g., hereditary breast or colon cancer) for the purpose of ensuring appropriate specialist intervention.^{88, 89} Alternatively, it can be directed more broadly towards identifying possible risk of a number of common multi-factorial disorders such as cancer, diabetes, and coronary heart disease.^{46, 49}

Family history data may be recorded as notes or lists within patient charts, represented as family trees or genetic pedigrees, or stored within computer databases which can be linked to decision support systems. In the last few years several computer-based pedigree drawing packages have been developed, such as genogram software.^{38, 90} It is not clear whether such approaches translate well from specialist use to application in primary care.

There is also no consensus on the extent or detail of family history information which needs to be recorded in primary care, compared with specialist genetics settings. The extent of cancer family history collection has to be adequate to enable PCPs to make appropriate clinical and prevention decisions, but it is not clear whether this necessarily requires the same approach as that used by a genetics specialist.³

Risk Assessment in Primary Care

There are several issues which may influence the translation of family history information into meaningful risk assessment for patients. These include the level of complexity of family history information which is actually required for risk assessment for any given disorder, the validity of risk stratification guidelines or algorithms, the kind of tools that exist to facilitate risk stratification, (and their effectiveness in practice), and the actual predictive value of risk assessment tools (RATs).

At its most simple, assessing familial risks associated with common adult-onset diseases requires setting a threshold where the family history indicates a cause for suspicion (i.e., dichotomizing risk into reassuring the patient or recommending further action). A more complex approach is to separate risk into three or more strata (e.g., “high”, “moderate” and “average”).^{91, 92} In general terms, individuals at “average” risk (the risk level of the general population) would be offered standard preventive advice, those at “moderate” risk would be offered a higher level of intervention, such as more extensive or more frequent surveillance, and those at “high” risk would usually be referred for specialist assessment and possibly considered for mutation testing.²

Risk assessment tools need to be valid, in terms of their clinical predictive value, but they must also be feasible for use in the intended settings, and generate benefits in the process or outcome of care when compared with current practice. Feasibility and effectiveness in practice may be influenced by the actual implementation format; for example, a risk stratification protocol could be presented in paper-and-pencil format, on a personal digital assistant, or on the desktop in a web-based format. Such tools may be passively disseminated, or accompanied by educational interventions and/or ongoing support from genetics professionals. Recent examples of web-based tools include Harvard's “Your Disease Risk”⁹³ and the Centers for Disease Control's (CDC) Family HealthWare.⁹⁴

The ACCE Framework

Tools for family history collection and risk assessment lend themselves to evaluation using the framework developed for genetic predictive testing by the Secretary's Advisory Committee on Genetic Testing.⁹⁵ This framework (see Table 1, derived from Yoon 2003), widely referred to as the “ACCE” framework, comprises four evaluative elements: analytic validity, clinical validity, clinical utility, and ethical legal and social issues.^{2, 96}

Thus, in terms of family history, analytic validity describes the ability of a family history tool to correctly identify the pertinent information on disease in relatives. This is dependent on the effectiveness of a tool in promoting acquisition of appropriate family history data, and also on the ability of an informant to provide accurate information. Clinical validity describes the ability of a RAT to use valid family history data to correctly predict or stratify cancer risk in the informant. Risk assessment tools may vary in their complexity, from simply identifying an elevated cancer risk in the family, to more detailed risk prediction scores—but all are dependent on valid risk stratification criteria. An effective risk prediction tool therefore depends on a valid family history tool, and may or may not also take account of non-genetic factors which modify disease risk. Clinical utility considers the evidence that family history assessment, risk stratification, and subsequent preventive or clinical interventions actually bring overall health benefit to the individual patient. The ethical, legal, and social issues component of the framework considers the impact and consequences of using a family history based approach from a broader societal perspective.

The aim of this review is provide a partial contribution to the evidence base underlying analytic validity (the ability of a tool to capture accurate family history data) and clinical validity (the ability of a RAT to correctly predict disease risk). The main focus is on describing the availability and format of available family history and RATs, and the evidence that these are more effective than current practice in promoting accurate family history collection and assessment in primary care and population settings. It is not within the scope of the review to assess either the evidence underlying risk stratification systems (i.e., the predictive value of guidelines or criteria), or the evidence that preventive or clinical interventions based on such stratification provide overall benefit to patients (i.e., clinical utility). However, the evidence assembled in this review is a crucial element of determining how best to capture and use family history information in primary care to promote the anticipated population health benefits.

Accuracy of Family History Reporting

Accuracy of a test (in this case reporting of family history) represents the proportion of all test results that are true (both positive and negative outcomes). If individuals reporting family history were 100 percent accurate they would correctly identify all relatives with cancer and all those without cancer. A number of metrics may be used to convey accuracy. Of these, sensitivity and specificity are not influenced by the underlying prevalence of the characteristic of interest in the population (in this case, positive family history). We therefore report sensitivity and specificity, where this is reported in (or can be calculated from) eligible papers. Consider the situation where “reporting of family history by the informant” is considered the “test”, and is compared to a “gold standard” (the real situation). In this context, sensitivity indicates how accurate informants are at identifying relatives who truly have cancer. If reporting is highly sensitive, only a few relatives with cancer will be reported as cancer-free. Conversely, if reporting is highly specific, only few relatives who are truly cancer-free are misreported as having cancer.

It is likely that accuracy of reporting will be influenced by both informant factors and factors relating to the method of capturing the family history data. As much as possible, we captured information on such attributes and considered how the results appeared to be influenced by them, although we did not attempt a formal regression analysis to examine their independent effects(s). We also examined reliability (repeatability and reproducibility) where this was possible, recognizing that this is also a product of accuracy of recall and consistency of reporting (informant factors) and performance of the instrument used to capture the data (tool factors). There are several measures of test-retest reliability such as intra-class correlation co-efficient and Cohen's kappa statistic. We note that there is no consensus on the ideal interval for assessing reliability of family history information, bearing in mind that the medical status of relatives inevitably changes over time.

As discussed in Chapter 1, three gold standards have been suggested for studies of family history taking: an “ideal” standard, a “best estimate diagnosis” (BED) standard and a “pragmatic BED” standard. We accepted the following gold standards for the presence or absence of cancer in the first and second degree relatives of the informant: (1) the relative's medical record, (2) confirmation of status by the relative's physician, (3) death certificate, (4) cancer registration, (5) direct confirmation by the relative in question. Ideally, accuracy studies should demonstrate verification of health status (presence or absence of cancer) both in relatives who are reported to have had cancer, and relatives reported not to have had cancer; however, in order to evaluate as wide a range as possible of the available literature, we did not exclude review studies which verified only the status of relatives reported to have had cancer.

We defined a priori what we meant by the degree of the relative. First degree relatives were defined as those who share one-half of their genetic information with the individual reporting family history—their full siblings, parents and children. Similarly, second degree relatives were those who shared one-quarter of their genetic information with the informant—their grandparents, grandchildren, uncles, aunts, and half-siblings.

Family History Collection Tools

We defined a FHxT as:

“A systematic and coherent approach used to capture and document family history, appropriate for the clinical setting, with the potential to lead to decision making by a clinician.”

This review focused on FHxTs which could be applied in the clinical setting, but we also included studies that described tools developed for research purposes, and for settings other than primary care, where we judged they appeared potentially applicable within primary care settings. We captured data on the following tool characteristics that may influence the clinical utility of the tool in current primary care practice.

• 1

  Patient targeting—“reactive” or “proactive”.

  • Reactive—the tool was intended to be used only to collect family history information from individuals with perceived or recognized familial risk of cancer, including individuals concerned about cancer risk.

  • Proactive—the tool was intended to be used to collect family history information from a general or targeted population coming into contact with primary care, irrespective of a known cancer risk or concern.

• 2

  Study setting in which the FHxT is being administered—“clinical” or “research”.

  • Clinical—the primary objective of the study was to assess the use of the FHxT in routine clinical practice.

  • Research—the primary objective of the study was to use the FHxT for purposes other than routine clinical practice, for example designed for data capture in epidemiological studies.

• 3

  Type of comparator—“best estimate” or “current practice”.

  • Best estimate—the comparator was information collected by a clinical genetic specialist interview or equivalent.

  • Current practice—the comparator was information collected in a way that was “standard” for the primary care setting, e.g., family history information recorded in patient charts.

Where a tool was not described as designed for or evaluated in a primary care setting, applicability was assessed by two independent reviewers against five criteria: length of tool, ease of completion, need for specialist knowledge, whether it was designed to capture data on at least all first degree relatives, and clarity of layout (including appropriate structure and logical sequence).

Risk Assessment Tools

While there is no one commonly accepted definition of a RAT, for the purposes of this study, we have followed the approach of Liu et al. who define a decision tool as:

“...an active knowledge resource that uses patient data to generate case specific advice, which supports decision making about individual patients by health professionals, the patients themselves or others concerned about them.”⁹⁷ (p90)

Defined thus, RATs have four essential characteristics:

• 1

  The tool is designed to aid a clinical decision by a health professional and/or patient (“user”);

• 2

  The tool focuses on decisions concerning individual patients (“target decision”);

• 3

  The tool uses patient data and knowledge from family history to generate an interpretation that aids clinical decision making (“knowledge component”);

• 4

  The tool is designed to be used before the health professional or patient takes the relevant decision (“timing”).

This definition encompasses a wide range of potential tool “technologies”, including computer-based decision support systems, reminder cards, guidelines, predictive scores, checklists, etc. Drawing on this definition, we have developed the following working definition of a “family history based cancer risk assessment/decision tool”, for use in this review:

“An active knowledge resource that uses family history data and other relevant evidence to generate case specific advice [knowledge component], designed to support decision making relating to management of cancer risk in individual patients [target decision component, timing component], by health professionals, the patients themselves, or others concerned about them [user component].”

We translated the four “essential characteristics” into this specific form for the context of this review:

• 1

  Users—health professionals, patients, members of the general population

• 2

  Target decision—clinical management (e.g., referral for genetic counseling), or individualized preventive management strategies (e.g., disease screening or surveillance)

• 3

  Knowledge component—a defined model or set of criteria which transform family history data into information which serves the target decision making process

• 4

  Timing—designed to be used before the health professional or patient takes the relevant decision.

The breadth of this definition potentially allows for the inclusion of a large number of guidelines, algorithms, statistical models, etc. In order to maintain the focus of this review on tools most likely to be feasible for use in primary care, we included only those which were explicitly developed for primary care, or where specialist genetics knowledge did not appear necessary to use the tool. We excluded tools where the only output was risk of carrying a cancer-associate mutation (e.g., BRCAPRO⁹⁸ or BOADICEA⁹⁹), rather than risk of disease, as we judged this required genetics specialist knowledge for interpretation. Noting also that there are many hundreds, possibly thousands, of guidelines which have been developed over the past few years around familial cancer risk, we included them only if they were part of a package, system, or intervention designed to foster their effective implementation in practice. Thus, widely used guidelines such as the modified Amsterdam criteria,¹⁰⁰ the Manchester scoring system,¹⁰¹ the UK NICE guidelines on familial breast cancer⁷² were not included unless they were part of such a system. For each tool which met the inclusion criteria, we collected data on the guideline(s) or evidence cited which appeared to form its knowledge component.

Topic Refinement

The first step during the topic assessment and refinement process was a teleconference with partner organizations. The Task Order Officer (TOO) invited topic experts and the McMaster multidisciplinary research team to define the scope of the topic to be addressed and to refine/clarify the preliminary research questions for this evidence report. An international Technical Expert Panel (TEP) was assembled to provide high level content expertise on this topic (Appendix E ^*) and to participate in conference calls on an as-needed basis throughout the data refinement and extraction phase. The TEP assisted in refining the research questions and raising methodological issues of relevance to this review.

The initial work order specified that the systematic review should be limited to adult populations and should examine the family history of at least one of the following cancers: (1) breast, (2) ovarian, (3) prostate, and (4) colorectal. The second and third questions of the review were limited to primary care settings or practitioners.

The first research question in this systematic review focuses on the accuracy of family history knowledge and reporting. The investigative team considered, but ultimately rejected, addressing this question by updating a previous systematic review.¹⁰² This review included original articles describing the accuracy of self-reported family history for breast, colon, ovarian, prostate, endometrial, and uterine cancers using verification from identified relatives' medical records, physician, death certificate, and/or verification within a population cancer registry. The limitations of this review included: lack of a delineated search strategy, overly specific search terms, non-reporting of agreement between reviewers, non-reporting of data collection forms used, and lack of clarity of reasons for excluding reports.

A number of issues relevant to the identification and evaluation of FHxTs were identified and discussed with the TEP, including: (1) the most important attributes that should be considered within each of these tools; (2) which of these elements were most relevant for primary care; and (3) the incremental value of the tool relative to current practice. The TEP recognized that the selection of gold standards for family history reporting and collection is arbitrary and that an “adequate” family history (for the purposes of making decisions relating to familial cancer risk) requires not only identifying relatives with and without the cancer, but also the relationship of the affected relative, the age of onset of cancer in those affected, and identification of several cancer types beyond the “target” cancer in question (e.g., family history of endometrial and kidney cancer is relevant in considering risk for hereditary nonpolyposis colorectal cancer).

For the purposes of the review, a definition of primary care was established with the participation of the partner at the CDC and the TEP. Primary care practitioners included family physicians/general practitioners, general internists, obstetricians, gynecologists (obstetrics and gynecology practitioners are PCPs for some women), nurses, nurse practitioners, physician assistants, nutritionists, behavior counselors.

Family history information is of clinical value only if it can be used for some form of meaningful risk stratification. Issues around risk assessment and stratification were explored with the TEP, particularly whether the various risk stratification algorithms or guidelines on which tools are based are themselves evidence-based—i.e., whether such algorithms or guidelines have adequate predictive value (i.e., clinical validity) and their use has been shown to improve patient or clinical outcomes (i.e., clinical utility). It was recognized that exploration of this would broaden the scope of the review to such an extent that it would become unmanageable. Therefore, it was determined that the validity of underlying algorithms or guidelines should be taken at face value. Thus, the focus of the review should be confined to evaluating whether tools were effective in facilitating the translation of a patient's family history information into a specific risk stratum, compared with current primary care practice, on the assumption that such stratification was worthwhile.

Search Strategy

The systematic review protocol search included the electronic databases MEDLINE^®, EMBASE^®, CINAHL^® and Cochrane Controlled Trials Register (CCTR)^® from 1990 to July 2007. In addition we retrieved and evaluated references from eligible articles. Hand searching was not undertaken for this review. However, we did review the publication types “letters” (normally excluded from reviews); the investigators suggested that, within the content area of cancer genetics, primary data information might be published as letters in some journals. We also undertook a search of relevant grey literature sources. Detailed search strategies and websites explored are listed in Appendix A.^*

Eligibility Criteria

A list of eligibility criteria was determined and standardized forms were developed in Systematic Review Software (SRS) for the purposes of this systematic review. The forms and help guides detailing the eligibility criteria can be found in Appendix B.^*

Study Selection

A team of study assistants was trained to apply the eligibility criteria in preparation for screening the title and abstract lists and the full text papers. All levels of screening were done in web-based Systematic Review Software (SRS) (TrialStat Corporation, Ottawa, Ontario Canada). Standardized forms and a training manual explaining the criteria were developed and reviewed with the screeners (Appendix B ^*). For the title and abstract phase, two reviewers evaluated each citation for eligibility. Articles were retrieved if either one of the reviewers judged it as meeting eligibility criteria or if there was insufficient information to determine eligibility. For screening of full text articles, two screeners came to consensus on the identification, selection, and abstraction of information. Disagreements that could not be resolved by consensus were resolved by one of our McMaster research team members. The level of agreement for inclusion of studies was measured using kappa statistics.

Data Extraction

Appropriate data collection forms were developed for use in the systematic review (Appendix B ^*). All eligible studies from the selection phase (full text screening) were abstracted onto a data form according to predetermined criteria. One data extractor transferred the data onto these forms, and another checked the answers for accuracy before they were entered into SRS. Data entries were verified by the investigators responsible for summarizing the different report results sections.

Quality Assessment of Included Studies. To assess the quality of primary studies, we utilized standardized rating scales with acceptable reliability and validity. The specific scale used was dependent on the study design and the research question. The Quality Assessment of Diagnostic Accuracy Studies (QUADAS)¹⁰³ was selected to evaluate studies primarily focused on accuracy (i.e., included in Q1). The Jadad scale was used for studies that were randomized controlled trials (RCTs).¹⁰⁴ For true observational study designs, the Down's and Black quality assessment scale was used.¹⁰⁵ Studies that were neither of these study designs were evaluated qualitatively without the use of formal checklists. The instruments used to evaluate quality are shown in Appendix B.^*

Summarizing Our Findings: Descriptive and Analytic Approaches

A qualitative descriptive approach was used to summarize study characteristics and outcomes. Multiple publications on the same study cohort were grouped together and treated as a single study with the most current data reported for presentation of summary results. Standardized summary tables explaining important study population and population characteristics, as well as study results, were created. Meta-analysis was not undertaken for eligible studies within this review as the clinical heterogeneity between studies was considerable.

For those papers evaluated for research Q1, where the actual numbers of true and false positive and negative results (TP, FP, TN, FN) were presented, or where enough information was given to allow us to calculate and estimate these numbers, we recalculated the sensitivities and specificities with the accompanying 95 percent confidence intervals (CI) where possible.

For those papers evaluated for research Q2, descriptive data on the attributes of FHxTs were presented. For those FHxTs that had been formally evaluated, we reported outcome data separately for those tools compared with best estimate, and those compared with current practice comparators.

For those papers evaluated for research Q3, we presented descriptive data on the attributes of RATs, including the evidence base, if any, underlying each tool. For those RATs that had been formally evaluated, we reported data on outcomes relevant to the use of the tool in supporting decisions by users in practice (e.g., the pattern of referrals from primary to specialist care, patient perceptions of their cancer risk, health professional confidence in counseling patients concerned about their risk, etc.). Data regarding the validity of the knowledge component of each RAT (e.g., the scientific basis for guidelines, the predictive value of a stratification system, etc.) were captured where possible, but it is not within the scope of this review to consider the quality of such evidence (see “Topic Refinement”, above).

General Approach

We undertook a broad approach to identifying studies evaluating accuracy of reporting family history. We did not limit studies to those presenting specific diagnostic accuracy metrics and included studies whose primary aim was to ascertain repeatability (variation observed when conditions are kept constant by using the same instrument and individual and repeating within a short time interval).

Studies Reviewed

A total of 20 publications evaluated the accuracy of reporting family history and were eligible for data extraction. One study was based on two publications^{10, 11} leaving a total of 19 unique studies. Study and patient characteristics (such as study design, setting recruited, cancer type, relatives evaluated and criterion standard evaluated) are detailed in Appendix C ^* evidence tables.

We further classified studies by the type of accuracy that was evaluated as follows: 1) those studies (16 studies in 17 publications) which evaluated accuracy of family history reporting by attempting to verify the cancer status of relatives (i.e., accuracy compared with a gold standard), and 2) those (three) which evaluated the repeatability or reliability of the informant's knowledge of family history rather than the true status of the relatives (i.e., no external gold standard).

For the purposes of this review we use the terms “affected” and “unaffected” to refer to those relatives who have had cancer, and those who have not, respectively. We present the results for accuracy according to these groupings, and with regard to specific participant characteristics, type of accuracy evaluated (gold standard or reliability), method of verification, and potential predictors or confounders of accuracy of reporting family history (Figure 3).

In general we can summarize the accuracy studies as predominantly having recruited participants who had cancer. Within the 19 studies (20 publications), there were three that recruited an entire sample of patients who were free of cancer; two studies involving individuals at high risk for colorectal⁷ or breast cancer,⁸ and one involving women undergoing mammography.⁹ In the four case control studies (five publications),¹⁰–¹⁴ the controls were derived from the general population matched for age,^{10, 11} spouses of the informants or regional general practice lists,¹⁴ and from a linkage from license registration and health care administration database.¹³

All studies were classified as case series except four which were case control studies. Several important factors restrict comparisons across accuracy studies, such as the cancer diagnosis of the informants and the cancer information collected about the relatives. There were more studies evaluating informants with breast cancer than other types of cancers; there was a single study evaluating ovarian cancer syndromes within the informants. Some studies probed only specific cancers within relatives while others reported on all cancers within their family histories. While there were only three studies with fewer than 100 informants, the number of relatives reported varied greatly between studies.

Studies Evaluating the Accuracy of Reporting by Verifying no Presence or Absence of Cancer in Relatives. Sixteen studies^{7, 8, 10}–^{17, 19}–²⁴ evaluated the accuracy of family history reports by attempting to confirm the true cancer status of the relatives about whom informants provided information. Eight studies ^{13, 14, 19}–²⁴ verified the cancer status in relatives reported to be affected and those reported to be unaffected. The other eight studies (nine publications)^{7, 8, 10}–^{12, 15}–¹⁸ only confirmed the cancer status of relatives reported to be affected. We considered the former studies to be of higher methodological rigor and therefore evaluated these two groups of studies separately.

Studies With Verification in Both Affected and Unaffected Relatives. Table 2 shows the eight studies that verified the cancer status both of relatives reported to be affected and unaffected. Three were case control studies^{13, 14, 19} that recruited participants with colon or colorectal cancer. The remaining five studies evaluated breast cancer patients and a single study evaluated patients with breast, ovarian or colorectal.²⁴ A single study²² evaluated the accuracy of relatives' perception of “awareness of cancer” rather than informants' accuracy in reporting family members with cancer. Three studies^{13, 14, 23} recorded the informant's recollection of any type of cancer in relatives, and the remaining studies examined reporting of relatives' colorectal cancer,^{19, 22} breast cancer,²⁰ breast or ovarian cancer,²¹ or one syndromic group of cancers²⁴ (breast, ovarian or colorectal). In general, family history informant characteristics such as mean age, ethnicity, or education were poorly reported (Table 2). Similarly, characteristics of the relatives were also poorly reported within these studies.

The methods of family history collection varied with face-to-face interviews in two studies,^{13, 14} mailed survey in four studies,^{19, 21}–²³ and two with telephone interviews.^{20, 24} The methods of verification of relatives' cancer status varied between studies; also, within some studies different methods were used for checking the status of relatives reported to be affected and those reported to be unaffected. The methods used were: (1) personal interview (reportedly affected) and cancer registry; (reportedly unaffected²³) (2) face-to-face interview, survey, and death registry;²⁴ (3) self report from mail-in survey of relatives;²² (4) relatives' medical chart records and survey; (type not specified)¹⁹ (5) cancer registry alone;^{13, 14, 20} and (6) combined strategy (medical record or cancer registry or death certificate).²¹

Table 3 shows the sensitivities and specificities in studies that evaluated the status of both reportedly affected and reportedly unaffected relatives, where sufficient data were presented to compute these. One study²² was excluded from Table 3 as it evaluated accuracy only in terms of “awareness” of parent or sibling's colorectal cancer. The sensitivity varied by the cancer of interest; for ascertainment of relatives with breast cancer, the range was 85 to 95 percent based on three studies; for colon cancer, 57 to 65 percent (studies using personal interview) and 86 to 90 percent (studies using telephone interview and self report) based on four studies; for ovarian cancer, 67 to 83 percent based on two studies; and for prostate cancer, 69 to 79 percent based on two studies. It is not clear to what extent the verification method of cancer registry versus medical records/death certificates contributed to the ranges observed within a cancer type and between the different cancer types. Similarly, it is difficult to establish how the various methods of collecting family history may have influenced the estimates of sensitivity.

In general, specificity across all cancer types and with varying modes of collection was consistently high, (Table 3). For ascertainment of relatives with breast cancer, the specificities were 95 to 98 percent; for colon cancer, 91 to 92 percent; for ovarian cancer, 96 to 99 percent; and for prostate cancer, 93 to 99 percent.

There were three case control studies that therefore allowed for comparison of reporting accuracy between cases and controls. They all involved cases who were patients with colorectal cancer, and controls who did not have cancer. The first study¹⁹ suggested that cases were slightly more accurate than controls (82 percent vs. 76 percent) in reporting history of colorectal cancer in relatives. The second¹⁴ indicated a sensitivity of 57 percent (95 percent CI 43–69) in cases compared with 53 percent (95 percent CI 31–74) in controls in reporting relatives with colorectal cancer. Within this study, the corresponding specificities were 99 percent (95 percent CI 98–99) in both cases and controls. The third study¹³ compared cases and controls with respect to accuracy of reporting several cancer types in their relatives: (1) sensitivity of reporting relatives' breast cancer - cases 85 percent (95 percent CI 55–98), controls 82 percent (CI NR); (2) sensitivity of reporting relatives' colorectal cancer - cases 65 percent (95 percent CI, 38–86), controls 81 percent (CI NR); (3) sensitivity of reporting relatives' ovarian cancer - cases 67 percent (95 percent CI, 9–99), controls 50 percent (CI NR); and (4) sensitivity for reporting relatives' prostate cancer - cases 69 percent (95 percent CI, 41–89), controls 70 percent (CI NR). The corresponding specificities were: 1) relatives' breast cancer status - cases 98 percent, controls 91 percent; 2) relatives' colorectal cancer status - cases 91 percent, controls 94 percent; 3) relatives' ovarian cancer status - cases 96 percent, controls 98 percent; and 4) relatives' prostate cancer status - cases 93 percent, controls 94 percent. Taken together, these data suggest broadly similar specificities across the reporting of cancer types and between cases and controls - i.e., generally, the participants with and without cancer themselves were fairly good at correctly identifying relatives without a history of cancer, irrespective of the specific cancer family history being enquired about. In contrast, the sensitivities were generally lower, meaning that informants appeared to miss some cancers in affected relatives; the highest sensitivities were seen for reporting relatives' history of breast cancer. The results also suggested some differences in sensitivities of reporting between cases and controls - controls being more likely than cases to miss colorectal and ovarian cancers in relatives. In addition, the data from this study would suggest differences in sensitivities such that controls are more accurate for colorectal cancer but less accurate for ovarian cancers. In contrast, the specificities were similar for the cancers evaluated, suggesting no difference between cases and controls with respect to their accuracy in identifying who of their relatives does not have specific cancers. These observations are based on a single study and therefore should be interpreted cautiously.

Studies With Verification in the Affected Relatives Only. Table 4 shows the eight studies (nine publications)^{7, 8, 10}–^{12, 15}–¹⁸ that verified the cancer status only of relatives reported to be affected by cancer. A single study (two publications) was a case control design^{10, 11} and the remaining were case series. Two studies involved participants who did not have cancer but who were at high risk for breast⁸ or colorectal cancer.⁷ Two studies^{15, 17} involved patients who had prostate cancer, and one study involved colorectal cancer patients;¹⁶ one study combined Li-Fraumeni Syndrome (LFS) and Hereditary Breast-Ovarian Syndrome (HBOCS)¹² (both women at genetic high risk and some with cancer) and one study (two papers)^{10, 11} involved women with breast cancer. A single study involved a range of participants with and without cancer.¹⁸

Five studies^{7, 12, 16}–¹⁸ assessed the informant's ability to report any cancer within relatives, and the remaining studies appeared to assess reporting of relative's breast cancer ^{8, 10, 11} or prostate cancer¹⁵ history. In general, informant characteristics such as mean age, ethnicity, or education were poorly reported. Similarly, characteristics of the relatives were also poorly reported (Table 4).

The methods of family history collection varied with face-to-face interviews used in three studies (four papers),^{10, 11, 15, 16} telephone interviews in one study,⁷ interview with mode not reported in one study,¹⁷ survey completed in the clinic in one study,⁸ and mailed survey in two studies.^{12, 18} The methods of verification of the relatives actual cancer status included: (1) personal or telephone interview with relatives and medical records,⁸ (2) relatives' medical chart records alone,^{10, 11, 15, 17, 18} and (3) a combined strategy (medical record or cancer registry or death certificate).^{7, 12, 16}

From five studies^{7, 12, 16}–¹⁸ that reported on the informant's ability to report any cancer within relatives, only two studies provided information on the percent agreement as a function of the cancer reported. One study¹⁸ indicated that breast and colorectal cancers had 93 percent and 89 percent agreement and lower rates of agreement for other cancers (42 percent for extra-colorectal alimentary tract and 37 percent uterine cancer). Another study¹⁷ showed similar results with higher percent agreements for breast, colon, and prostate cancer (95, 92, and 86 percent respectively) in patients with prostate cancer. One study ¹² who evaluated subjects with LFS and HBOCS found differences in the accuracy of reporting, with 85 percent agreement and 92 percent agreement with the reported cancers within their relatives.

Two studies reported on the accuracy of breast cancer within relatives and the percent agreement varied from 89 percent in one study⁸ (with greater accuracy in living relatives with unilateral disease 94 percent) to a sensitivity of 90 percent (CI 95 percent 81–96) in a second study.^{10, 11} The specificity for this latter study^{10, 11} was estimated at 3 percent suggesting errors in reporting of unaffected relatives. One study¹⁵ reported 90 percent agreement for relatives with prostate cancer. Another study¹⁶ reported on the accuracy of colorectal cancer in relatives, with a sensitivity of 61 percent (CI 95 percent 36 – 83) and a specificity of 96 percent (CI 95 percent 88–99). Although, the magnitude of the agreements are generally high for reporting on some cancers, caution should be used when interpreting the results from studies that evaluate accuracy by confirming the status of the affected relatives only, as these contain errors and bias.

Other Factors That May Affect Reporting Accuracy. A variety of factors which could potentially influence accuracy of family history reporting were considered in some studies. Table 5 shows the factors that have been evaluated within some of these studies and, indirectly, the degree of evidence for each of these. We examined 15 characteristics, although some were only evaluated in a small number of studies. Those characteristics infrequently evaluated were: (1) type of first degree relative (1DR), (2) vital status of the relative, (3) number of relatives, (4) cancer history of interest, (5) cancer type of the informant, (6) race of the informant, (7) marital status, (8) laterality within breast cancer, (9) population versus clinic setting recruitment, (10) health insurance status, and (11) gender or age of diagnosis of the relative. It is difficult to generalize for these factors from this heterogeneous series of studies evaluating informants with different cancers and reporting on different cancers within their relatives. Moreover, some of the studies did not actually statistically evaluate differences between the factors of interest; thus, these findings should be regarded as indicating attributes that could be further evaluated in the future research.

Eight studies (nine publications)^{8, 10, 11, 13}–^{15, 18, 19, 24} evaluated the effect of age of the informant on accuracy; no clear trend was observed, and it was not possible to separate any effect of informant age from the possible effects of their own cancer type, gender, or differences in how age was categorized.

Six studies^{7, 13, 14, 18, 19, 24} evaluated the effect of the informant's gender on accuracy, and suggested no general effect. One study¹³ suggested that women might be more accurate in correctly identifying relatives who had ovarian cancer. Another⁷ suggested that there were gender differences in knowledge of paternal versus maternal family history. A third²⁴ suggested that men may over-report cancers compared to women.

Six studies^{12, 14, 15, 18, 23, 24} evaluated whether accuracy varied with the degree of relative whose status was being reported; there was a consistent trend towards increased accuracy of reporting for 1DRs compared to second degree relatives (2DR) or third degree relatives (3DRs) (Table 5). Several studies^{14, 23} noted challenges in confirming the true status of 2DRs and also that fewer 2DR and 3DRs were identified overall, suggesting the potential for reporting and confirmation biases.

Five studies (six publications)¹⁰–^{13, 15, 19} evaluated the effect of education level using a variety of categorizations; all but one study¹² showed an effect on accuracy of reporting.

Quality Assessment of Studies

We evaluated quality of the accuracy studies at several different levels. At one level, we considered that the method by which the cancer status of the relatives was evaluated was of great importance in determining accuracy of reporting. At another level, we applied traditional internal validity criteria for study designs that included a comparison group or were considered diagnostic in their design. Since so few of the studies were of traditional study design with control groups, the majority of standardized assessment scales could therefore only be applied to a subset of papers. If we considered all the studies as “diagnostic” in their design, the QUADAS (a quality assessment scale for diagnostic studies) could be applied to most studies. However, not all 14 criteria (or biases) applied to the “diagnostic test” of “family history collection” were relevant in the context of accuracy of reporting; we selected three criteria from the QUADAS to compare the different studies.

Methodological Issues in the Verification of the Cancer Status of the Relatives. For accuracy of family history reporting, we considered verification of the status of both the affected and unaffected relatives to be of the highest quality. Studies that verified the status of the affected relatives only were considered to be of lesser quality or more susceptible to bias with respect to accuracy of reporting.

A number of difficulties were identified by authors with regards to ascertaining the cancer status of the relatives. The range of estimates of difficulties in obtaining some type of confirmation varied from 31 percent¹⁹ to 9 percent.²¹ Some of the difficulties with verification of cancer status of the relative included: (1) errors in medical records or pathology reports,^{8, 21} (2) death of relative prior to registry formation or other form of record keeping,²¹ (3) relative emigrated to another geographic region, for which medical records were not available to the researchers,^{8, 21} (4) informants provided incorrect address or contact information for hospitals where relatives were treated,⁸ (5) retrieval of death certificate information was impossible due to peculiar national laws affecting access by researchers or it was certain the files had been destroyed,¹⁸ (6) some difficulty obtaining medical records of fathers compared to brothers, mothers, and sisters,¹⁷ (7) reports concerned relatives for a branch of the family not of interest to the genetic investigation,¹⁸ (8) the reported cases were late onset common type tumors in distant relatives not likely of interest in the referral,¹⁸ and (9) informants were not in touch with the relatives concerned, so consent could not be obtained.¹⁸ Some studies found it difficult to obtain medical records of deceased relatives when recruitment of relatives for consent depended upon the informants contact.⁹ There was some suggestion that verification rates were lower among negative relatives¹⁹ as these tended to have less physician visits. Studies undertaken in countries with longstanding national cancer and death registries linked with service provision databases, tended to report very high rates of retrieval (97–98 percent) of verification of diagnoses on relatives.¹⁶

Although there were a variety of possible factors that impeded verification of the cancer status of the relative, not all studies excluded from the analysis those informants or relatives for which there were some difficulties in complete confirmation. Note that many studies did not compare the characteristics of the informants who did not wish to contact relatives for their medical records relative to those that did; similarly, comparisons between those relatives that provided consent to medical records and those that did not were not consistently undertaken.

QUADAS Assessment of Methodological Quality for Diagnostic Studies. We applied the QUADAS to those studies that verified the status within their relatives. The QUADAS, a 14 item quality assessment scale for diagnostic studies, was used to evaluate all studies eligible for accuracy of reporting. From these items, three were considered to be of greatest relevance to identifying potential biases within these studies that considered the collection of family history as the “diagnostic test” of interest and the method of verification as the “reference test”. The first challenge was to assume that the “diagnostic test” was the same method of family history collection, in order to compare ratings across studies; clearly, the tools or methods used to collect family history varied significantly amongst studies. The second assumption, we made was that the reference standards specified within each study were equivalent across studies; that is that cancer registry verification and death certificate verification were equivalent.

Three items from the QUADAS were selected to evaluate spectrum bias, verification bias (both differential and partial), and blinding of those who verified the cancer status of the relatives. If present within the studies, each of these biases will result in overestimation of accuracy.

Spectrum Bias. The first question within the QUADAS asks: Was the spectrum of patients' representative of the patients who will receive the test in practice? Theoretically, being asked to take the “test” of cancer family history collection may be received by any person (with or without cancer) in clinical practice. Thus, it was challenging to define which informants are not “typical” of those likely to be tested in practice.

We would indicate the presence of spectrum bias, when the study population did not reflect the spectrum of informants likely to be seen within the clinical setting. For example, patients recruited due to their high risk for familial cancer syndromes would not reflect the spectrum of patients who would report cancer “family history”, albeit they are an important group to evaluate. Similarly, in those studies with informants with cancer of differing severity or who were differentially assigned to study groups, the likelihood of spectrum bias is evaluated as high. We considered a sufficient spectrum of disease should include participants who reflect a complete range of staging (severity) of their cancer if the informant had cancer when the family history was collected. Additionally we believe that an adequate spectrum should reflect informants that included both genders in those studies that did not affect sex-specific organs, such as ovaries or prostate.

When considering the eight studies that verified the status of both the affected and unaffected relatives, the potential for spectrum bias was evident. In general, these studies did not report information on the informants with respect to the severity of disease. One case control study¹³ specified that the cases were “first primary cases” while the others of the same study design did not specify; however, there is still potential for spectrum bias in these studies. One of the studies evaluating breast cancer informants included women of restricted age (< 40 yrs), one third of subjects with bilateral breast cancer, referred to university hospital oncology centre.²¹ Another²³ included informants that were English speaking, North American born, without brain metastases and had a least one 1DR with breast cancer. Both these studies, although they reflect patients likely to be seen in cancer clinics, do not represent the spectrum of breast cancer patients and therefore these studies have spectrum bias.

When considering those studies that evaluated the status of the affected relatives alone, the potential for spectrum bias was also evident. Two studies^{7, 8} recruited cancer free informants who were at very high risk for familial cancers due to a history of 1DRs already diagnosed with the cancer of interest. For the remaining studies, the severity of cancer within the informants was not detailed. This suggests the potential for spectrum bias.

Verification Bias. The fifth question within the QUADAS asks: Did the whole sample or a random selection of the sample, receive verification using a reference standard? Partial verification bias occurs when not all members of the study group receive confirmation of the diagnosis by the reference standard. Similarly, differential verification bias can occur if a subgroup of patients is given a different reference standard test. Partial verification bias can occur if some of the relatives identified by the informant did not have their cancer status verified. Even in studies where both affected and unaffected relatives were evaluated, we did observe that some studies were not able to verify the status of some of the relatives for many of the reasons stated above. One study,¹⁹ (which employed very rigorous ascertainment methods of reportedly affected relatives, even sending notes to hospitals overseas for determining the status of deceased relatives), indicated that they did not attempt to check the medical record of all relatives who were cancer free (the overwhelming majority). Other studies^{7, 13, 19, 20, 22} limited their evaluation or reporting to 1DR only; this in itself may reflect a type of differential verification bias in that not all relatives reported by the informants were verified. In those studies that evaluated only the affected relatives, clearly partial verification bias was present. The presence of partial or differential biases may lead to overestimation of accuracy.¹⁰⁶

Blinding of Those Verifying Cancer Status in Relatives to the Status of the Informant. The eleventh question of the QUADAS states: Were the reference standard results interpreted without knowledge of the results of the index test? In the context of family history collection, our interest was in having those who verified the status of the relatives blinded to the cancer status of the relative and possibly the informant. It is possible that the research assistant extracting the cancer status of the relative, having knowledge of their cancer status, might interpret information (for example, from medical charts) differently than if they were not aware of the cancer status of the relative. Problems with lack of blinding may be less likely to occur in studies that use linkages with cancer or hospital registries; presumably the criteria for verification are not dependent on interpretation by a research assistant. However, there are errors associated with linking databases.

Of the eight studies that evaluated the status of both affected and unaffected relatives, three^{13, 14, 20} relied solely on linkages with cancer or population health registries, and one⁷ on patient report or health records alone; the remaining four studies used a combination of interview, health records and death registries. For those studies that evaluated the affected relatives alone, a single study¹⁸ used computerized linkage alone with patient records to ascertain the status of the relative. Overall, blinding of the status of the relative or the informant was not undertaken in the majority of studies.

Methodological Quality Assessment for Case Control Studies. We applied traditional internal validity criteria to the four case control studies (five publications),^{10, 11, 13, 14, 19} using the Down's and Black standardized quality assessment scale.¹⁰⁵ One study¹⁹ originated as a case control study but undertook a sample from the original to perform a validation study on accuracy of reporting; informants were selected on the basis of having relatives with cancer rather than their cancer status. We did not evaluate the quality of this study using the Down's and Black scale. The range of composite quality scores varied between 14 and 17 (from a possible score of 23), indicating a moderate level of quality for the three case control studies. One of the main methodological flaws was the omission of descriptions of the distribution of principal confounders in two of the studies (three publications).^{10, 11, 13} In addition, only one study¹³ enrolled subjects who appeared to be representative of the general population from which they were recruited and only one study (two publications)^{10, 11} indicated that cases and controls were recruited over the same time period. It was impossible to tell, based on the information contained in the studies, whether cases and controls were recruited from the same source population. There was insufficient information in all four studies to assess blinding, but all studies had reports of losses to follow up. The authors of one study¹² adjusted for potential confounders in the analysis.

The potential for selection or information bias in these four case control studies is difficult to assess. The lack of reporting on recruitment and blinding does not necessarily mean that the authors ignored these issues. It is possible that all subjects were recruited from the same source population and all subjects and investigators were blinded. The authors may simply not have reported this information in the published manuscripts.

Studies Reviewed

A total of 39 different tools, implemented in 40 unique studies, and reported in 45 publications passed full text criteria. Our initial focus was on identifying studies that described FHxTs developed or used in a primary care setting; however, after careful review, we noted that many studies described tools used in other settings that appeared potentially relevant to primary care (criteria for “primary care applicability” is outlined in Chapter 2). We also sent email queries to all authors of eligible studies that did not provide sufficient detail of the FHxT or a copy of the tool. Fifteen authors (of 16 publications) ^{8, 10, 11, 16, 17, 21, 23, 25}–³³ did not respond in time for the publication of this review and therefore we were unable to determine whether the reported FHxT was applicable for use within primary care. For those studies for which we evaluated the FHxT, six tools from seven publications^{13, 18}–^{20, 24, 34, 35} were assessed as inappropriate for primary care; all of these had been developed and used in research settings. The scoring system and scoring of actual FHxTs is displayed in Appendix B.^* Of the remaining 22 publications, four ³⁶–³⁹ described the prototype and final versions of the same FHxT (RAGS/GRAIDS), which we counted as a single tool; and two^{40, 41} were companion publications. Thus, 18 distinct tools, from 22 publications, were identified as being applicable to primary care settings (Figure 4). Full study details are summarized in the evidence table (Appendix C,^* Table 2).

Description of Tools

Target User. Fourteen tools⁴²–⁵⁵ were designed for completion by patients, and four tools (eight publications)³⁶–^{41, 56, 57} were designed for use by health professionals.

Format. Eleven tools^{43, 45}–^{49, 51}–⁵⁵ were paper-based, generally in some form of questionnaire or structured questions. Four tools (eight publications)³⁶–^{41, 44, 50} were presented in a form for use on a desktop or laptop computer, including web-based and touch screen applications, and one on a personal digital assistant.⁵⁷ One tool⁴² was an automatic telephone interview, and one was a structured interview schedule.⁵⁶

Cancer Type. Fifteen tools, reported in nineteen articles,³⁶–^{43, 45}–^{50, 52, 53, 55}–⁵⁷ were designed to collect data on family history of breast or breast/ovarian cancer. Nine tools (ten publications) ⁴⁰–^{42, 46}–^{50, 52, 57} captured data on colorectal cancer and two^{40, 41} tools (three publications)⁴⁰–⁴² on prostate cancer. Five tools (six papers)^{36, 37, 42, 47, 48, 57} also captured data on one or more additional cancer types. For two,^{51, 54} the tool appeared to invite information on any cancer type.

Clinical Setting. Four tools (seven publications)³⁶–^{39, 48, 49, 56} described tools which were implemented in family practice settings, and four tools^{46, 52, 54, 57} in internal medicine clinics. One tool⁴⁷ was implemented in a gastrointestinal clinic, and another⁴⁵ in a screening mammography setting. Three tools^{46, 54, 55} were designed for use in cancer centers or clinics and three⁴²–⁴⁴ were implemented in genetic clinics. One tool (two publications)^{40, 41} was web-based and designed for use by any health professional, and the remaining tool⁵³ was used in a large population-based research study. The published reports indicated that eight of the tools were used in a proactive way,^{46, 48, 49, 51, 52, 54, 55, 57} eight (12 papers) in a reactive manner,^{36, 38}–^{41, 43}–^{45, 47, 53, 56} and two in a mixed approach.^{42, 50}

Links to Risk Assessment Tools. The output of five tools (nine publications)³⁶–^{41, 44, 45, 57} was linked directly to some form of defined risk assessment tool (RAT) (i.e., the family history data were converted directly into a risk categorization), although several of the publications describing other tools also described companion RATs.

Content of FHxTs. Fourteen tools³⁶–^{39, 42}–^{45, 47}–^{52, 54}–⁵⁶ reported in seventeen publications, were designed to capture data on all, or selected, 1DRs. Eleven tools (fourteen papers)³⁶–^{39, 42, 44, 45, 47, 49, 50, 52, 54}–⁵⁶ were designed to capture data on all or some 2DRs, and one⁴⁹ on grandparents only. Five tools^{42, 44, 45, 47, 50} explicitly went beyond 2DRs, although not necessarily to capture all 3DRs. For the remaining tools, the extent of family history enquiry was not explicitly described. For all tools except five^{48, 51, 53, 55, 57} there were explicit instructions for users to capture data on relatives on both sides of the family. Two tools^{49, 54} were designed to explicitly capture ethnicity data. Further details of the data captured are presented in Summary Table 7.

Other Family History Tools. Eleven web-based FHxTs were also identified during the grey literature search. Nine tools were actually available from the web, and these are listed with relevance scores in Appendix B.^* For all except one, (JamesLink)⁵⁰ which was included in the main review, no information was provided on their development or evaluation, which precluded their inclusion in the main review. The highest scoring of these tools for applicability to primary care were the Family History Tool developed by American Academy of Family Practice¹⁰⁷ and the U.S. Surgeon General's Family History Initiative.¹⁰⁸

Evaluating the Family History Tools

The tools were evaluated using a range of study designs. In order to avoid ambiguity in terminology, we drew a distinction between the concepts of “comparator” and “control” (or “controlled”). In keeping with the methods described in Chapter 2, we use the term “comparator” to refer to the use of a reference method to assess the extent, nature and/or accuracy of the family history data captured by the tool in question, the comparators being either “ideal”, best estimate interview, or current (“standard”) practice. We use the term “controlled” to indicate a study design where there are at least two arms, one of which is the tool in question and the other an alternative method of capturing family history data. Thus, in a controlled design, participants are assigned (randomly or otherwise) to either the “tool” group or the control group. We considered crossover studies, where the order of data capture (tool or comparator method) was reversed for some participants, to be controlled studies. Table 8 describes the distribution of studies, in which tools were used, between the four possible categories of study design. We noted that one tool 44 was evaluated in a controlled study, but that no comparator for family history data capture was used, and no outcomes were reported which were relevant to the tool performance as a method of family history data collection (although outcomes relevant to performance as a RAT are presented under Question 3).

Using this approach, for the purposes of this review, we considered those studies which were uncontrolled studies with no comparator as descriptive, and those which either had a comparator or were controlled to be evaluative, so long as outcomes were reported which were directly relevant to the use of the tool as a method of capturing family history data.

Validity and Reliability

Six tools (nine publications) were described as having undergone a development or piloting phase³⁶–^{39, 42, 45, 48, 49, 51} including one tool (two publications) (Risk Assessment in Genetics, RAGS)^{38, 39} which was the prototype for the Genetic Risk Assessment and Decision Support (GRAIDS) tool,^{36, 37} and a self-completion tool which was developed from a previously validated interview schedule.⁵¹ Five studies assessed acceptability and ease of completion of the tool.^{36, 37, 42}–⁴⁴ Qualitative techniques were also described in studies of four tools, including semi-structured interviews with practitioners^{38, 39} and patients,⁴⁹ and focus groups with practitioners.^{40, 41, 49} Three studies,^{42, 44, 45} reported how long it took to complete the tool, ranging from 8 to 30 minutes. One study⁴² reported test-retest reliability of 97 percent for 1DR, and 93 percent for 2DR respectively, and 98 percent for cancers identified.

Six tools were presented in seven descriptive papers,^{40, 41, 48, 53}–⁵⁶ without a comparator group or control arm. One study of a family history tool embedded in a RAT⁴⁴ presented no outcome data pertaining specifically to performance in capturing family history data.

The performance of the 11 remaining tools was assessed in some way against a defined comparator. For five tools,^{42, 43, 45, 49, 51} this was a genetics interview. For one tool,⁵¹ the self-completion questionnaire was assessed against the parent interview schedule administered by non-genetics investigators. Six tools (eight publications)³⁶–^{39, 47, 50, 52, 57} were compared with current practice in some form. This included the family history as recorded in patient charts, and accuracy or completeness of pedigrees derived from simulated patient histories drawn without access to a tool.

Outcomes

Evaluated Against Genetics Interview. Acheson and colleagues⁴² described an automated telephone interview tool which was evaluated in a sample of genetics patients. Pedigrees obtained by the tool were blindly compared with those obtained from their clinic interview with a genetic counselor. There was an overlap between the data captured by the tool and the interview. The tool was statistically significantly better than genetics interview at identifying 2DRs and first cousins, and identified more cancers in 2DR and distant relatives. When the risk stratification based on the tool and interview pedigrees was compared, there was good agreement (kappa=0.70) for the breast cancer risk assessment, and moderate agreement for colorectal cancers and all cancers combined. Three families classified as high risk by the tool would be classified low risk on the basis of the interview, and one family classified as low risk by the tool would be classified high risk by the interview pedigree. The tool showed high test-retest reliability.

Qureshi and colleagues⁴⁹ described a paper-based, self-completion family history questionnaire, which was compared with a genetics interview conducted by trained researchers. On the basis of the family history captured, 24 percent of tool histories, and 36 percent of interview pedigrees, suggested possibly elevated disease risk which would warrant further investigation. The interview identified 15 percent more 1DRs, and 51 percent more 2DRs, than the tool. The validity of the risk assessments was not determined by a full genetics assessment, so it is not possible to conclude whether the tool was less sensitive or more specific than the interview comparator.

Benjamin and colleagues⁴³ assessed a standard paper-based, mailed, self-completion family history questionnaire with a clinical genetics interview, as part of a study whose primary aim was to evaluate a companion RAT. Using the interview as the gold standard, the tool had 95 percent sensitivity and 96 percent specificity for family breast cancer risk assessment. On the basis of the tool data alone (before the interview), 51 percent of patients would be assessed as having an elevated risk of familial breast cancer; following the genetics interview, this figure was 62 percent.

Fisher and colleagues⁴⁵ assessed a paper-based, patient-completed family history questionnaire in a study whose primary aim was to assess its embedded risk categorization scheme. The participants were women attending for routine breast screening, and the history obtained by the tool was confirmed by follow up telephone interview by a genetic counselor. The authors report that this was to check that the tool data reflected the women's current knowledge of their family history, not to verify it. Of 45 women classified at population risk by the tool, none were reassigned a higher risk on the basis of the genetics interview. Of 45 women classified at elevated risk, none were reclassified as population risk. Further validation of the risk status of the participants through full genetic assessment was not reported.

Kelly and colleagues⁵¹ describe a paper-based, patient-completed tool which was assessed in a sample of cancer patients. In a study whose primary aim was to explore psychosocial outcomes related to accuracy of family history reporting, they compared the questionnaire with an interview-based version of the same tool, using a randomized crossover trial design. The authors report around 77 percent concordance for reporting relatives' age, 81 percent concordance for reporting of relatives' diagnoses, and 82 percent concordance for reporting of age of diagnosis. There were no discrepant data on whether or not a relative had cancer. The order of completion of tools was not associated with differences in these outcomes.

Evaluated Against Current Practice. Emery and colleagues describe the development of a family history tool and RAT (GRAIDS), the prototype for which was RAGS.³⁶–³⁹ GRAIDS was evaluated using a pragmatic cluster randomized controlled trial,^{36, 37} but no outcomes relating to performance as a FHxT were specifically reported. However, data were reported from a evaluation of the RAGS prototype,³⁹ in which 36 family physicians used three different methods to draw pedigrees and assess the risk of simulated patients. Pedigrees produced using the RAGS tool were statistically significant and more likely to be accurate than those prepared by a genetics software package (Cyrillic) or by traditional pen and paper methods (median correct pedigrees, 5.0/6 for RAGS, 3.5/6 for Cyrillic, 2.0/6 for pen and paper). Participating physicians also preferred RAGS (75 percent) over the other methods (8 percent preferring Cyrillic and 17 percent preferring pen and paper).

Frezzo and colleagues⁴⁶ compared a paper-based, patient-completed family history questionnaire with a genetics interview in a quasi-randomized parallel group study. Of the 39 internal medicine patients who completed the tool, two were identified at elevated risk of breast/ovarian cancer, three at risk of colorectal cancer, and one at risk of prostate cancer. Review of these patients' charts revealed only one patient at elevated risk, of colorectal cancer. In the group whose risk was assessed by interview, the corresponding figures are five at risk for breast/ovarian, and four at risk of colorectal cancer, on the basis of the interview, compared with two and two, respectively, on the basis of chart audit. No data were presented regarding the outcome of eventual genetic risk assessment, if any, of the participants.

Schroy and colleagues⁵⁷ developed an educational intervention for internal medicine residents and assessed the effect of a software tool designed for use on a personal digital assistant. Patients' family history relevant to colorectal cancer risk was assessed by a structured interview with a research assistant. Patients' charts were then audited to assess whether positive and negative colorectal cancer family histories were correctly documented. Of 33 residents to whom the software was sent, 29 acknowledged receipt, two acknowledged downloading it, and one indicated that they had used it clinically. Residents supplied with the tool were no more likely than control residents to document a positive cancer family history in patients' charts (41 percent versus 48 percent), but they were statistically significantly more likely to document a negative family history (89 percent versus 48 percent). The study had low statistical power to detect small to medium effects, and the residents supplied with the tool also received extra educational intervention compared with controls.

Sweet and colleagues⁵⁰ describe the JamesLink system, which is a touch screen, patient-completed tool for capturing family history data. In a study of 362 ambulatory cancer patients, data for 165 indicated moderate or high risk status when reviewed by a geneticist; of these, 16 percent were consistent with a family cancer syndrome. Of 101 patients in the high risk category on the basis of tool data, the chart records suggested family cancer history for 69; seven of the latter had received a full genetics assessment. It was noted that the charts of only 69 percent of patients using JamesLink had family history information available.

Grover and colleagues⁴⁷ prospectively assessed concordance between family history information captured by a paper-based, patient-completed family history questionnaire and then subsequently (and independently) recorded in their cancer clinic charts. They noted discordance between data recorded by the two methods. For 127 (41 percent) of the cases in which there was discordant data, 37 charts (29 percent) had reported a negative cancer history, or not documented a cancer history, which was captured by the tool. For 69 patients (54 percent), only some cancers captured by the tool were documented in the notes, and in 21 patients (17 percent), the tool and the notes were completely discordant. Charts did not document 32 percent of cancers reported by patients in the tool, and a third of notes missed cancers in 1DRs captured by the tool.

Murff and colleagues⁵² compared a paper-based, self-completion family history questionnaire with the charts of 310 internal medicine patients. They noted that the tool identified more 1DRs and 2DRs with colorectal, breast, or ovarian cancer than the charts and were more likely to capture the age of diagnosis for affected relatives, as well as more likely to identify relatives who were diagnosed before the age of 50. For all cancers together, the age of diagnosis was recorded in the chart for about 62 percent of affected 1DRs compared with 95 percent of those captured in the tool. The corresponding figures for 2DRs were 27 percent and 76 percent, respectively. These differences were highly statistically significant. Out of 48 patients who were identified as being at increased risk, the tool identified 29 who would have been missed by charts alone.

In summary, compared to genetic interviews as a gold standard, many FHxTs performed well. However, the studies reported here are limited because the genetic interviews were not supplemented with confirmation of relatives' reported medical histories. Compared with current practice, generally the family history documented in patient charts, FHxTs appeared to identify more relatives, more relatives with cancer, and more details about these relatives. In some cases, this would lead to reassignment of risk category and altered prevention plans. Again, validation of the “true” status of relatives was not performed.

Quality Assessment of Studies

Quality assessment using standardized checklists was undertaken on seven observational studies, five parallel RCTs, and one study⁵¹ that was a crossover trial in which cancer patients were randomized to the order of either a personal interview or a survey and a second study. The quality scores for the seven observational studies^{10, 11, 13, 34, 46, 48, 53} ranged from 14 to 21, thereby indicating a moderate to high level of quality. Initial reporting of hypotheses, interventions, outcomes, and sample characteristics was transparent and complete. However, the authors of only three of the studies^{34, 46, 53} listed important confounders (two adjusted for confounding in the analysis^{46, 53}) and one author⁵³ reported on blinding. Reporting of subject recruitment was also lacking. Confirmation that subjects were representative of the entire population from which they were drawn was provided in two studies;^{11, 46} recruitment of cases and controls from the same source population was mentioned in three studies.^{19, 48, 53}

The five parallel RCTs scored either a 4^{36, 44, 55} or 5^{39, 57} on the extended Jadad quality scale.¹⁰⁹ Major quality issues centered around a failure to describe randomization,^{44, 55} non-reporting of blinding,^{36, 39, 44, 55, 57} and non-reporting of withdrawals,^{44, 55} or methods used to assess adverse effects.^{36, 39, 57}

The absence of information on issues such as recruitment, randomization, and blinding suggests potentially biased results. Since it is not possible to assess whether the absence of information is linked to poor methods or poor reporting, the actual impact of any biases cannot be ascertained.

Other Methodological Aspects. Few studies described a sample size calculation.^{23, 36, 37, 39, 42, 49} Further, for comparative studies where concealment was necessary in qualitative assessment of the FHxT, only a few studies provided evidence that this had been performed.^{43, 49}

The participants of most studies would have had a better recall of their family history than the general public due to the fact that very few studies used an unselected general population.^{46, 48, 49, 54} Special populations included, for example, respondents with the cancers of interest,^{47, 51} or on a cancer registry,²⁵ and patients seen in specialist clinics.⁴²–^{45, 50} Also, the sequence of FHxT evaluation against comparator may have affected patient recall. The FHxT was given first followed by the best estimate in six studies.^{23, 43}–^{45, 47, 49} In one study, interpretation would have been affected by the paper family history questionnaire and structured “best estimate” interview having identical formats, with both approaches being delivered immediately after each other.⁵¹ Other study designs affecting interpretation included non-randomized allocations^{46, 49, 52} and variable response rate to FHxT. When reported, this varied from 40 percent⁴⁹ to 98 percent.⁴⁷ Non-completion of items accounted for about half the errors in an in-office self-completed FHxT.⁴⁵

General Approach

For the purposes of this review we followed the definition of RAT as described in Chapter 2. Some papers were identified which described tools consistent with this definition but which were not developed for use by PCPs, or were evaluated in settings other than primary care. We included some where we considered them to be “potentially applicable to primary care”, in that they did not appear to require specialist genetics knowledge to be applied as intended.

Studies Reviewed

Sixteen publications, representing ten distinct tools, were included in this section of the review. Full study details are summarized in evidence tables (Appendix C ^*), which include information on the evidence cited in support of risk stratification and/or recommended clinical actions. Table 9 presents a description of the tools, assessed against the defined tool characteristics. All tools fulfilled the criterion of timing of use (designed to be used before the health professional or patient takes the relevant decision).

Description of Tools

Cancer Type. Six tools, reported in seven papers,⁴³–^{45, 58}–⁶¹ were designed to assess risk of breast or breast/ovarian cancer only, four tools (seven papers) were designed to assess risk of breast/ovarian and colorectal cancer,^{31, 36}–^{39, 62, 63} and one tool (two papers) focused on breast/ovarian, colorectal and prostate cancer.^{40, 41} No tool was identified that focused solely on ovarian cancer risk, colorectal cancer risk, or prostate cancer risk.

Clinical Purpose of Tool. All ten tools (16 papers) were designed to, in simple or complex ways, stratify individuals into risk categories, and all had a component which indicated some form of clinical or personal action.

Target User. Three of the tools^{31, 44, 45} were designed for use by patients or the general population, the remainder having been designed for health professionals.

Knowledge Component. Each of the ten tools indicated at least one basis for the knowledge component. These components included: the Claus model;³⁶–^{39, 43, 44} the Gail model;^{31, 40, 41} national recommendations (e.g., French National Agency for Health Evaluation,^{40, 41} the Australian National Breast Cancer Centre,⁴⁵ the U.S. Preventive Services Task Force,⁵⁸ and the Scottish Executive Health Department;^{62, 63} guidelines developed by professional groups (e.g., the UK Cancer Family Study Group^{43, 60, 61} and the American Medical Association;^{31, 58}) and guidelines developed by local groups.^{36, 37, 58, 59} For one tool (four papers),³⁶–³⁹ it was indicated that it was designed to facilitate the implementation of appropriate knowledge components in general, not any specific guideline or risk calculation program.

Implementation Format. Five of the tools (nine papers)³⁶–^{41, 44, 62, 63} were presented in a computer or web-based format and the other five (six papers)^{43, 45, 58}–⁶¹ were presented in document-based format (Table 10). The five computer-based tools incorporated some form of family history data capture with risk calculation and guideline-based recommended actions.^{31, 36}–^{41, 44, 62} Of the document-based tools, one was a paper-based form with checklist for each relative and an embedded scoring system,⁵⁹ two were paper questionnaires incorporating suggested actions;^{43, 45} one was a pocket laminated card;⁵⁸ and one was an information pack with a laminated card and other components.^{60, 61}

Applicability to Primary Care. Of the seven tools intended for use by professionals, five were developed explicitly for use by PCPs—either family physicians (four tools, 9 papers)³⁶–^{39, 58, 60}–⁶³ or physicians working in ambulatory care settings (one tool, two papers).^{40, 41} Two appeared to have been developed in settings other than primary care, or without involving primary care practitioners, but intended for eventual use in that setting.^{43, 59} One patient tool³¹ was developed in a primary care setting, and the other two ^{44, 45} were considered potentially applicable to use in primary care settings.

Evidence of Effectiveness. Findings related to the development of one distinct tool (RAGS/GRAIDS)³⁶–³⁹ is presented across a number of publications. In general, we report findings for this as one distinct tool, but, where appropriate, we present (and clearly indicate) separate data relating to the evaluation of the prototype version (RAGS)^{38, 39} and the current version (GRAIDS).^{36, 37} For four tools (nine papers)³⁶–^{39, 44, 60}–⁶³ data were presented relating to effectiveness against a defined comparator, in achieving outcomes relevant to supporting decisions by users in practice. One tool³¹ was evaluated in an uncontrolled before-after study.

Data are reported to the evaluation of four tools (seven papers)^{31, 36, 37, 60}–⁶³ implemented in routine practice settings, including the GRAIDS tool, and three studies of two tools^{38, 39, 44} where evaluations were conducted under “laboratory-type” conditions, including the RAGS prototype tool.^{38, 39} Table 11 summarizes the key points of these studies, including the range of outcomes measured. The remaining studies were tool development or descriptive studies, or the outcomes presented related to the validity or evidence base underlying the stratification system used rather than practice related outcomes.

Quality Assessment of Studies

Standardized quality assessment checklists were employed on the five studies that used randomized trial design. The Jadad scores ranged from 4 to 6.^{36, 39, 44, 60}–⁶³ Major problem areas were a failure to report whether the studies were blinded^{39, 44, 60, 62} and a failure to report numbers of withdrawals.^{44, 60, 61}

The potential for bias in these studies appears quite low. Concerns about non-differential misclassification are always relevant when there is no blinding, but it is impossible to say whether subjects and investigators were not blinded or whether the authors of the manuscripts simply omitted mention of blinding in their published articles.

Outcomes

Of the evaluative studies of tools directed towards professionals, one (two papers) (the RAGS prototype) was conducted under “laboratory-type” conditions^{38, 39} and three (five papers) were implemented in routine practice settings,^{36, 60}–⁶³ including the GRAIDS tool.^{36, 37} In the first of these, the computer-based RAGS prototype application^{38, 39} was compared with pen and paper risk calculation and a specialist risk calculation software package, Cyrillic. The evaluation showed a statistically significant effect of the tool on clinical management decision making for hypothetical cases presented in vignette form. In the study by Watson and colleagues,^{60, 61} a hereditary breast cancer information pack (presented with or without an active educational co-intervention) was compared with no intervention. An analysis of referral letters subsequently received by the relevant genetics centers and breast clinics indicated a statistically significant trend across the three groups in terms of compliance with referral criteria. In the study by Emery and colleagues,³⁶ a randomized controlled cluster trial was used to evaluate a complex intervention which comprised a web-based decision support system (the GRAIDS software, for which RAGS was the prototype) and a nominated “lead clinician” within the practice who received extra training in use of the software and was expected to manage all patients expressing concerns about family history of colorectal or breast cancer. All physicians and nurses in intervention practices also received a short educational session on cancer genetics and an introduction to the GRAIDS software. The control intervention was a mailed paper copy of the relevant regional guidelines, along with a short educational session on cancer genetics. The intervention arm contained an “adaptive” sub-group, in which extra training or software adjustment was used to increase actual use of the intervention. The primary outcome was appropriateness of referrals made to the regional genetics clinic, as assessed by comparison of each referral letter with the regional guidelines. For both cancer groups combined, 95 percent of referrals made by physicians in the intervention group met the guideline criteria, compared with 79 percent in the control group, a statistically significant result. For breast/ovarian cancer referrals, the proportions were 93 percent and 73 percent, respectively (statistically significant) and for colorectal cancer referrals, the proportions were 99 percent and 92 percent (not statistically significant). Overall, there were no statistically significant differences in proportions of patients who were subsequently assessed as being at increased cancer risk by genetics specialists. At the patient level, cancer worry scores were lower in those referred from intervention practices than from control practices, but no statistically significant differences were observed in knowledge or risk perception scores. The fourth study^{62, 63} compared a stand-alone computer based decision support tool with a control intervention of national guidelines disseminated by mail to family physicians. All practices within the health care administrative region were included in the trial, and all intervention practices received the intervention in some form. The primary outcome was physician confidence in four domains related to assessing risk, making clinical management decisions, and counseling patients, and no statistically significant differences were detected between intervention and control groups for any of the four domains. No statistically significant differences between groups were observed in secondary outcomes related to patients' risk perceptions, beliefs about breast cancer causation, or the risk of referred patients as assessed by genetics specialists.

Of the evaluation of tools directed towards patients, one was conducted under laboratory-type conditions,⁴⁴ and one was evaluated under conditions approaching routine practice.³¹ The former⁴⁴ was an evaluation of the patient oriented “GRACE” tool. It was framed as an equivalence or non-inferiority trial, but was not statistically powered for testing of a priori hypotheses. The comparator was a consultation with a nurse specialist who used the same evidence base to assess risk and offer advice. Outcomes related to patient acceptability, risk perception, anxiety and cancer worry, were all either statistically non-significant, or favored the control arm. In the second study;³¹ the Cancer Risk Intake System (CRIS), a touch screen system for patients, was implemented in three primary care clinics. On the basis of family and other history, patients received tailored printouts including up to three messages regarding cancer prevention, to be used as an aid for discussions with their physician. A before-after evaluation suggested that the proportion of patients reporting a physician discussion about tamoxifen use increased from 4.8 percent at baseline to 27.7 percent after using CRIS; the corresponding pre- and post-figures for cancer genetic counseling were 2.8 percent and 28.2 percent, and for colonoscopy were 16.1 percent and 45.2 percent. The lack of a control intervention makes it difficult to assess the extent to which completing the baseline survey acted as a co-intervention.