4Discussion

Perspective

The key questions of this study involve broad analyses that should encompass several model components simultaneously, to determine how model factors affect outcomes among medically complex geriatric hip fracture patients. Current hip fracture outcomes conclusions are drawn largely from information derived from only a few model components at a time. Therefore, the resulting outcomes information lacks context and applicability within the greater realm of salient hip fracture outcomes. Moreover, the most commonly studied outcome is mortality, while perhaps more important and relevant outcomes, such as pain and function, are less frequently and more inconsistently addressed.

RCTs conducted by the orthopaedic surgeon community often focused on short-term process factors that were of interest to surgeons and the manner in which operations are carried out. Factors such as the type of fracture, operative treatment details, and short-term implant-related complications were often descriptively reported, with much less emphasis placed upon functional outcomes that are meaningful to patients. Very few outcomes were assessed from the patient’s perspective. Baseline patient data was often collected, but it was often an incomplete set of patient characteristics and subsequently not used in the analysis. Further, the full complement of other factors relevant to patients’ functional outcomes, such as inpatient protocols and rehabilitation, were not collected. Therefore, conclusions from the RCT literature regarding patient outcomes are tenuous at best.

Conversely, the observational literature, although replete with patient factors, often disregarded fracture type, fracture stability, and type of implant and assumed a homogeneous effect of these factors on functional outcomes. While these studies laid a solid foundation for further inquiry of patient factors because they relate to functional outcomes, the existing observational studies tend to overlook a large segment of U.S. hip fracture patients who are physically dependent, cognitively impaired, or both. Indeed, it is important to distinguish the clinical course of patients who fracture their hips in nursing homes, and are destined to return there, from those admitted from the community. The Mayo Clinic population-based studies were exceptions to this observation.⁹

Study Quality

In general, although the orthopaedic study quality was low, it is improving, specifically within the last several years. We note both improvements in study conduct and improvements in the clarity of reporting, particularly in patient tracking and reporting post-discharge including better compliance with CONSORT recommendations.

This new initiative to improve outcomes reporting will likely result in the ability to link fracture type, pattern, and implant to outcomes only if well-designed, sufficiently powered trials with adequate followup and consistently used outcome measures are conducted. Better effort needs to be made to reassess debilitated elderly patients after acute inpatient stays in order to minimize nonmortality losses to followup. Unfortunately, the literature that was available for this review reflects a period of research prior to this effort and underlines the reasons for which the research community undertook improving research quality. The following discussion of the limitations of the literature can be used in a constructive manner to encourage the research community to continue the improvement trajectory that has already begun.

The most important factors within the RCTs that substantially limit the strength of the study conclusions are high and ambiguous patient attrition, inadequate power, inadequate or unreported randomization schemes, and poor comparability of outcome measures across studies.

High and ambiguous patient attrition. Attrition imposes two problems: (1) bias and (2) sample loss, which can affect power. Although mortality is known to be high among elderly hip fracture patients, it is important to distinguish death from sample loss. Mortality is an important outcome in its own right. It may be the ultimate functional outcome. Given a high mortality rate, many treatment advantages may be measured in terms of time in various functional states (i.e., months of improved function).

The reporting of losses to followup was poor and often difficult or impossible to determine. Studies commonly included inpatient mortality, but most lacked the number of patients analyzed per group at each followup time point after the acute inpatient stay. Few studies included the number of patients who followed up at intermediate assessment points, and many studies did not report the number of patients per treatment group that were assessed at the final followup. Instead, authors often reported intermediate and final outcomes as percent of cases, without clearly stating the actual number of patients examined per time point. The reasons for loss to followup were often vaguely reported, if at all, commonly mentioning “ill-health” as the reason for nonassessment. In other cases, the flow of patients through a study was reported by CONSORT statements, a very welcome recent improvement to study reporting in this literature. However, frequently the number of patients for whom functional outcomes were reported was not equivalent to the number of patients reported in the CONSORT statement, and the reason for the discrepancy was not reported. While mortality as a source of attrition is difficult to address, some investigators did take measures to minimize loss to followup through employing telephone interviews for patient assessed functional outcomes.

Another issue was investigator selection of analysis samples. Investigators often chose to exclude patients with implant failure or fracture nonunion from any further assessments and from all subsequent analyses. Analyses by subsamples of patients may be necessary to assess technical aspects of a device. However, if these subsamples are used for functional assessments, it may lead to biased results. Any subgroup analysis must be determined in advance and cannot be based on the outcomes.

Randomization issues. Randomization for surgical procedures for hip fractures, especially when many of the procedures are handled as emergency surgery events, is complicated by the fact that the surgeon may not have the full set of information necessary for clinical decisions until the surgery has begun. While an unavoidable complication, even with the best intentions, it leaves the door open for systematic bias to enter the randomization process. A number of studies reported randomization of patients prior to full application of exclusion criteria. For example, in some cases patients whose fractures were unable to be satisfactorily reduced were excluded after having been randomized to a study arm. Further, reporting of the randomization process itself was often cursory. About half of the studies reported the randomization scheme to be closed or in sealed envelopes. Only approximately one-quarter of included studies had information about sequence determination and implementation, which included randomization processes not recommended, such as basing randomization on medical record numbers or days of the week.

Other patient population issues. A large proportion of studies contained liberal inclusion criteria and minimal exclusion criteria, if any. Investigators with the stated or implied study focus of geriatric hip fracture patients often enrolled at least 10–20 percent of high energy trauma patients, those younger than age 50. Except in only a few studies, these patients were often not isolated in the analysis or by treatment group, likely due to problems with low power. A significant number of studies, particularly those prior to 1996, made no attempt to exclude patients with pathologic, cancer-related fractures, nor distinguish them in the analysis.

One important factor for determining inclusion criteria is identification of the fracture type and pattern. Surgeons used multiple fracture classification systems to identify fracture patterns, and the mapping of these patterns across classifications is not consistent. Additionally, surgeons show little reliability in their own ability to use these classification systems, which varies with surgeon experience and classification scheme.²⁰³ Since hip fracture patterns are varied and complex, nearly all studies aggregated multiple fracture subtypes based on the general stability of the collective fracture patterns to enable them to state conclusions about categories rather than subtypes of fractures. Often, this aggregation was needed, since certain fracture patterns are uncommon and few cases were available of some fracture subtypes despite long enrollment periods. For femoral neck fractures, the aggregation of subtypes into displaced or nondisplaced was highly consistent across studies, where Garden III-IV patterns were consistently labeled as displaced fractures.

In contrast, the aggregation of fracture patterns into stable or unstable categories within the intertrochanteric fracture studies was highly inconsistent and undermines many of the conclusions drawn that were based on fracture stability. Within the AO/OTA fracture classification system, (Figure 9 in Chapter 1) there are nine patterns of pertrochanteric fractures, and the frequency of each of fracture subtype is not evenly distributed. In the Fung et al. study, there was no consensus among surgeons about how to dichotomize the classifications into stable and unstable patterns.²⁰³ Thus, it is difficult to conclude if there was consistency in the stable versus unstable grouping of fractures within the existing RCTs, particularly among fractures in the mid-range of the intertrochanteric fracture classifications, the OA/OTA 31-A2 subtypes. Surgeons’ decisions to label fractures as unstable in RCTs when other surgeons would label them as stable fractures (i.e., AO/OTA 31-A2.1) artificially increases sample sizes when studying unstable fractures. Such sample size augmentation served to buffer the complication rates among fractures labeled as unstable, since the unstable treatment groups subsequently consisted of what other surgeons consider to be both stable and unstable fractures. In one study of unstable intertrochanteric fractures that included only AO/OTA 31-A2 fractures (three subtypes), two-thirds of patients had AO/OTA 31-A2.1 fractures, which are commonly considered to be stable fractures.¹¹⁷ This aggregation pattern was common among studies of unstable intertrochanteric hip fractures, and subtype grouping within any of the classification systems appeared to be investigator dependent.

Inconsistent baseline data were collected across trials, and even less were reported. The patient factors that were described in the methods section of each study often far exceeded what was included in the patient baseline information table. Patient age and gender were consistently reported. However, other factors known or believed to impact functional outcomes and mortality, such as level of pre-fracture function, race or ethnicity, fracture pattern, pre-fracture residence, cognitive impairment, injury mechanism, and comorbidities, were often not reported. Of particular note is the lack of race or ethnicity in patient baseline information.

Sample size and power. Few orthopaedic outcomes studies reported a sample size calculation or discussed power in relation to their primary study outcome. The average number of patients per study arm was approximately 75 patients prior to attrition, and the final number of cases analyzed was often not accompanied by a sample size calculation. The majority of studies lacked a sufficient number of cases to detect a clinically important difference between two treatment groups for a given outcome, even if one existed. In general, it appeared from discussion comments that many authors underestimated the magnitude of the attrition from high mortality and other losses to followup among frail, geriatric hip fracture patients. Also, the numbers of patients enrolled in RCTs are frequently too small to control for other factors (surgeon and hospital factors, rehabilitation site) on outcomes (function, mortality, pain, residence).

Inconsistency in functional outcomes. Table 3 in Chapter 3 clearly illustrates the idiosyncratic nature of outcome reporting that has historically predominated in the literature. Most of the outcomes reported in trials published prior to 2000 used only surgeon-reported outcomes. Many RCTs in this analysis did not report any functional patient outcomes. Of those with functional outcomes, many did not report group scores in a useable manner. Outcomes tables frequently lacked the number of patients in the analysis sample by group, or any identifiable measure of variation (range, standard error of the mean, standard deviation), impairing comparisons.

Other quality issues. We note a recent, positive shift in the style of outcomes reported in trials from exclusively surgeon-reported outcomes to both patient- and surgeon-reported outcomes. The benefits of using validated quality of life and outcomes assessment tools such as the SF-36, EQ-5D, WOMAC, and others is not yet fully reflected in the existing literature. Although we find this an encouraging sign, this trend is not yet consistent and appears to be investigator dependent.

Despite considerable efforts in the RCT literature to describe the technical aspects of performing hip fracture procedures, factors known to be both critical to the success of a procedure and surgeon dependent were often not reported. The degree to which fractures were realigned or reduced in relation to anatomic alignment, the extent to which the reduction held while an implant was placed, and the proximity of the final implant position to optimal were reported in a minority of studies. While the studies that included such details were explicit in doing so, few attempted to associate the quality of the reduction and implant position to implant failure or reoperation. None attempted to associate fracture or implant position to functional outcomes.

Surgeries performed by orthopaedic residents were common. A number of studies reported that residents performed all or most of the operative cases in the study.^{45,47,102,114,124,160,162,163} Another fifth of the studies used large numbers of surgeons, where experience level undoubtedly varies widely, or explicitly stated the staff included both junior and senior surgeons. When both experienced surgeons and residents performed cases, we most often could not tell which surgeries were performed by residents. Few studies separated outcomes by surgeon seniority.

General levels of surgical experience are commonly not reported, or are only reported descriptively and collectively, and not by treatment group. Surgeons’ experience with each device in a given study was rarely reported. This was particularly common in the intertrochanteric hip fracture studies, where intramedullary nails had previously been used for long bone fractures but were newly used for fractures of the hip. Many surgeons had no experience with intramedullary nails or only two to five cases prior to study participation, yet were reported to be accustomed to the plate/screws devices used in the other treatment arm. Specific familiarity with each device per surgeon was rarely reported.

Many RCTs tested newly introduced devices. However, once the initial new implant questions of complication rates, implant failures, and other post-operative complications, such as infection, were moderately addressed in comparison to an existing device, studies often failed to inquire further with additional studies to evaluate outcomes that are meaningful to patients. Equivalent complication rates do not necessarily equate to equivalent functional outcomes. Also, followup data can be misleading if critical groups are eliminated. Followup among patients who experienced implant failure was often incomplete or fully excluded from analyses. Not all patients with nonunion, implant failure, and device related pain underwent another surgery. Therefore, reoperation rates, particularly in this elderly patient population, likely underreport actual complication rates, since many older patients are too frail to undergo an additional procedure after their hip fracture surgery. Reoperation rates have also been shown to be subject to local variation, even within study protocols.¹⁸⁵ It is important to report on the outcomes of patients who had postoperative orthopaedic complications but did not have additional surgery to correct the problem, since the RCT literature lacks information as to which complications most adversely affect postoperative function, particularly ambulation.

The effects of post-hospital therapy, including rehabilitation, were often excluded entirely from hip fracture RCT analyses. Most studies that briefly listed in-hospital mobilization protocols indicated that protocols were the same for both treatment groups but provided no further information regarding patients’ post-acute care. Most studies did not report the type, intensity, and duration of post-acute rehabilitation. Inpatient stays have shortened considerably in the United States in recent years, accompanied by an increased used of post-acute rehabilitation, which is not reflected in the RCT literature we reviewed. Also, nearly all RCTs were conducted in Europe, which reduces comparability to U.S. populations.

Current analysis practices allow too much room for both Type I and Type II errors in relating clinical characteristics and interventions to outcomes. Significance levels were often not corrected for multiple outcome comparison, which opens the door for finding significant differences in devices that could be purely due to chance. The study by Utrilla et al. is an example.⁹² Only one outcome was found to be significant at an unadjusted level. Had the outcome been adjusted to account for the possibility of significant difference in one out of seven outcomes purely by chance, there would have been no differences found between the devices. On the other hand, the small patient numbers previously mentioned, compounded with loss due to mortality and attrition, increase the likelihood of a Type II error, not finding a difference when there was one. While reporting of power calculations has been disappointingly low, the frequency of power calculation reports has increased in recent years.

Among studies that used existing assessment tools for THA patients, such as the Harris Hip Score, many patients scored below the lowest possible category (<70=poor) at all followup time points. Differences across treatment groups were often nonsignificant, which may be due to a lack of instrument sensitivity to minimal differences in highly disabled patients. Differences in mean values across treatment groups that were reported as statistically significant most often would not be expected to show relevant clinical differences.

Outcomes assessors were generally not blinded to the treatment, which increases the opportunity for biased assessment and reporting. Several recent studies identified that the functional assessors were not directly involved with the study, but only one study indicated that the patients remained clothed during functional assessment so the examiner could not determine from the incision which surgery a patient had undergone.

Finally, it was difficult or impossible to determine the degree of industry funding for many studies, particularly those prior to 2000. After that time, formal funding disclosure statements started to appear in articles from several major journals. Many studies appeared to be entirely unfunded works performed within one hospital or one academic center. Still, despite some disclosures of study funding, most did not contain specific details about the presence or absence of individual author’s consulting or design arrangements with device or bone cement companies.