Home > Methods - Surgical Options for Inguinal...

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

Treadwell J, Tipton K, Oyesanmi O, et al. Surgical Options for Inguinal Hernia: Comparative Effectiveness Review [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012 Aug. (Comparative Effectiveness Reviews, No. 70.)

  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Methods

Review Team

The evidence review team included expertise in medicine, surgery, systematic review, public health and health services research. Additional content expertise was provided by experienced hernia surgeons who were involved as key informants and/or members of the technical expert panel; these groups provided input on the Key Questions, reviewed the protocol, answered specific questions during the review process, and reviewed the document.

Topic Development and Refinement

Development and refinement of the topic occurred between June 16, 2010, and October 15, 2010. This process involved reviewing this clinical area, devising an initial analytic framework and list of Key Questions, obtaining the input of five key informants, revising the Key Questions and scope based on feedback received, and posting for public comment. The key informants included one surgeon with expertise in adult hernia surgery, a surgeon with expertise in pediatric hernia surgery, two individuals from payer organizations, and one individual from a mesh manufacturer.

Finalization of the protocol occurred between October 15, 2010, and April 8, 2011. The Key Questions were posted on the AHRQ website for public comments for one month. We also received input from the technical expert panel, which comprised four individuals: a surgeon from the United States with expertise in adult hernia surgery who had also been a key informant; a surgeon from the United Kingdom who also had expertise in adult hernia surgery; a surgeon who had expertise in pediatric hernia surgery and who had not been a key informant; and a product specialist from a mesh manufacturer.

Analytic Framework

Figure 1 is an analytic framework that depicts the events that individuals experience while undergoing treatment for inguinal hernia. Throughout the figure, numbered circles indicate Key Questions addressed in this report. The left side of the framework lists the four patient populations: (1) adults with pain-free primary inguinal hernia, (2) adults with painful inguinal hernia without strangulation, (3) pediatric patients with possible contralateral inguinal hernia, and (4) pediatric patients with inguinal hernia (the last two populations can overlap). To the right of these are the various intervention options, including watchful waiting (WW), as well as several surgical options. The surgical options are categorized based on whether the approach is open or laparoscopic and whether a mesh is used. Postintervention outcomes are divided into three categories: surrogate outcomes (operation time and conversion to open, neither of which were included in this report), patient-oriented effectiveness outcomes (RC, length of hospital stay, hospital visits, office visits, return to daily activities [RTDAs], return to work [RTW], quality of life [QOL], and patient satisfaction [SFN]), and adverse events (including chronic pain, infection, small bowel perforation, and hematoma).

The analytic framework is intended to show how the nine Key Questions fit within the general scope of inguinal hernia management. It is divided into four columns vertically, reading from left to right as Populations, Interventions, Surrogate Outcomes, and Patient-Oriented Outcomes. The Populations column lists the different types of patient populations of interest, including adults with pain-free primary inguinal hernia (Key Question 1), adults with painful inguinal hernia without strangulation (Key Questions 2 through 7), pediatric patients with possible contralateral hernia (Key Question 8), and pediatric patients with primary inguinal hernia (Key Question 9). Each of these populations has a relevant choice between interventions. For adults with pain-free primary inguinal hernia, it is the choice between watchful waiting and surgery; for adults with painful inguinal hernia without strangulation, it is the choice between different types of surgery; for pediatric patients with possible contralateral hernia, it is the choice between watchful waiting and surgery; for pediatric patients with primary inguinal hernia it is the choice between open surgery and laparoscopic surgery. These intervention choices are represented in the second column, labeled Interventions. Key Questions 2 through 7 for adults, as well as Key Question 9 for pediatric patients, are represented in this column. Key Question 2 involves comparisons between open mesh surgery and laparoscopic mesh surgery, and has three subquestions (for primary, bilateral, and recurrent hernia). Key Question 3 involves comparisons within the group of open mesh procedures (and therefore the number appears within the box for open mesh), and Key Question 4 involves comparisons within the group of laparoscopic mesh procedures (and therefore the number appears within the box for laparoscopic mesh). Key Questions 5 (comparison among meshes) and 6 (comparisons among fixation methods) each appear in both the box for open mesh and the box for laparoscopic mesh because they apply to both sets. Key Question 7 only applies to the latter however, as it involves the relationship between surgical experience with laparoscopic repair and hernia recurrence. The third column lists two surrogate outcomes, operation time and conversion to open. These outcomes are measured during or soon after surgery, and are not included in this report, because they are of less importance than the ones listed in the final column, Patient-Oriented Outcomes. These latter outcomes include all of the outcomes to be examined in this review, including hernia recurrence, hospital visits, length of hospital stay, office visits, return to daily activities, return to work, quality of life, patient satisfaction, and adverse events.

Figure 1

Analytic framework. Note: Circled numbers are Key Questions

Key Questions

This report addresses nine Key Questions, which are listed below. Most questions refer to “patient-oriented effectiveness outcomes.” These include RC, hospital visits, length of hospital stay, office visits, RTDA, return to work, QOL, and patient SFN. We also examined adverse events, including rates of long-term pain.

Among adults with pain-free primary inguinal hernias:

Key Question 1. Does hernia repair differ from WW in patient-oriented effectiveness outcomes and/or adverse events?

Among adults with painful inguinal hernias without incarceration/strangulation:

Key Question 2. Does open hernia repair with a mesh differ from laparoscopic hernia repair with a mesh in patient-oriented effectiveness outcomes and/or adverse events?

Key Question 3. Do different open mesh-based repair procedures (e.g., Lichtenstein repair, mesh plug) differ in patient-oriented effectiveness outcomes and/or adverse events?

Key Question 4. Do different laparoscopic mesh-based repair procedures (e.g., TAPP repair, TEP repair) differ in patient-oriented effectiveness outcomes and/or adverse events?

Key Question 5. Do different mesh products differ in patient-oriented effectiveness outcomes and/or adverse events?

Key Question 6. Do different mesh-fixation methods (e.g., no fixation, sutures, glue) differ in patient-oriented effectiveness outcomes and/or adverse events?

Key Question 7. For each type of laparoscopic mesh repair, what is the association between surgical experience and hernia recurrence?

Among pediatric patients (aged 21 years or younger):

Key Question 8. For a possible contralateral hernia, does same-operation repair/exploration differ from WW in patient-oriented effectiveness outcomes and/or adverse events?

Key Question 9. Does open hernia repair without a mesh differ from laparoscopic hernia repair without a mesh in patient-oriented effectiveness outcomes and/or adverse events?

Search Strategy

Information professionals performed literature searches within the Evidence-base Practice Center (EPC) Information Center who followed established guidelines and procedures as identified by the Director of Health Technology Assessment/EPC Information Center. Below is an overview of the search process; specific search strategies are listed in Appendix A.

Consistent with our evidence-based search protocol, for all Key Questions the following databases were searched on the OVID SP platform, utilizing the one search and deduplication features: MEDLINE and PreMEDLINE; Embase; the Cochrane Library, including the Central Register of Controlled Trials, the Cochrane Database of Methodology Reviews, the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects, and the Health Technology Assessment Database; and the United Kingdom National Health Service Economic Evaluation Database, were also searched for unique reviews, trials, economic analyses, and technology assessments.

Search terms were identified by: (1) reviewing relevant systematic reviews on similar topics that are identified by the research staff, (2) reviewing how other relevant studies are indexed, their subject heading terms, and keywords, and (3) reviewing MeSH and EMTREE indexes for relevant and appropriate terms. After reviewing these, a combination of subject headings and keywords were identified. Search strategies were developed using these terms. Once developed, search strategies were reviewed by senior research analyst(s) and the Director of the Health Technology Assessment/EPC Information Center. No limits on language were applied by the search, and search dates were established as January 1, 1990, to November 17, 2011 (studies published before 1990 likely describe procedures no longer being used commonly or outcomes that are not likely to be predictive of current outcomes). A study design filter was applied to retrieve systematic reviews and clinical trials. We also examined reference lists for possible additional articles. Searches will be updated during the peer-review period, and any additional studies will be incorporated into the final report. Nonjournal publications and conference proceedings from professional organizations, private agencies, and government agencies were also screened. Other mechanisms used to retrieve additional relevant information included review of bibliographies and reference lists from peer-reviewed and gray literature. (Gray literature consists of reports, studies, articles, and monographs produced by federal and local government agencies, private organizations, educational facilities, consulting firms, and corporations. These documents do not appear in the peer-reviewed journal literature.)

Literature search results were initially reviewed by the information professional. Using the Key Questions and inclusion/exclusion criteria identified by senior research analyst(s), the information professional assessed relevancy and retrieved results. Feedback from the senior research analyst(s) and the Director of the Health Technology Assessment/EPC Information Center—including details regarding gaps in the search strategy, as well as articles (identified by the senior research analysts) not retrieved by the searches—was integrated into the search strategy using key terms and subject headings. The updated strategy was re-run in all identified databases. Additional results were scanned, and relevancy was assessed by the information professional. New results were downloaded and forwarded to senior research analyst(s) for review. Hand searches of reference lists in identified articles were also reviewed for possible inclusion.

To check the accuracy of abstract screening inclusion/exclusion, we randomly selected 10 percent of the articles excluded at the abstract level for rescreening by a second person. None of the articles were subsequently selected for inclusion.

Study Selection

The inclusion criteria are listed below in separate categories pertaining to (1) publication type, (2) study design, (3) patient characteristics, (4) treatment characteristics, and (5) data.

Publication criteria

  1. Publication must have been a full article; abstracts alone were not included because they do not include sufficient details about experimental methods to permit an evaluation of study design and conduct, and they also may contain only a subset of the measured outcomes.32,33
  2. To capture the most relevant data, we included studies published on or after January 1, 1990. Studies published before 1990 likely describe procedures no longer being used commonly or outcomes that are not likely to be predictive of current outcomes.
  3. To avoid double-counting of patients, when several reports of overlapping patients are available, only outcome data from the report with the largest number of patients were included. We included the data when a smaller report provided data on an outcome that was not provided by the largest report. Multiple publications of the same study (e.g., publications reporting subgroups, other outcomes, or longer followup) were identified by examining author affiliations, study designs, enrollment criteria, and enrollment dates.
  4. Studies must have been published in English. Moher and colleagues demonstrated that exclusion of non–English-language studies from meta-analyses has little impact on the conclusions drawn.34 Juni and colleagues found that non-English studies typically were of lower methodological quality and that excluding them had little effect on effect-size estimates in the majority of meta-analyses they examined.35 Although we recognize that in some situations exclusion of non-English studies could lead to bias, we believe that the few instances in which this may occur do not justify the time and cost typically necessary for translation of studies to identify those of acceptable quality for inclusion in our review.34,35 Due to the prevalence of non–English-language studies of inguinal hernia repair, however, we examined the English abstracts of these studies in an attempt to assess the degree of bias resulting from their exclusion.

Study design criteria

5.

For questions comparing interventions (i.e., all Key Questions except Key Question 7 on surgical experience), the study must have either randomized patients to treatments or used an analytic method to address selection bias, such as intentional baseline matching on multiple characteristics, propensity scoring, or other analytic approach. Studies with large differences at baseline between groups (regardless of whether they were randomized), or that entailed confounding by indication, were excluded. Studies comparing meshes or mesh-fixation methods must not have confounded results by differences in surgical procedures for inserting the mesh. For Key Question 7 on surgical experience, a control group was not required; however, the study must have provided data on the relation between surgical experience and outcomes. The definition of surgical experience must have been specific to laparoscopic mesh hernia repair, not simply a measure of general experience such as the surgeon's age.

6.

Studies could be prospective or retrospective, but retrospective studies must have used consecutive enrollment (or enrollment of a random sample of eligible participants).

7.

The treatments being compared must have been administered during the same time period, so that any observed difference between treatment outcomes were not attributable to differential time frames.

Patient criteria

8.

To be included for a given Key Question, the study must have provided data for which at least 85 percent of the patients had the condition specified in the Key Question. For example, for Key Question 2a, we included only data points for which at least 85 percent of the patients were adults with painful primary inguinal hernia without incarceration/strangulation.

9.

We used a flexible definition of “adulthood,” defining “adults” as anyone aged 18 years or older, and we defined the “pediatric population” as anyone aged 21 years or younger. This means that studies enrolling those aged 18 to 20 years could have been included as either an adult study or a pediatric study, depending on the average age of those enrolled.

Treatment criteria

10.

The study must have provided sufficient information about the treatments for one to determine that the data addressed one of the Key Questions.

11.

The study must not have described a specialized and novel hernia repair that has not been widely practiced by other surgeons. This is to maintain the focus of the report on the most common types of repair.

12.

The hernia repair must not have been performed simultaneously with another operation (e.g., prostatectomy). Surgical complications of combined operations make it difficult to isolate aspects of the hernia repair itself.

Data criteria

13.

The study must have reported data on at least one of the included outcomes for at least one of the Key Questions.

14.

Outcome data must not have relied on retrospective recall (e.g., in an interview long after the procedure had been performed) because such outcomes may not accurately reflect patients' experiences.

15.

For some outcomes in the adult population, we included data points at least 6 months after treatment (RC, QOL, and patient SFN). For all other outcomes (and in the pediatric population), there was no minimum followup.

16.

We included data points capturing at least 10 patients with the condition of interest who represented at least 50 percent of eligible enrolled patients.

The principal investigator performed an abstract screen on all abstracts, and a randomly selected 10 percent of the abstracts were rescreened by a second person, with disagreements resolved by consensus. For full-article screening, the first screening was performed by the team member responsible for that Key Question, and we randomly selected 10 percent of the articles excluded at the full article for rescreening by a second person, with disagreements resolved by consensus. None of the articles were subsequently selected for inclusion.

Data Extraction and Management

We extracted study information into spreadsheets in Microsoft Excel, including the following:

  • General study characteristics. Author, publication year, country, study design, number of centers, dates of patient enrollment, type of setting, length of followup, funding source, and which Key Question(s) the study addressed
  • Patient enrollment criteria. All authors' reported patient enrollment criteria
  • Treatment characteristics. Specific procedure, specific mesh (if applicable), fixation method (if applicable), number of surgeons, surgeons' prior experience with the repair procedures performed, surgical setting (i.e., specialized hernia center or general surgery), type of anesthesia, and methods of followup for data collection
  • Baseline characteristics. Number of enrolled patients, age, sex, comorbidities, hernia type(s), presurgical pain level, presurgical quality-of-life scores, presurgical functional activity scores, unilateral/bilateral, primary/recurrent, and any other reported important patient characteristics at baseline
  • Risk-of-bias items. See the next section.
  • Data. We extracted the numerical data necessary for us to compute an effect size (such as an odds ratio (OR) or standardized mean difference) and its standard error for all included outcomes for each study. These may include means, standard deviations (SDs), counts, proportions, results of authors' statistical tests, or other statistical details, depending on what was reported. If the study did not report sufficient information to permit computation of an effect size, we extracted what was reported.

The data points were first extracted by the team member(s) responsible for that Key Question, and a 10 percent randomly selected subset of the data points were checked by a second person, with disagreements resolved by consensus.

Individual Study Risk-of-Bias Assessment

We assessed the risk of bias (i.e., internal validity) separately for each outcome and each time point of each study. The reason for outcome specificity is that some subjective outcomes are more susceptible to bias than other outcomes. The reason for time-point specificity is that longer followup often results in attrition or right-censoring, which may yield patients who are somewhat different from the full set of enrolled patients and also may introduce a systematic difference between the groups being compared.

For all studies with control groups (regardless of whether patients were randomly assigned to groups), we assessed risk of bias using the items below. All but one of these items were selected from a pool of items typically used by this EPC for technology assessments. The seventh item was devised specifically for this project because of the importance of length of surgical experience in hernia repair. Each of these items was answered as “Yes,” “No,” or “Not reported.”

  1. Were patients randomly assigned to the study's groups?
  2. Was there concealment of group allocation?
  3. For nonrandomized trials, did the study employ any other methods to enhance group comparability?
  4. Was the process of assigning patients to groups made independently from physician and patient preference?
  5. Was the comparison of interest prospectively planned?
  6. Were the two groups treated concurrently?
  7. For questions comparing two procedures, did the two groups' surgeons have similar numbers of prior operations performing the procedure they performed in the study? (This is not relevant to Key Question 1 because it does not involve comparing procedures.)
  8. If patients received ancillary treatment(s), was there a ≤5 percent difference between groups in the proportion of patients receiving each specific ancillary treatment?
  9. Did patients in different study groups have similar levels of performance on the outcome of interest at the time they were assigned to groups?
  10. Were the study groups comparable for all other important factors at the time they were assigned to groups?
  11. Were those who assessed the patient's outcomes blinded to the group to which the patients were assigned?
  12. Was the outcome measure of interest objective and was it objectively measured?
  13. Was there ≤15 percent difference in the length of followup for the two groups?
  14. Did ≥85 percent of enrolled patients provide data at the time point of interest?
  15. Was there a ≤15 percent difference between groups in the percentage of patients provided data at the time point of interest?

We categorized the risk of bias for each outcome/time point in each study as “Low,” “Medium,” or “High” risk of bias using the following method:

  • In order to be considered Low risk of bias, the study must meet the following conditions:
    • Randomized (item 1).
    • Concealment of allocation (item 2) OR blinded outcome assessors (item 11) OR both.
    • Good baseline comparability for both outcome (item 9) and other patient characteristics (item 10).
    • Good baseline comparability on surgeons' number of prior operations performing the compared procedures (item 7).
    • If NOT blinded outcome assessors (item 11) (or NR blinded outcome assessors), then the outcome was objective (item 12).
    • ≤15 percent difference in length of followup between groups (item 13).
    • ≥85 percent of enrolled patients provided data to this time point (item 14).
    • ≤15 percent difference in data provision rates to this time point (item 15).
  • In order to be considered High risk of bias, the study must meet AT LEAST TWO of the following conditions:
    • Process of assigning patients to groups NOT made independently from physician and patient preference (item 4)
    • Not good baseline comparability for either the outcome (item 9) or other patient characteristics (item 10)
    • Retrospective (item 5)
    • Difference in ancillary treatments ≥5 percent (item 8)
    • Not a blinded outcome assessor (item 11) AND a subjective outcome (item 12)
  • In order to be considered Medium risk of bias, the study neither met the conditions for Low risk of bias nor the conditions for High risk of bias.

All risk-of-bias category assignments (as Low, Moderate, or High) were performed by the principal investigator and a second review team member independently, with disagreements resolved by consensus.

Data Synthesis

For each Key Question, we determined the specific treatment comparisons that were made by the included studies. A study with more than two groups would contribute to more than two comparisons and possibly more than one Key Question. We considered each treatment comparison separately. When choosing among multiple comparisons within a study to be entered into an overall analysis, we prioritized the more common procedures (e.g., Lichtenstein, TEP).

Within each treatment comparison, we examined all of the included outcomes from all of the relevant studies. The outcomes were divided into eight categories: RC, hospital-related, including the length of hospital stay and subsequent hospital/office visits, the time to RTDA, the time to return to work (RTW), QOL, patient SFN, pain including visual analog scale (VAS) scores and the rates of chronic pain (PAIN), and other adverse events not involving pain (ADV). QOL was measured using the Short Form (SF)-36 health survey by most of the studies included in this report. The Short Form 36 (SF-36) quality of life instrument covers eight health concepts and is a measure of health status. The SF-36 is comprised of eight scaled scores and a single item score that provides an indication of perceived change in health. Other outcomes such as RTDA, RTW, and long-term pain were reported separately from QOL by these studies.

Within each category, the data were reported in different ways. For example, some studies reported the hazard ratio for hernia recurrence (with its 95 percent confidence interval [CI]) across the entire followup period, others reported the two groups' recurrence rates at a specific time point (e.g., x percent and y percent recurrence rates at 1 year after surgery), and other reported the two groups' recurrence rates at median followup (e.g., x percent and y percent recurrence rates with a median followup of 17 months and a range of followup from 8 months to 35 months). Within each category, we judged which studies could be combined based on the specific outcomes and methods of reporting.

Regarding time points, we used three categories: short-term (defined as ≤1 month after surgery), intermediate-term (defined as between 1 month and 6 months after surgery), and long-term (defined as ≥6 months after surgery). When a study reported multiple time points of the same outcome within the category, and we had to decide which time point to include in a meta-analysis with other studies, we chose the latest time point within that category.

We performed meta-analysis wherever appropriate and possible. This decision depended on the judged clinical homogeneity of the different study populations, co-interventions, and outcomes, as well as what is reported by those studies. For some outcomes (length of stay, RTDA, return to work, and pain (measured in VAS) score), many studies did not report SDs or other measures of dispersion that could be used to calculate SDs. To enable inclusion of these studies in meta-analysis we estimated the SDs by pooling the SDs of studies that did report them for these outcomes. Forest plots for all meta-analyses appear in the Figures section.

In the choice of effect size metrics, for hernia recurrence we used the relative risk (RR), because of its ease of interpretation and also because some studies only reported an adjusted RR, thus only an RR meta-analysis could include all of the studies. For all continuous outcomes, we used the weighted mean difference, which is on the same scale as the measured outcome. For adverse events and pain reported dichotomously, we analyzed ORs.

To aid interpretation, for each outcome in the review, we estimated the smallest difference between groups that could still be considered clinically significant (minimum clinically significant difference or MCSD). This definition aids interpretation in two main ways: (1) to determine whether a statistically significant difference is important and (2) to determine whether a statistically nonsignificant difference is small enough to exclude the possibility of an important difference. Our estimates were based on published literature, FDA guidance, or the consensus of the research team and TEP.

After hernia repair, a key outcome is RC. For this outcome, we define the MCSD as 3 percentage points (e.g., 1 percent vs. 4 percent for two separate treatments). This was based on statements in two multicenter trials (the U.S. Department of Veterans Affairs trial36 and the Medical Research Council trial)37 that such a difference is clinically meaningful. For other anticipated outcomes, we used the following approaches concerning the definitions of minimum clinical significance:

  • Length of hospital stay, RTDA, RTW: 1-day difference between groups. For RTDA and RTW, this had been defined as 1 week in the review protocol, but the review team decided to change it to 1 day upon finding that the typical RTDA after inguinal hernia surgery is about 10 days and the typical RTW is about 14 days. On that scale, a week is clearly too large to be considered the “minimum” clinically significant difference, so we changed it to 1 day. This was outlined in a review protocol amendment dated August 9, 2011.
  • Number of hospital visits/number of office visits: 20 percent difference between groups (e.g., means of 5 visits and 4 visits).
  • Quality of life: 5 percent of the range of the scale (e.g., 5 points on the SF-36, which ranges from 0–100).
  • Patient SFN: A one-level change (this outcome is typically measured on an ordinal scale representing various levels of SFN)
  • Pain: If reported as a continuous measure, 20 percent of the range of the scale (e.g., two points on the VAS, which typically ranges from 0–10).38 If reported as a dichotomous measure, we defined the MCSD as an OR of 1.25. This means that if the confidence limits of the OR were fully within the range of 0.80 to 1.25, a conclusion of equivalence may be appropriate. The U. S. Food and Drug Administration uses this same range when setting criteria for concluding bioequivalence.39
  • Other adverse events: We defined the MCSD in the same way that we did for dichotomous pain.

If meta-analysis was deemed appropriate and possible for a given comparison and a given outcome, we performed DerSimonian and Laird random-effects meta-analysis40 using Comprehensive Meta-Analysis software (Biostat Inc., Englewood, NJ). Meta-analyses for a given Key Question were performed by the team member responsible for that Key Question. To measure heterogeneity, we used both I2 and tau. Both are used because I2 can increase simply by increasing the numbers of patients in the studies (whereas tau is a more direct measure of heterogeneity),41 but tau is more difficult to interpret because its scale is different for different effect sizes. We defined substantial heterogeneity as a value of tau greater than the MCSD for that outcome. If this occurred, and there were 10 or more studies of the same patient outcome of the same treatment comparison, we conducted meta-regressions using the permutation test42 in Stata software (Stata Corp., College Station, TX). Where possible, we investigated up to 10 covariates in these meta-regressions (percentage of patients with bilateral hernia, percentage of patients with recurrent hernia, mean age, percent of laparoscopic patients undergoing TEP, percentage of open patients undergoing Lichtenstein, percentage of centers in university or specialist settings, concealment of allocation, similar levels of prior surgical experience, outcome rater blinding, and length of followup).

Strength of Evidence Rating

We used the system described in the Effective Health Care (EHC) Methods Guide10 to rate the strength of the evidence (SOE) for the major outcomes for each Key Question. SOE is defined as one's confidence in the evidence supporting a conclusion. It includes four core domains (risk of bias, consistency, precision, and directness) as well as four optional domains (large magnitude of effect, all plausible confounders would reduce the effect, publication bias, and dose-response association). In the EHC grading methodology, the directness domain does not encompass applicability, which is considered outside of the evidence rating system (we discuss our applicability methods in the next section). The various domains were considered together to rate the evidence for the outcome as High, Moderate, Low, or Insufficient. The rating was done by two independent analysts, and disagreements were resolved by consensus. Owens and colleagues, 2009,10 defined the four ratings as:

  • High—“High confidence that the evidence reflects the true effect. Further research is very unlikely to change our confidence in the estimate of effect.”
  • Moderate—“Moderate confidence that the evidence reflects the true effect. Further research may change our confidence in the estimate of effect and may change the estimate.”
  • Low—“Low confidence that the evidence reflects the true effect. Further research is likely to change the confidence in the estimate of effect and is likely to change the estimate.”
  • Insufficient—“Evidence either is unavailable or does not permit a conclusion.”

If there were no studies for a given treatment comparison or Key Question, we rated the evidence as Insufficient.

The SOE system requires one to choose the major outcomes that will receive an SOE rating. We carefully considered each Key Question and chose the following outcomes to receive SOE ratings:

  • Key Question 1 (surgery vs. WW): Quality of life, long-term pain, hernia strangulation or incarceration
  • Key Question 2 (open vs. laparoscopic surgery): RC, length of hospital stay, RTDA, return to work, QOL, patient SFN, long-term pain, epigastric vessel injury, small bowel injury, small bowel obstruction, urinary retention, hematoma, wound infection
  • Key Question 3 (comparing different types of open surgery): RC, length of hospital stay, RTDA, return to work, short-term pain, intermediate-term pain, seroma, urinary retention, hematoma, wound infection
  • Key Question 4 (comparing different types of laparoscopic surgery): RC, length of hospital stay, RTDA, return to work, short-term pain, intermediate-term pain, urinary retention, hematoma, wound infection
  • Key Question 5 (comparing different meshes): RC, QOL, patient SFN, long-term pain, feeling of a foreign body, infection, bleeding
  • Key Question 6 (comparing different mesh fixation approaches): Same as for Key Question 5
  • Key Question 7 (association between surgical experience and recurrence): SOE was not rated because no rating system exists for evidence on association
  • Key Question 8 (comparing different types of open surgery): We did not rate SOE because no studies met inclusion criteria
  • Key Question 9 (pediatric open vs. laparoscopic surgery): RC, length of hospital stay, RTDA, patient/parent SFN

We used the following approach to combine the SOE components and determine a rating (High, Moderate, Low, or Insufficient) for a given outcome of a given treatment comparison. We determined whether the combined evidence on that outcome was sufficient to permit a conclusion about the direction of the effect (either favors treatment A, favors treatment B, or indicates approximate equivalence by ruling out the MCSD). The third possibility was considered when the evidence was sufficiently precise to rule out the possibility of a clinically important difference.

If the evidence did not permit a conclusion about the direction of the effect, then the rating was Insufficient (abbreviated INSUFF in our SOE tables). If it was sufficient, then we assigned point values to the four core domains as follows: Risk of bias +2/+1/0 for low/moderate/high; Consistency +1/0/0 for consistent/inconsistent/unknown; Directness +1/0 for direct/indirect; and Precision +1/0 for precise/imprecise. For the additional domains, we sometimes added 1 for a large magnitude of effect, and we sometimes subtracted 1 for potential publication bias or selective outcome reporting (e.g., if a third or fewer of the studies included for that comparison had actually reported that outcome). The other two additional domains (all plausible confounders would reduce the effect, and dose-response association), were not relevant to any of our Key Questions. We added the points for the various domains, and 5+ indicated an SOE rating of High; 4 points indicated an SOE rating of Moderate; 3 points indicated an SOE rating of Low; and 2 or fewer points indicated an SOE rating of Insufficient.

All SOE category assignments (High, Moderate, Low, Insufficient) were performed by the principal investigator independently from the team member(s) responsible for that Key Question, with disagreements resolved by consensus.

Applicability Assessment

For this evidence report, we assessed the applicability of evidence for each Key Question. As defined in the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care Program Methods Guide for Comparative Effectiveness Reviews of Medical Interventions, applicability is “the extent to which the effects observed in published studies are likely to reflect the expected results when a specific intervention is applied to the population of interest under “real-world” conditions.”43 Applicability depends on context and cannot be assessed with a universal rating system.43 Thus far, no system has been developed for rating the applicability of a body of evidence for inguinal hernia repair.

Assessment of the applicability of a body of evidence is a complex task and involves addressing a series of methodological questions. These questions include:

  • What are the population of interest and the “real world” conditions relevant to the stakeholders of this evidence report? From whose perspectives should the applicability of the evidence be evaluated? This evidence review potentially serves multiple stakeholders, such as policymakers, clinicians, and patients and families. Different stakeholders may have different populations of interest and different applicability issues for consideration.
  • What factors may affect the applicability of a study? What factors need to be considered in the assessment of applicability? While the PICOS (i.e., population, intervention, comparator, outcome, and setting) approach may be used to identify these factors,43 some of the factors may have already been considered, at least in part, in the study inclusion/exclusion process.
  • How would the impact of each of these factors be judged or graded? The answer to this question is not always straightforward. For example, it is difficult to judge the exact degree by which the findings of a study that only included patients of 55 years of age or older apply to the younger population. The judgment is often made on a subjective basis.
  • How would the impacts of these various factors be synthesized to reach a general conclusion about the applicability of an individual study? Studies included in evidence reviews may report different applicability-related data (e.g., different types of comorbidities) or report the same types of data (e.g., duration of hernia) in different ways (e.g., reported as longer or less than 6 weeks vs. in average years). No validated instrument is currently available for accommodating these differences to reach a general conclusion about the applicability of a study.
  • When the evidence consists of multiple studies, how would the applicability of different studies be synthesized to reach a general conclusion about the applicability of the evidence? We did not identify any validated instrument for this type of synthesis.

Given these unresolved methodological issues, we chose a practical approach to assessing the applicability of evidence for this evidence review. The goal of our assessment is to provide useful information to concerned stakeholders in making judgment on whether the evidence is applicable to the population or conditions of their interest.

We first abstracted data from each included study on factors that may affect the applicability of the study. We primarily focused on factors in three areas that are most relevant to the inguinal hernia repair topic:

  • Population: demographic characteristics (e.g., age, sex, race, and ethnicity), comorbidity or general physical fitness (e.g., chronic cough, cardiovascular conditions, pulmonary functions, body mass index (BMI), activity assessment scale, and physical component summary), and types of hernia (e.g., primary vs. recurrent, unilateral vs. bilateral, reducible vs. irreducible, and hernia duration)
  • Intervention and comparators: inguinal repair procedure being compared, periods of the procedure being performed, co-interventions (e.g., type of anesthesia and perioperative use of antibiotics), and experience of the surgical team
  • Setting: geographic (e.g., the United States, Canada, or European countries) and clinical (e.g., academic medical centers vs. community hospitals) settings

Based on a review of the data abstracted, we narratively summarized any patterns reflected from these factors that could potentially affect the applicability of the evidence. We made no attempt to generate any rating or score for the applicability of the evidence, due to the methodological issues discussed. Our narrative summaries were intended to raise stakeholders' attention to potential applicability issues embedded in the evidence. All applicability sections (applicability was not rated on a scale based on the applicability guidance chapter) were written by a clinical team member.

Peer Review and Public Commentary

As part of a newly instituted process at AHRQ, the draft report was reviewed before peer review by the Task Order Officer (TOO) and an AHRQ associate editor (a senior member of a sister EPC). The revised draft report was then sent to invited peer reviewers and was simultaneously uploaded to the AHRQ Web site where it was available for public comment for 28 days. All reviewer comments (both invited and from the public) will be collated and individually addressed. The EPC responses to all comments were documented in a disposition of comment document which will be posted on the Effective Health Care Web site about 3 months after Web publication of the evidence report. The authors of the report had final discretion as to how the report was revised based on the reviewer comments, with oversight by the TOO and Associate Editor.

Cover of Surgical Options for Inguinal Hernia: Comparative Effectiveness Review
Surgical Options for Inguinal Hernia: Comparative Effectiveness Review [Internet].
Comparative Effectiveness Reviews, No. 70.
Treadwell J, Tipton K, Oyesanmi O, et al.

AHRQ (US Agency for Healthcare Research and Quality)

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...