LITERATURE FLOW

The literature flow diagram (Figure 1) summarizes the results of the study selection process (full list of excluded studies available in Appendix C).

Figure 1

Literature Flowchart.

LITERATURE OVERVIEW

We identified 1,207 potentially relevant citations (Figure 1). Including 2 publications recommended by experts, we applied the inclusion and exclusion criteria to these 1,209 titles. A total of 501 abstracts were reviewed at abstract stage. From these, a total of 136 abstracts were excluded for the following reasons: not acupuncture (N = 55), not a condition of interest (N = 47), not a systematic review (N = 14), background (N = 10), duplicate (N = 4), not outcome of interest (N = 3), and wrong population (N = 3). After reference mining the cited literature in our screened full-text articles, we identified an additional 5 titles to be reviewed at the full-text stage, resulting in a total of 370 publications. From these, 266 publications were excluded for the following reasons: did not use formal method for grading evidence (N = 239), review of reviews (N = 13), not a systematic review (N = 6), unavailable (N = 5), background (N = 1), no data (N = 1), and not acupuncture (N = 1). A full list of excluded reviews from the full-text review is included in Appendix C.

A total of 104 publications were retained for further review to potentially be included on the map. Of these, 41 reviews were excluded from the map for the following reasons: the review overlapped a more recent/larger review which was already included on the map (N = 33), duplicate (N = 3), review of reviews (N = 2), no outcome of interest (N = 1), not an intervention of interest (N = 1), and comparison (N = 1). See Appendix B for a full list of publications not included on the map because they overlapped with an included review. We included 63 publications in this map.

Characteristics of Included Reviews

The number of studies included for acupuncture in the included reviews ranged from 1 study to 73 studies. Twenty-eight reviews included fewer than 10 studies about acupuncture, 25reviews included 10 to 25 studies, and 10 reviews included 25 or more studies. Eighteen of the included reviews were completed by the Cochrane Collaboration, with 1 review published as a journal article in a peer-reviewed journal.⁹ The US Agency for Health Research and Quality conducted 3 of the included reviews, with 1 review published as a journal article in a peer-reviewed journal.¹⁰

The country of origin for reviews varied, with the highest number of reviews originating from China (N = 22). Other countries included Australia (N = 4), Brazil (N = 1), Italy (N = 1), Korea (N = 6), Taiwan (N = 1), United Kingdom (N = 2), and the United States (N = 7). Nineteen reviews involved teams from multiple countries; teams included reviewers from China and Norway,¹¹ China and Australia,¹² China (Hong Kong) and the United Kingdom,¹³ Spain and the United Kingdom,¹⁴ Canada and the United Kingdom,¹⁵ Germany, the United Kingdom, and the United States,¹⁶ and Korea and the United States.¹⁷

Forty-seven reviews included more than 1 type of acupuncture, while 16 reviews included only 1 type of acupuncture as the intervention. Almost all of the mapped reviews included manual or standard acupuncture as the intervention, with the exception of 1 review including only electro-acupuncture as the intervention for the reduction of prescribed opioid use in chronic non-cancer pain.¹⁸ A variety of comparators were included in the reviews, often involving more than 1 comparator. Thirty-six reviews included more than 1 comparator and conducted separate analyses of the effect of acupuncture by comparator, while 7 reviews that had included more than 1 comparator did not conduct separate analyses. Fourteen reviews employed active or usual care only, and 8 reviews included sham or placebo as comparator only.

The included 63 reviews were categorized into 41 conditions, of which 14 conditions were further categorized into sub-conditions: back pain (N = 4), cancer-related pain (N = 4), chronic fatigue syndrome (N = 2), depression (N = 4), fertility (N = 4), fibromyalgia (N = 4), headache (N = 5), insomnia (N = 2), mixed pain-not specific (N = 3), osteoarthritis (N = 2), other acute pain (N = 3), post-operative pain (N = 3), shoulder pain (N = 2), and substance use disorder (N = 2). These conditions and sub-conditions were then grouped by type of condition, resulting in 5 maps (Figure 2). Three reviews discussed multiple conditions and thus appeared in the maps more than once.^19–21 Most conditions were related to pain, which were separated into 2 maps: general pain (N = 23) and musculoskeletal pain (N = 11). The remaining conditions were categorized into maps for mental health (N = 12), women’s health (N = 9), and other conditions (N = 9). Appendix D shows the breakdown of conditions and related sub-conditions by map.

This map includes 9 conditions that were not part of the 2014 map (Table 1).

Table 1

Conditions in 2022 Evidence Map Not in 2014 Evidence Map.

Because we applied an additional criterion that reviews had to report a method used for grading certainty of evidence in order to be included in this map, a few conditions that had appeared in the 2014 map were not included in this map (Table 2). Four such conditions are plantar heel pain, nausea, restless leg syndrome, and blood pressure. For plantar heel pain, we identified 1 publication for inclusion but it did not use a formal method for grading certainty of evidence and was not included in the map. For nausea, we did not identify citations to be reviewed at full text. For restless leg syndrome, we identified 2 publications for inclusion but neither used a formal method for grading certainty of evidence and were not included in the map. For blood pressure, we identified 2 publications for full-text review. One review was excluded because it was not about acupuncture, and the other did not use a formal method for grading certainty of evidence and was not included in map.

Table 2

Selected Conditions in 2014 Evidence Map Not in 2022 Evidence Map.

For ease of comparison, we divided the included conditions into 5 evidence maps:

• All pain (other than musculoskeletal pain)^{9, 13, 15, 17–19, 22–38} (Figure 2A)
• Musculoskeletal pain^{10, 12, 14, 16, 20, 21, 39–43} (Figure 2B)
• Mental health conditions^44–54 (Figure 2C)
• Women’s health^55–63 (Figure 2D)
• Other conditions^{11, 64–71} (Figure 2E)

EVIDENCE MAPS

In each evidence map, columns correspond to whether a conclusion of the review was that 1) there was a benefit of acupuncture relative to a comparison treatment, or 2) there was no benefit of acupuncture relative to the comparison treatment. Columns are not mutually exclusive. A review could have more than 1 conclusion, and those conclusions could differ in the benefit of acupuncture.

Rows correspond to GRADE Working Group grades of evidence:⁷

• High certainty: We are very confident that the true effect lies close to that of the estimate of the effect.
• Moderate certainty: We are moderately confident in the effect estimate: the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different.
• Low certainty: Our confidence in the effect estimate is limited: the true effect may be substantially different from the estimate of the effect.
• Very low certainty: We have very little confidence in the effect estimate: the true effect is likely to be substantially different from the estimate of effect.

All rows are mutually exclusive. The top row indicates that at least 1 conclusion in the review was rated by its authors as having high (or strong) certainty of evidence (also sometimes called strength of evidence or quality of evidence). The middle row indicates that at least 1 conclusion was rated as moderate certainty of evidence (and none rated as high or strong, in which case it would be in the right-hand column). The bottom row indicates that all conclusions in the review were rated as low or very low certainty of evidence. Since GRADE assesses certainty of evidence, it is possible for a body of evidence to demonstrate low or moderate estimates of effect but with high certainty of evidence; conversely, it is possible to have evidence demonstrating high effectiveness but with low certainty.

Each conclusion (or general conclusion, see below) is then mapped onto this framework and identified by the name of the condition or sub-condition, eg, “pain management in cancer,” “fibromyalgia,” “migraine,” etc. Colors are used to distinguish between the types of comparison treatments: conclusions only about comparisons to sham/placebo, conclusions only about comparisons to active therapies or usual care, conclusions where the comparison treatments were a mix of these and no subgroup analysis was presented, and conclusions where comparison treatments were a mix of these with subgroup analyses. Symbols are used to identify the few reviews specific to certain types of acupuncture, namely reviews of electro-acupuncture only. We were only able to report to the degree of specificity the original authors report. When they called it manual acupuncture, we called it manual acupuncture. When they called it electro-acupuncture, we called it electro-acupuncture only. When they referred to it simply as acupuncture, we classified it as manual acupuncture, since the systematic reviews that included multiple types of acupuncture and specified the type for each included study had shown the great majority of included studies were about manual acupuncture.

The size of the bubble is used to indicate how many original research studies were included in the review. For example, in Figure 2A, the large yellow circle in the left-hand column indicates there is a review about fibromyalgia that included between 10-25 original studies and had at least 1 conclusion that was rated as high certainty of evidence that acupuncture was better than the comparison treatment of sham/placebo. In the same figure, the small light blue dot in the lower right-hand corner indicates there is a review about kidney stone pain that included fewer than 10 original studies where all conclusions were rated as low or very low certainty of evidence that acupuncture was not of greater benefit than the comparison treatment of active/usual care.

As noted above, reviews could contain more than 1 conclusion and enter the map at different spots. Also note that for ease of presentation we made the following decisions. If a review had 3 or fewer conclusions, we extracted and mapped them all. If a review had more than 3 conclusions (some reviews had >10 conclusions with, for example, separate statements for each kind of acupuncture assessed, each different comparison treatment, and each different assessed outcome), rather than attempt to map all of these, we instead mapped the overall conclusion the review authors gave to the overall body of evidence (usually found in the abstract or summary).

Figure 2

Condition Maps.

Three high-level observations are worth making. First, most published reviews were about painful conditions, and there are more mapped conclusions for painful conditions than for all other conditions combined. Second, the number of reviews with at least 1 conclusion rated as high certainty of evidence is very small (N = 3). Third, although a greater number of reviews have at least 1 conclusion rated as moderate certainty of evidence, the majority of reviews reported conclusions rated as low or very low certainty of evidence.

In addition to these maps, we collected all conclusions rated as high certainty of evidence in Table 3 and all conclusions rated as moderate certainty of evidence in Table 4. Lastly, all conclusions from mapped reviews are collected in a large appendix table (Appendix E).

The conclusions from the 3 systematic reviews graded as high certainty of evidence by the original review authors (Table 3) are:

• No difference between acupuncture and sham acupuncture in birth outcomes in patients undergoing embryo transfer (as part of in vitro fertilization).⁵⁷
• Better pain, sleep, and general status outcomes in patients with fibromyalgia syndrome treated with acupuncture compared to sham.⁷²
• Better pain relief in patients with shoulder pain treated with acupuncture compared to sham.²¹

There are many more conclusions authors of included systematic reviews graded as moderate certainty of evidence (see Table 4). More than 75% of these conclusions were comparing acupuncture to sham or control acupuncture, or no treatment. Only a small number of these conclusions were about comparisons of acupuncture to usual care or other active therapies. About 25% of the conclusions rated as moderate certainty were findings that acupuncture was no better than the comparator. A little more than half of the conclusions rated as moderate certainty were about painful conditions or pain outcomes.

All of the remaining conclusions from the remaining reviews were judged by the original authors as being low or very low certainty of evidence, meaning “Our confidence in the effect estimate is limited. The true effect may be substantially different from the estimate of effect” or “We have very little confidence in the effect estimate.” See Appendix E.

Table 3

Conclusions Rated as High Certainty of Evidence from Systematic Reviews Included in the Evidence Map.

Table 4

Conclusions Rated as Moderate Certainty of Evidence from Systematic Reviews Included in the Evidence Map.

Adverse Events

In addition to maps of effectiveness outcomes, we also created a map for adverse events. Most of the 63 included reviews assessed adverse events, with 16 reviews explicitly grading evidence for adverse events. Figure 3 presents these 16 reviews mapped by certainty of evidence conclusions about adverse events,^{8, 17, 19, 20, 31, 32, 35, 37, 45, 48, 50, 52, 56, 58, 62} of which 3 reviews appeared twice showing different certainty of evidence conclusions for different comparators.^{17, 20, 47}

Much like our approach for maps described above, only reviews with certainty of evidence conclusions specifically for adverse events were included in this map. The certainty of evidence conclusions were reviewed separately from conclusions for effectiveness outcomes such that it is possible to find a low or very low certainty of evidence conclusion for benefit of acupuncture and a high certainty of evidence conclusion for more adverse outcomes in the acupuncture group.⁵⁸

This map shows 3 categories depicted in rows: whether the certainty of evidence conclusion of the review for adverse events was low or very low, moderate, or high. As for columns, we listed whether there were fewer adverse events in the acupuncture group, no difference between groups, insufficient evidence to determine difference between groups, or more adverse events in the acupuncture group. A review could be mapped more than once for adverse events if different comparators had different certainty of evidence conclusions for adverse events. As we did for the effectiveness maps, we mapped each conclusion by name of condition or sub-condition. The legend for this map is the same, with colors denoting comparators, shapes denoting types of acupuncture, and size of bubble used to indicate the number of original research studies about acupuncture included in the review.

In the 16 mapped reviews that had included certainty of evidence conclusions about adverse events, a majority of the reviews reported either fewer adverse events in the acupuncture group (low or very low certainty of evidence) or no difference between groups (very low to high certainty of evidence). Only 2 reviews reported more adverse events in the acupuncture group. The first review about anovulatory infertility concluded that “true acupuncture probably worsens adverse events compared to sham acupuncture” (moderate certainty of evidence).⁵⁸ The second review about electro-acupuncture for carpal tunnel syndrome concluded there were more adverse events in the electro-acupuncture group (very low certainty of evidence).¹⁷ See Table 5 for additional details about included reviews with certainty of evidence conclusions for adverse events.

As part of our search, we also identified 3 reviews that were solely about adverse events, and not necessarily restricted to adult health conditions.^73–75 One review was an overview of existing systematic review (ie, a review of reviews) and included 17 existing reviews.⁷³ About half of these were based on case reports. The authors were unable to calculate incidence rates, though they did conclude that serious complications were “rare.” A second review was restricted to adverse events of auricular therapy, which included more kinds of therapy than just acupuncture (such as auricular bloodletting therapy).⁷⁴ The third systematic review collected 33 years’ worth of case reports of adverse events in China.⁷⁵ 182 cases were found, including 30 cases of pneumothorax, 37 cases of central nervous system injury, 22 cases of organ injury, 17 cases of infection, 10 cases of hemorrhage, 7 cases of broken needles, etc.

Although an incidence rate cannot be calculated from these estimates because there is no denominator, we can hazard an upper bound estimate if we assume that in any year no more than 1% of the Chinese population received acupuncture. Over 33 years, then, even if the number of case reports in this review is an underestimate of the true number by a factor of 10, or even a factor of 100, the incidence rate of serious adverse events is likely exceedingly small (potentially less than 1 in 100,000 patients).

Figure 3

Adverse Events.

Table 5

Certainty of Evidence Conclusions for Adverse Events in Reviews Included in Evidence Map.