• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Evans I, Thornton H, Chalmers I, et al. Testing Treatments: Better Research for Better Healthcare. 2nd edition. London: Pinter & Martin; 2011.

Cover of Testing Treatments

Testing Treatments: Better Research for Better Healthcare. 2nd edition.

Show details

Chapter 2Hoped-for effects that don’t materialize

Some treatments are in use for a long time before it is realized that they can do more harm than good. Hoped-for effects may fail to materialize. In this chapter we explain how this may come about.


Do not imagine that only drugs can harm – advice can be lethal too. Many people have heard of the American childcare specialist Dr Benjamin Spock, whose best-selling book Baby and Child Care became a bible for both professionals and parents, especially in the USA and the UK, over several decades. Yet in giving one of his pieces of well-meaning advice Dr Spock got things badly wrong. With seemingly irrefutable logic – and certainly a degree of authority – from the 1956 edition of his book until the late 1970s he argued: ‘There are two disadvantages to a baby’s sleeping on his back. If he vomits he’s more likely to choke on the vomitus. Also he tends to keep his head turned towards the same side . . . this may flatten the side of the head . . . I think it is preferable to accustom a baby to sleeping on his stomach from the start.’

Placing babies to sleep on their front (prone) became standard practice in hospitals and was dutifully followed at home by millions of parents. But we now know that this practice – which was never rigorously evaluated – led to tens of thousands of avoidable cot deaths.1 Although not all cot deaths can be blamed on this unfortunate advice, there was a dramatic decline in these deaths when the practice was abandoned and advice to put babies to sleep on their backs was promoted. When clear evidence of the harmful effects of the prone sleeping position emerged in the 1980s, doctors and the media started to warn of the dangers, and the numbers of cot deaths began to fall dramatically. The message was later reinforced by concerted ‘back to sleep’ campaigns to remove once and for all the negative influence of Dr Spock’s regrettable advice.

How advice on babies’ sleeping position changed with time.

How advice on babies’ sleeping position changed with time


Dr Spock’s advice may have seemed logical, but it was based on untested theory. Other examples of the dangers of doing this are not hard to find. After having a heart attack, some people develop heart rhythm abnormalities – arrhythmias. Those who do are at higher risk of death than those who don’t. Since there are drugs that suppress these arrhythmias, it seemed logical to suppose that these drugs would also reduce the risk of dying after a heart attack. In fact, the drugs had exactly the opposite effect. The drugs had been tested in clinical trials, but only to see whether they reduced heart rhythm abnormalities. When the accumulated evidence from trials was first reviewed systematically in 1983, there was no evidence that these drugs reduced death rates.2

However, the drugs continued to be used – and continued to kill people – for nearly a decade. At the peak of their use in the late 1980s, one estimate is that they caused tens of thousands of premature deaths every year in the USA alone. They were killing more Americans every year than had been killed in action during the whole of the Vietnam war.3 It later emerged that, for commercial reasons, the results of some trials suggesting that the drugs were lethal had never been reported (See Chapter 8, p97).4


At one time, doctors were uncertain whether pregnant women who had previously had miscarriages and stillbirths could be helped by a synthetic (non-natural) oestrogen called diethylstilboestrol (DES). Some doctors prescribed it and some did not. DES became popular in the early 1950s and was thought to improve a malfunction of the placenta that was believed to cause these problems. Those who used it were encouraged by anecdotal reports of women with previous miscarriages and stillbirths who, after DES treatment, had had a surviving child.

For example, one British obstetrician, consulted by a woman who had had two stillborn babies, prescribed the drug from early pregnancy onwards. The pregnancy ended with the birth of a liveborn baby. Reasoning that the woman’s ‘natural’ capacity for successful childbearing may have improved over this time, the obstetrician withheld DES during the woman’s fourth pregnancy; the baby died in the womb from ‘placental insufficiency’. So, during the woman’s fifth and sixth pregnancies, the obstetrician and the woman were in no doubt that DES should again be given, and the pregnancies both ended with liveborn babies. Both the obstetrician and the woman concluded that DES was a useful drug. Unfortunately, this conclusion based on anecdote was never shown to be correct in fair tests. Over the same period of time that the woman was receiving care, unbiased studies were actually being conducted and reported and they found no evidence that DES was beneficial.5

Although there was no evidence from fair tests that DES was helpful in preventing stillbirths, the DES story did not end there. Twenty years later evidence of harmful side-effects began to emerge when the mother of a young woman with a rare cancer of the vagina made a very important observation. The mother had been prescribed DES during pregnancy and she suggested that her daughter’s cancer might have been caused by the drug.6 This time the observation was correct, but most importantly it was shown to be correct. Since then, numerous studies have shown a range of serious side-effects of DES in both men and women who had been exposed to DES before they were born. These side-effects included not only an increased frequency of rare cancers but also other abnormalities of the reproductive system.

By the time it was officially declared that DES should not be used in pregnancy, several million people had been exposed to the drug. Knowing what we know now, if doctors had used the most reliable research evidence on DES available in the 1950s, many fewer would have prescribed it, because DES was never actually proved to be effective for the condition for which it had been prescribed in the first place. Tragically, this lack of evidence of benefit was widely overlooked.7


In women going through the menopause, hormone replacement therapy (HRT) is very effective in reducing the distressing hot flushes that are commonly experienced, and there is some evidence that it may help to prevent osteoporosis (bone thinning). Gradually, more and more beneficial effects were claimed for HRT, including prevention of heart attacks and stroke. And millions of women, advised by their doctors, began using HRT for longer because of claims of these and other extra benefits. However, the basis of these claims was very shaky.


In January 2004, a hysterectomy patient wrote this letter to The Lancet:

‘In 1986 I had a hysterectomy because of fibroids. The surgeon also removed my ovaries and found that I had endometriosis as well. Because I was then only 45 years old and would have had an immediate menopause, I was put onto hormone replacement therapy (HRT). The first year I took conjugated oestrogens (Premarin), but from 1988 until 2001 I had oestrogen implants every 6 months, given to me privately by the surgeon who did the operation. I was always a little dubious about having the treatment, since I felt I just did not have control over things once the implant was done, and also after several years had many headaches. Apart from that I felt very fit.

However, my surgeon assured me that HRT had so many advantages and that it suited me, which I agreed with. As time went on, HRT was reported to have more and more benefits and was not just the cosmetic drug it seemed to have been used for in its early years. It was now good for the heart, osteoporosis, and part defence against strokes. Every time I visited my surgeon, he seemed to have more evidence about the advantages of taking HRT.

My surgeon retired in 2001 and I went to my National Health Service doctor. What a shock! He told me the exact opposite of my private surgeon – that it would be a good idea to come off HRT: it could increase the risk of heart disease, strokes, and breast cancer, and be the cause of headaches. I did have one more implant and then went onto Premarin for a short while, but since then I have not used HRT for about 8 months. My doctor said it would be my decision whether to stay on it or not. I was so confused . . .

I cannot understand how HRT and all its wonderful advantages can be reversed in such a short space of time. How can a layman like myself come to a clear decision? I have spent many hours discussing and thinking about whether I should have stayed on HRT, although so far I have not suffered many ill effects. I am very confused about the whole issue and I am sure other women feel the same.’

Huntingford CA. Confusion over benefits of hormone replacement therapy. Lancet 2004;363:332.

Take heart attacks alone. For over 20 years, women were told that HRT would reduce their risk of this serious condition – in fact the advice was based on the results of biased (unfair) studies (see Chapter 1 and Chapter 6). Then, in 1997, there was a warning that the advice might be wrong: researchers from Finland and the UK reviewed, systematically, the results of well-conducted studies.8 They found that, far from reducing heart disease, HRT might actually increase it. Some prominent commentators dismissed this conclusion, but its tentative result has now been confirmed by two large well-conducted trials. Had the effects of HRT been assessed properly when it was first introduced, women would not have been misinformed and many of them would not have died prematurely. To make matters worse, we now know that HRT increases the risk of stroke and of developing breast cancer.9

Overall, HRT continues to be a valuable treatment for women with menopausal symptoms.10 However, it is tragic that it was so heavily promoted specifically as a way of reducing heart attacks and stroke. Although the increased chance of these serious conditions is modest, the total number of women affected is very large indeed because HRT has been so widely prescribed.


Even if inadequately assessed treatments do not kill or harm, they can waste money. Eczema is a distressing skin complaint affecting both children and adults. The skin lesions are both unsightly and very itchy. Although the use of steroid creams helps in this condition, there were concerns about the side-effects of these treatments, such as thinning of the skin. In the early 1980s a natural plant oil extract – evening primrose oil – emerged as a possible alternative with few side-effects.11 Evening primrose oil contains an essential fatty acid called gamma-linolenic acid (GLA) and there were plausible reasons for using it. One suggestion, for example, was that the way in which GLA was transformed within the body (metabolized) was impaired in patients with eczema. So, theoretically, giving GLA supplements should help. Borage oil, also known as starflower oil, contains even higher amounts of GLA and this was also recommended for eczema.

Timeline for evidence about and use of evening primrose oil in eczema.

Timeline for evidence about and use of evening primrose oil in eczema

GLA was believed to be safe but was it effective? Numerous studies were done to find out but they gave conflicting results. And the published evidence was heavily influenced by studies sponsored by the companies making the supplements. In 1995, the Department of Health in the UK requested researchers unconnected with the manufacturers of evening primrose oil to review 20 published and unpublished studies. No evidence of benefit was found. The Department never made the report public because the manufacturers of the drug objected. But five years later another systematic review of both evening primrose oil and borage oil by the same researchers – this time it was published – showed that in the largest and most complete studies there was no convincing evidence that these treatments worked.12

There was one unturned stone – perhaps GLA only worked in very high doses. In 2003, even this claim was knocked on the head by a carefully conducted fair test.13 Ironically, by the time these results were published, the UK Medicines Control Agency (MCA, which subsequently became the Medicines and Healthcare products Regulatory Agency, MHRA) had finally, in October 2002, withdrawn the product licences for two major evening primrose oil preparations because there was no evidence that they worked.

Nevertheless, since no concerns were expressed about the safety of evening primrose oil, it is still widely available over the counter as a ‘dietary supplement’ for various conditions. Regarding its use for eczema, claims of effectiveness are couched in vague terms such as ‘people with eczema may find relief ’, ‘may be helpful’ and ‘has certain medicinal properties that may act as an anti-inflammatory for conditions such as eczema’.


  • Neither theory nor professional opinion is a reliable guide to safe, effective treatments
  • Just because a treatment is ‘established’ does not mean it does more good than harm
  • Even if patients do not suffer from inadequately tested treatments, using them can waste individual and community resources
Copyright © 2011 Imogen Evans, Hazel Thornton, Iain Chalmers and Paul Glasziou.
Bookshelf ID: NBK66199
PubReader format: click here to try


Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...