Home > For Consumers > Can measurements show if a treatment...

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

Informed Health Online [Internet]. Cologne, Germany: Institute for Quality and Efficiency in Health Care (IQWiG); 2006-.

Informed Health Online [Internet].

Can measurements show if a treatment works?

Last Update: October 27, 2013; Next update: 2016.

In the 1980s, millions of people were treated with drugs intended to prevent sudden cardiac death. But it later turned out that the exact opposite happened: the drugs increased the death rate. What went wrong, and what can we learn from this experience?

Some people develop a specific form of irregular heartbeat (arrhythmia) after a heart attack. They have an increased risk of sudden cardiac death. To try to make this irregular heartbeat return to normal, researchers developed drugs called 1c antiarrhythmic agents in the 1970s. Clinical trials showed that these drugs did make heartbeats return to normal in electrocardiographs (ECGs, also sometimes called EKGs). Because of this supposedly positive effect, these antiarrhythmic agents were used a lot in the 1980s.

In the late 1980s, a group of researchers initiated a trial on class 1c antiarrhythmic agents, called the CAST trial. This trial not only looked at the effect the drugs had on people’s heartbeat, but also how they affected the death rate (mortality) from sudden cardiac death. The results were alarming: compared to the group who had taken a dummy drug (placebo), the rate of sudden cardiac death was twice as high in the group who had used an antiarrhythmic agent.

Plausible is not enough

Why had people been treated for many years with drugs that double the death rate? Because experts had drawn the wrong conclusions: Irregular heartbeat was known to increase the risk of sudden cardiac death. So they concluded that drugs against irregular heartbeat might be able to lower this risk. From a medical point of view, this conclusion seemed to be perfectly plausible. But it still turned out to be wrong.

The results of the CAST trial are now considered to be a prime example of why measurements alone cannot be relied upon. For a long time, the ECG measurements were considered to be a substitute measurement of the risk of dying. Criteria that are used in trials to substitute an important endpoint are also called surrogate endpoints or surrogate markers (from the Latin surrogatum, meaning substitute).

Endpoints that are important to patients – such as mortality, heart attacks, quality of life or the length of hospital stays – on the other hand, are called patient-relevant endpoints. The term “patient-relevant” reflects the fact that it concerns issues that are important to the people who have a medical condition – for example whether a treatment helps them live longer, spares them from going to the hospital, reduces their symptoms, prevents complications, or helps them cope better with their condition in daily life.

Table: Examples of surrogate endpoints and the corresponding patient-relevant endpoints:

Surrogate endpointPatient-relevant endpoint
High cholesterol levelsHeart attack
Low bone densityBroken bone
Irregular heartbeatSudden cardiac death
High blood pressureStroke, heart attack
Tumor does not respond to treatment Death, reduced quality of life

Correlation does not say anything about cause and effect

Another example of a misleading surrogate endpoint is bone density as an indicator for the risk of bone fractures in women after menopause.

In the 1980s, a trial was done to test whether sodium fluoride taken in addition to calcium can lower the risk of broken bones in women with low bone density (osteoporosis). Examining the bones revealed that the bone density of women who took sodium fluoride increased. Yet they still had more broken bones than women who had only taken a placebo in addition to calcium. The reason for this is that, although sodium fluoride increased bone density, it also changed the composition and quality of the bone tissue. This made the bones more brittle.

Particular diseases are often associated with abnormal laboratory data or measurements. Links like this can help understand a disease better – but do not allow us to draw any conclusions about cause-and-effect relationships. Because of this, it cannot be assumed that a treatment that improves a certain laboratory measurement will also reduce the risk of a related medical condition.

Most surrogate endpoints do not take into account the complex processes happening in the body. Sometimes particular values deviate from the norm in healthy people too. Or a treatment influences the surrogate endpoint, but not the patient-relevant endpoint. A treatment can also affect a surrogate endpoint without influencing the patient-relevant endpoint. So most surrogate endpoints are not reliable when considered on their own. They can be misleading when used to evaluate how well a treatment works.

Surrogate endpoints: tempting, but only rarely reliable

The reason why studies often only use surrogates, and not the endpoints that are important for patients, is that they are a lot easier to measure: for example, a study will quickly show whether a medication lowers blood pressure. But it can take years for researchers to find out whether this also prevents diseases like heart attacks.

Another reason is that studies on surrogate endpoints need far fewer participants. There are statistical reasons for this: Heart attacks, for example, are quite rare, so a large number of people must be monitored to see clear differences between the different treatment groups. In a trial on blood pressure, however, changes in blood pressure can be measured for each individual participant, which means that only few participants are needed to see if there is an effect.

Caution is needed when laboratory data and physical measurements are used as surrogates in trials to measure how much patients benefit from a treatment: Just because a drug lowers blood pressure or blood sugar, it will not necessarily protect against heart attacks or strokes. This has to be tested in trials that, for example, not only look at blood pressure levels, but also at the effect of the drug on cardiovascular diseases.

Using surrogate endpoints only makes sense if they really say something about the benefits of a treatment. This is the case if the effect of a treatment on the surrogate endpoint predicts how the treatment affects the patient-relevant endpoint. In rare cases, researchers might have to content themselves with correlations that are less conclusive. In those cases, however, they need to be aware of the associated uncertainty.

Laboratory data are important in medicine

Laboratory data and physical measurements are anything but useless in medicine, though. They are needed to make diagnoses, gauge or monitor the progress of a condition, or check whether a treatment is working or a dose is right. Someone with type 1 diabetes, for example, will regularly monitor blood sugar levels in order to adjust his or her insulin dose. Laboratory tests and ECGs are used to diagnose a heart attack.

Even though there are usually problems with using measurements as substitutes for patient-relevant endpoints, there are sometimes ethical reasons for using surrogate endpoints in studies. For instance, if there is a serious disease for which there is no effective treatment, it may make sense to introduce a new treatment even without knowing exactly what effect it will have.

This was the case with the first HIV drugs: Trials had shown that they could significantly reduce the number of human immunodeficiency viruses (HIV) in the body. But there were no trials showing that this leads to fewer people developing AIDS or dying as a result. Because there were no alternative treatments and HIV progresses rapidly if left untreated, the drug regulatory authorities approved these drugs anyway. We now know that this saved thousands of people with HIV from an early death.

Our information is based on trials using patient-relevant endpoints

The Institute for Quality and Efficiency in Health Care (IQWiG, Germany), has been given a legal mandate to assess the benefits and harms of medical interventions. To do this, IQWiG analyzes trials with patient-relevant endpoints or with reliable surrogates, if available, and generally looks at whether patients can actually benefit from a particular treatment. Such studies are also the basis for our articles on informedhealthonline.org.

That is why we only rarely talk about the effectiveness of a medical intervention, but rather about its benefits and/or harms. The fact that a treatment is effective (e.g. lowers blood sugar levels) does not automatically mean that patients will also benefit from it. There is also often not yet enough proof that an intervention has a benefit, or studies only suggest that it has a benefit.

Sources

  • Bucher HC. Studien mit Surrogatendpunkten – Nutzen und Grenzen in der klinischen Entscheidungsfindung. Internist 2008; 49: 681-687.
  • Bucher HC, Guyatt GH, Cook DJ, Holbrooke A, McAlister FA. Users’ guides to the medical literature XIX. Applying clinical trial results. A. How to use an article measuring the effect of an intervention on surrogate end points. JAMA 1999; 282: 771-778. [PubMed: 10463714]
  • Epstein AE, Hallstrom AP, Rogers WJ, Liebson PR, Seals AA, Anderson JL et al. Mortality following ventricular arrhythmia suppression by encainide, flecainide, and moricizine after myocardial infarction. The original design concept of the Cardiac Arrhythmia Suppression Trial (CAST). JAMA 1993; 270: 2451-2455. [PubMed: 8230622]
  • Fleming TR, DeMets DL. Surrogate end points in clinical trials: are we being misled? Ann Intern Med 1996; 125: 605-613. [PubMed: 8815760]
  • Grimes DA, Schulz KF. Surrogate end points in clinical research: hazardous to your health. Obstet Gynecol 2005; 105: 1114-1118. [PubMed: 15863552]
  • Institute for Quality and Efficiency in Health Care (IQWiG, Germany). General Methods: Version 4.0. September 23, 2011 [Accessed on: October 24, 2012].
  • Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG). Aussagekraft von Surrogatendpunkten in der Onkologie: Rapid Report; Auftrag A10-05. November 21, 2011 [Accessed on: October 11, 2012]. (IQWiG-Reports; Volume 80).
  • Mangiapane S, Velasco Garrido M. Surrogatendpunkte als Parameter der Nutzenbewertung. Köln: Deutsches Institut für Medizinische Dokumentation und Information (DIMDI); 2009. (Health Technology Assessment; Volume 91)
  • IQWiG health information is written with the aim of helping people understand the advantages and disadvantages of the main treatment options and health care services.

    Because IQWiG is a German institute, some of the information provided here is specific to the German health care system. The suitability of any of the described options in an individual case can be determined by talking to a doctor. We do not offer individual consultations.

    Our information is based on the results of good-quality studies. It is written by a team of health care professionals, scientists and editors, and reviewed by external experts. You can find a detailed description of how our health information is produced and updated in our methods.

© IQWiG (Institute for Quality and Efficiency in Health Care)

IQWiG (Institute for Quality and Efficiency in Health Care)

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...