In this post I will write about how the evidence for how well medicines work, and the risk of side effects are not always what they seem. I’m going to show that relative risk reduction (or relative risk increase) always looks a lot bigger than absolute risk reduction or absolute risk increase. Journals and advertisements always report relative risk reduction for medicines or other treatments because they look more impressive. On the other hand, side effects are almost always reported as absolute risk increase because that looks a lot smaller. I will show you how to calculate both kinds of risk reduction and increase. I will argue that absolute risk reduction or increase is what you really want to know before deciding to take a medicine or other treatment.
Any study of a medicine or treatment always has a group that gets the treatment and a group that gets a placebo. That is the only way to know if the medicine or treatment really works. There is always a placebo effect for any medicine. That is, a certain portion of the people who get a placebo get better. If the number of people who get better from the actual treatment is higher than the number of people who get better from the placebo, then the treatment works.
People who participate in a study know that they might get the treatment or a placebo, but they don’t know which one they got until the end of the study. In medical terminology, they are blinded from knowing whether they got treatment or placebo. If the study is double blind (the most reliable kind) then neither the investigators who administer the treatment nor the participants know which study participants got the study treatment or the placebo until the end of the study.
If the treatment works better than the placebo, that result can be reported in several different ways
Relative Risk Reduction or Increase
Relative risk is the proportion of people who have the disease or condition being studied in the treatment group divided by the proportion of people who have the condition in the placebo group.
For example, let’s say we are testing a treatment to prevent diabetes. The control group and the treatment group each have 100 people. 30 people get diabetes in the control group and only 10 people get diabetes in the treatment group. The risk in the treatment group is 10/100 = 0.1 The risk in the placebo group is 30/100 = 0.3 To compare those risks we divide the risk in the treatment group by the risk in the control group. 0.1/0.3= .33. That means that the relative risk of diabetes in the treatment group is 1/3 of the risk in the placebo group. To change that to the relative risk reduction percentage we use the formula 100(1- Relative Risk). Plugging our values into that formula gives 100(1-.33)= 67%. In other words, the treatment reduces the risk of diabetes by 67% relative to the risk of diabetes in the placebo group. That sounds like a big treatment effect!
All medicines or treatments have some side effects. Placebos can have side effects too, especially if people are told (as they must be) what are the possible side effects of the treatment. Placebo side effects are called “nocebo” effects.
In the hypothetical diabetes study described above, let’s say that a side effect of the treatment is bladder infection. Let’s suppose that 5 out of the 100 people in the treatment group get a bladder infection while only 1 out of 100 in the control group get a bladder infection. The relative risk of a bladder infection in the placebo group is 1/100=0.01. The risk of bladder infection in the treatment group is 5/100=.05. The relative risk of getting a bladder infection in the placebo group compared to the treatment group is .01/.05=0.2. To change that to a relative percentage increase, we use our formula again. 100(1-.2)=80%, This means that the risk of the side effect of bladder infection is 80% more likely in the treatment group compared to the risk of bladder infection in the placebo group.
Absolute Risk Reduction or Increase
Relative risk reduction or increase does not take into account the baseline risk of getting the disease or condition. Absolute risk reduction does take into account the baseline risk.
In our example above the absolute risk reduction is 30% (risk of diabetes in the control group) minus 10% (risk of getting diabetes in the treatment group) = 20%. That means that treatment reduces risk of diabetes by 20%. Notice that this is a much smaller number than the 67% relative risk reduction, but it more accurately reflects how much the treatment would reduce your risk of diabetes.
In our example the absolute risk increase of getting a bladder infection with treatment is 5% (the risk of bladder infection in the treatment group) minus 1% (the risk of getting bladder infection in the control group) = 4%. That means that the risk of getting a bladder infection from the treatment is 4% more than no treatment. Again, a much smaller number than relative risk increase of 80%.
Number Needed to Treat (or Harm)
Another way to look at how well a medicine or treatment works compared to placebo is the number of people that need to be treated in order to help one person. This is called Number Needed to Treat, abbreviated as NNT. The NNT = 1 divided by the absolute risk reduction. In our example the absolute risk reduction of getting diabetes in the treatment group was 20%. 1 divided by 0.20 = 5. That means you would need to treat 5 people to prevent 1 case of diabetes with this hypothetical treatment.
Number Needed to Harm, abbreviated as NNH is 1 divided by absolute risk increase. In our hypothetical example NNH = 1 divided by .04 = 25. This means that you would need to treat 25 people for one person to get a bladder infection
A Real World Example: Fosamax to prevent hip fracture in women with osteoporosis
In the real world, we rarely see a treatment effect as big as in our hypothetical example. Let’s look at a real study on a real medicine. Here are some numbers from a big four year study on using Fosamax (generic name alendronate) to prevent hip fracture in women with osteoporosis (thinning of bones). This study was reported in the Journal of the American Medical Association (JAMA) in 1988. It was the first large study to show that alendronate reduced hip fractures in women with osteoporosis. The study included over 4000 women with osteoporosis (shown by a type of bone scan called a DEXA scan). There were 2,218 women in the placebo group and 2,214 women in the treatment group. In the placebo group 812 women (36.6%) had severe bone thinning in the hips by Dexa scan. In the treatment group 819 (37%) women had severe bone thinning in the hips by Dexa scan. Over the four years there were 18 hip fractures in the placebo group (2.2%) and there were 8 hip fractures in the treatment group (1%)
Now lets do our calculations:
Relative Risk = .01 divided by .022 = .45. Relative Risk Reduction = 100(1-.45)= 55%. . This large relative risk reduction is what the article describing the study reported.
Absolute Risk Reduction = 2.2% – 1% = 1.2%. As you can see, the absolute risk reduction is tiny. Taking alendronate for 4 years reduced hip fracture by only about 1%.
Number Needed to Treat = 1/1.2% = 83.3. That means you would need to treat 83 women for four years to prevent one hip fracture.
Side Effects
Muscle or bone pain
In other studies, muscle or bone pain, sometimes severe was reported by 4% in the treatment group and 2.5% in the placebo group.
Relative Risk = .025 divided by .04 = .625. Relative Risk Increase = 100(1-.625)=37.5%. That means that bone and muscle pain are 37.5% more likely in the treatment group relative to the risk in the placebo group.
Absolute Risk Increase = 4% – 2.5% = 1.5%. This means that there is only a 1.5% increase in risk of muscle or bone pain when taking alendronate.
Number Needed to Harm = 1 divided by 1.5% = 67. That means you would need to treat 67 people for one person to have muscle or joint pain. Notice that the number needed to harm is less than the number needed to treat to prevent 1 hip fracture!
Osteonecrosis of the jaw
This is a rare but very serious side effect of alendronate. Assuming treatment group effect of .01% versus placebo of essentially zero let’s do our calculations.
Relative Risk Increase: since this complication is so rare, there are no trials comparing it with placebo.
Absolute Risk Increase: .01 %.
Number Needed to Harm: 1 divided by .01 % = 1000. This means that 1000 people would be treated before you would see one person with this serious complication.
Bottom Line
Journal articles and advertisements almost always report relative risk reduction for medicines or treatments because it makes the effect of the medicine or treatment look bigger. Side effects are almost always reported as absolute risk increases because it makes them look smaller.
Absolute Risk Reduction or Increase is what you really want to know when you are considering taking a medicine or treatment. Most of the time you will not be able to calculate absolute risk reduction, because you won’t have the actual percentages from the placebo and the treatment groups. Don’t despair though. Dr. Google is there to help. Using the following Google search will usually give you the absolute risk reduction. Type in “absolute risk reduction for (name of medicine or treatment).
Once you have the Absolute Risk Reduction (ARR) or Absolute Risk Increase (ARI)you can calculate for yourself the Number Needed to Treat or the Number Needed to Harm. Just divide 1 by either ARR or ARI.