Relative Risk

Understanding Cancer Screening Tests: Absolute vs Relative Risk Reduction

The benefit of cancer screening tests like pap smears, colonoscopy, mammography and others are reported in two ways. The most common way is the relative risk reduction. This is a ratio of the risk in the screened group divided by the risk in the non-screened group. Relative risk does not take into account the baseline risk in the whole population. The other way of reporting benefit of a screening test is called absolute risk reduction. Absolute risk reduction is the risk in the non screened group minus the risk in the screened group. Relative risk reduction always looks a lot bigger than absolute risk reduction because it does not take into account the baseline risk. Absolute risk reduction is what you really want to know. Absolute risk reduction lets you know how much your risk is reduced by taking the screening test. It is always a lot lower than the relative risk reduction. Absolute risk reduction of the most common cancer screening tests is very low, usually 1% or less.

Here are some examples:

  • Mammography: Relative breast cancer death risk reduction 30%; Absolute cancer death risk reduction 1%
  • Colonoscopy: Relative colon cancer death risk reduction 50%; Absolute death risk reduction 0.15%
  • Pap Smear: Relative cervical cancer death risk reduction 80%; Absolute cervical cancer risk reduction .08%
  • PSA (test for prostate cancer): Relative risk reduction 64%; Absolute risk reduction .09%

Another number that can be helpful is called Number Needed to Screen (NNS). NNS is the number of people who need to be screened to prevent 1 death from the disease. NNS is just 1 divided by the absolute death risk reduction for the screening test. Here are the NNS’s for the examples above.

  • Mammography: NNS 1/.01= 100 (this is mammograms every 2 years from age 50-75 so the the 100 patients means about 1100 mammograms).
  • Colonoscopy: NNS 1/.15=667
  • Pap Smear: NNS 1/.08=1,440
  • PSA: NNS 1/.09 =1,111

Sensitivity and Specificity

Any test, including cancer screening tests have a certain sensitivity and specificity.

Sensitivity

The sensitivity of a test is the probability that the test will detect the disease if it is present. In other words it measures how likely it is to get a false negative test. The higher the sensitivity, the less likely the test will be negative if the person has the disease being tested for. It is expressed as a percentage.

Specificity

The specificity of a test is the probability that a person with a positive test will have the disease. In other words it measures how likely it is to get a false positive test. The higher the specificity, the more likely a person with a positive test will have the disease. It is also expressed as a percentage

An ideal test has both a high sensitivity and specificity. Lets look at the sensitivity and specificity of our cancer screening tests.

  • Mammography: sensitivity 72%; specificity 98%
  • Colonoscopy: sensitivity 85%; specificity 90%
  • Pap Smear with HPV testing: sensitivity 95%; specificity 97%
  • PSA: Sensitivity 30%; Specificity 91%

Bayes Formula

All of these cancer screening tests have high specificity but somewhat less sensitivity except for Pap smears with HPV testing, which have high sensitivity and high specificity. So why are the absolute death reductions so low? Part of it has to do with something called Bayes Formula. It turns out that the chance of a false positive has to do not just with the specificity, but also the frequency of the disease in the population being screened. If the frequency of the disease in the population being screened is low, then even with a test that has high specificity, the chance of a positive test being a false positive is higher than than the specificity would suggest. The frequency of all of the above cancers is low in any 1 year in the population so that means that false positive cancer screening screening tests are common. Below are population frequencies for each cancer per year.

  • Breast cancer: annual prevalence in women 0.13%. Chance of a positive mammogram being a true positive: 28%. This means that a positive mammogram has 72% chance of being a false positive. On the other hand, a negative mammogram has an 8.7% chance of being a false negative, that is of missing a breast cancer
  • Colorectal cancer: annual prevalence in population .03%. Chance of a positive colonoscopy being a true positive: 2.5%. That means that a colonoscopy that finds something only has a 2.5% chance of being cancer. On the other hand, a negative colonoscopy has only a .005% chance of being a false negative. That means a negative colonoscopy has only a tiny chance of missing a cancer.
  • Cervical Cancer: annual prevalence in population 0.0077%. Chance of a positive pap smear being cancer: 0.24%. That means that 99.86% of positive pap smears with HPV testing will not be cervical cancer. On the other hand the chance that a negative pap smear with HPV testing will be a false negative is .00041%. Obviously a negative pap smear with HPV has an infinitesimally small chance of missing a cervical cancer. Although the chance of finding a cervical cancer is very low, the pap smear with HPV also finds precancerous changes in the cervix. Treatment of these precancerous cells prevents cervical cancer from developing. That is a big reason why the prevalence of cervical cancer is so low.
  • Prostate Cancer: annual prevalence in men .66%. Chance of a positive PSA (>4) being a true positive 2.18%. That means a PSA of >4.0 has a 98% chance that no prostate cancer is present. On the other hand a PSA of <4.0 has a 5.1% chance of missing a prostate cancer.

The somewhat lower sensitivity of mammography, colonoscopy and especially PSA means that false negatives are fairly common, for these tests.

The combination of false positives, false negatives and low prevalence of these cancers in the population all contribute to the small absolute death risk reduction for all four of these cancer screening tests. For patients at substantially higher risk, such as strong family history of breast or colon cancer, the screening tests perform much better, because the high risk population has a much greater disease prevalence than the general population.

Over Diagnosis

Another problem with cancer screening tests is over diagnosis. Over diagnosis means that a positive test finds a cancer, but the cancer grows so slowly or spontaneously disappears so that it never would have caused any symptoms in the person. Over diagnosis then leads to unnecessary treatment. So let’s look at the over diagnosis rate for our four cancer screening tests.

  • Mammography: Over diagnosis rate for women 40 and over is 12%. This means the 12% of women diagnosed with breast cancer by mammography will be treated for cancer unnecessarily.
  • Colonoscopy: The over diagnosis problem with colonoscopy results from the removal of polyps. All visible polyps are removed during colonoscopy. The polyps that have some chance of turning into cancer are called adenomatous polyps. Only 8% of these turn into invasive colon cancer over 10 years. That suggests that 92% of the adenomatous polyps removed at colonoscopy would never turn into cancer. Removal of all adenomatous polyps does prevent some colon cancers. It is not possible to know at the time of removal which polyps are going to progress. The cost of prevention of some colorectal cancers is substantial over diagnosis.
  • Pap Smear with HPV: Overdiagnosis of precancerous cervical lesions is high. We now know that cervical cancer is caused almost exclusively by the HPV virus. On the other hand, women often clear an HPV infection on their own without treatment. This is particularly the case with young women, which is why pap smears and HPV testing are not recommended before age 21. Precancerous cervical lesions are graded CIN1-CIN3, CIN3 being the most severe. Overdiagnosis rates are higher for the lower grade lesions, which most often clear on their own. The figures for over diagnosis over women’s lifetime were 70.6% for CIN1+, 63.2% for CIN2+, and 50.0% CIN3+.
  • PSA test for prostate cancer: Low grade prostate cancer is common as men age. Many of these cancers would never cause symptoms during the lifetime of the men. Current estimates are that 60% of prostate cancers detected by PSA would never cause symptoms or death from prostate cancer. Treatment of prostate cancer often results in permanent urinary incontinence and/or sexual dysfunction. This very large over diagnosis and therefore unnecessary treatment is why PSA testing is so controversial. There are certain populations of men who are at high risk of aggresive prostate cancer and these men are probably the only ones who should have routine PSA testing. Here is a link to a risk calculator for prostate cancer: PCPT Risk Calculator.

Bottom Line

Despite the high specificity of cancer screening tests, Bayes Formula shows that false positive tests will be more frequent than true positive tests. For mammograms, colonoscopy and PSA the somewhat low sensitivity means that there will be some false negative tests. In other words, they will miss a few cancers. Pap smear with HPV has the lowest chance of missing a cancer. Over diagnosis is a problem with all cancer screening tests, resulting in unnecessary treatment. This is particularly a problem for breast cancer and especially prostate cancer. The low absolute death risk reduction values and the over diagnosis problems for these tests do not mean you should not be screened, especially if you are in a higher risk population due to family history or other causes of higher cancer risk. All of these screening tests save lives, just not as many as the relative risk values suggest. The vast majority of people will not benefit from these tests and some will be harmed by unnecessary treatment, but a small but substantial number will have their lives saved.

Evidence-based Medicine: What You Need to Know

In this post I will write about how the evidence for how well medicines work, and the risk of side effects are not always what they seem. I’m going to show that relative risk reduction (or relative risk increase) always looks a lot bigger than absolute risk reduction or absolute risk increase. Journals and advertisements always report relative risk reduction for medicines or other treatments because they look more impressive. On the other hand, side effects are almost always reported as absolute risk increase because that looks a lot smaller. I will show you how to calculate both kinds of risk reduction and increase. I will argue that absolute risk reduction or increase is what you really want to know before deciding to take a medicine or other treatment.

Any study of a medicine or treatment always has a group that gets the treatment and a group that gets a placebo. That is the only way to know if the medicine or treatment really works. There is always a placebo effect for any medicine. That is, a certain portion of the people who get a placebo get better. If the number of people who get better from the actual treatment is higher than the number of people who get better from the placebo, then the treatment works.

People who participate in a study know that they might get the treatment or a placebo, but they don’t know which one they got until the end of the study. In medical terminology, they are blinded from knowing whether they got treatment or placebo. If the study is double blind (the most reliable kind) then neither the investigators who administer the treatment nor the participants know which study participants got the study treatment or the placebo until the end of the study.

If the treatment works better than the placebo, that result can be reported in several different ways

Relative Risk Reduction or Increase

Relative risk is the proportion of people who have the disease or condition being studied in the treatment group divided by the proportion of people who have the condition in the placebo group.

For example, let’s say we are testing a treatment to prevent diabetes. The control group and the treatment group each have 100 people. 30 people get diabetes in the control group and only 10 people get diabetes in the treatment group. The risk in the treatment group is 10/100 = 0.1 The risk in the placebo group is 30/100 = 0.3 To compare those risks we divide the risk in the treatment group by the risk in the control group. 0.1/0.3= .33. That means that the relative risk of diabetes in the treatment group is 1/3 of the risk in the placebo group. To change that to the relative risk reduction percentage we use the formula 100(1- Relative Risk). Plugging our values into that formula gives 100(1-.33)= 67%. In other words, the treatment reduces the risk of diabetes by 67% relative to the risk of diabetes in the placebo group. That sounds like a big treatment effect!

All medicines or treatments have some side effects. Placebos can have side effects too, especially if people are told (as they must be) what are the possible side effects of the treatment. Placebo side effects are called “nocebo” effects.

In the hypothetical diabetes study described above, let’s say that a side effect of the treatment is bladder infection. Let’s suppose that 5 out of the 100 people in the treatment group get a bladder infection while only 1 out of 100 in the control group get a bladder infection. The relative risk of a bladder infection in the placebo group is 1/100=0.01. The risk of bladder infection in the treatment group is 5/100=.05. The relative risk of getting a bladder infection in the placebo group compared to the treatment group is .01/.05=0.2. To change that to a relative percentage increase, we use our formula again. 100(1-.2)=80%, This means that the risk of the side effect of bladder infection is 80% more likely in the treatment group compared to the risk of bladder infection in the placebo group.

Absolute Risk Reduction or Increase

Relative risk reduction or increase does not take into account the baseline risk of getting the disease or condition. Absolute risk reduction does take into account the baseline risk.

In our example above the absolute risk reduction is 30% (risk of diabetes in the control group) minus 10% (risk of getting diabetes in the treatment group) = 20%. That means that treatment reduces risk of diabetes by 20%. Notice that this is a much smaller number than the 67% relative risk reduction, but it more accurately reflects how much the treatment would reduce your risk of diabetes.

In our example the absolute risk increase of getting a bladder infection with treatment is 5% (the risk of bladder infection in the treatment group) minus 1% (the risk of getting bladder infection in the control group) = 4%. That means that the risk of getting a bladder infection from the treatment is 4% more than no treatment. Again, a much smaller number than relative risk increase of 80%.

Number Needed to Treat (or Harm)

Another way to look at how well a medicine or treatment works compared to placebo is the number of people that need to be treated in order to help one person. This is called Number Needed to Treat, abbreviated as NNT. The NNT = 1 divided by the absolute risk reduction. In our example the absolute risk reduction of getting diabetes in the treatment group was 20%. 1 divided by 0.20 = 5. That means you would need to treat 5 people to prevent 1 case of diabetes with this hypothetical treatment.

Number Needed to Harm, abbreviated as NNH is 1 divided by absolute risk increase. In our hypothetical example NNH = 1 divided by .04 = 25. This means that you would need to treat 25 people for one person to get a bladder infection

A Real World Example: Fosamax to prevent hip fracture in women with osteoporosis

In the real world, we rarely see a treatment effect as big as in our hypothetical example. Let’s look at a real study on a real medicine. Here are some numbers from a big four year study on using Fosamax (generic name alendronate) to prevent hip fracture in women with osteoporosis (thinning of bones). This study was reported in the Journal of the American Medical Association (JAMA) in 1988. It was the first large study to show that alendronate reduced hip fractures in women with osteoporosis. The study included over 4000 women with osteoporosis (shown by a type of bone scan called a DEXA scan). There were 2,218 women in the placebo group and 2,214 women in the treatment group. In the placebo group 812 women (36.6%) had severe bone thinning in the hips by Dexa scan. In the treatment group 819 (37%) women had severe bone thinning in the hips by Dexa scan. Over the four years there were 18 hip fractures in the placebo group (2.2%) and there were 8 hip fractures in the treatment group (1%)

Now lets do our calculations:

Relative Risk = .01 divided by .022 = .45. Relative Risk Reduction = 100(1-.45)= 55%. . This large relative risk reduction is what the article describing the study reported.

Absolute Risk Reduction = 2.2% – 1% = 1.2%. As you can see, the absolute risk reduction is tiny. Taking alendronate for 4 years reduced hip fracture by only about 1%.

Number Needed to Treat = 1/1.2% = 83.3. That means you would need to treat 83 women for four years to prevent one hip fracture.

Side Effects

Muscle or bone pain

In other studies, muscle or bone pain, sometimes severe was reported by 4% in the treatment group and 2.5% in the placebo group.

Relative Risk = .025 divided by .04 = .625. Relative Risk Increase = 100(1-.625)=37.5%. That means that bone and muscle pain are 37.5% more likely in the treatment group relative to the risk in the placebo group.

Absolute Risk Increase = 4% – 2.5% = 1.5%. This means that there is only a 1.5% increase in risk of muscle or bone pain when taking alendronate.

Number Needed to Harm = 1 divided by 1.5% = 67. That means you would need to treat 67 people for one person to have muscle or joint pain. Notice that the number needed to harm is less than the number needed to treat to prevent 1 hip fracture!

Osteonecrosis of the jaw

This is a rare but very serious side effect of alendronate. Assuming treatment group effect of .01% versus placebo of essentially zero let’s do our calculations.

Relative Risk Increase: since this complication is so rare, there are no trials comparing it with placebo.

Absolute Risk Increase: .01 %.

Number Needed to Harm: 1 divided by .01 % = 1000. This means that 1000 people would be treated before you would see one person with this serious complication.

Bottom Line

Journal articles and advertisements almost always report relative risk reduction for medicines or treatments because it makes the effect of the medicine or treatment look bigger. Side effects are almost always reported as absolute risk increases because it makes them look smaller.

Absolute Risk Reduction or Increase is what you really want to know when you are considering taking a medicine or treatment. Most of the time you will not be able to calculate absolute risk reduction, because you won’t have the actual percentages from the placebo and the treatment groups. Don’t despair though. Dr. Google is there to help. Using the following Google search will usually give you the absolute risk reduction. Type in “absolute risk reduction for (name of medicine or treatment).

Once you have the Absolute Risk Reduction (ARR) or Absolute Risk Increase (ARI)you can calculate for yourself the Number Needed to Treat or the Number Needed to Harm. Just divide 1 by either ARR or ARI.