Scientific Studes

Assault on Science

I recently read a very interesting book called Science Under Siege. The authors are Peter Hotez, pediatrician and vaccine researcher at Texas Children’s Hospital in Houston, Texas and Michael Mann, the prominent climate researcher who found strong evidence for manmade global warming. It is an excellent book and I highly recommend reading it. Public confidence in science has significantly declined since the pandemic, but the majority of polls continue to show strong support for science (see this link to the Pew Research Center). On the other hand, there is a well funded, sophisticated effort to discredit science and scientists. Many people do not understand how science is done, how it is funded, and its contributions to society. In this post I will write about all of those things, as well as the people and groups who are funding the assault on science and scientists.

What is the Scientific Method?

The scientific method is a systematic way to advance knowledge. Here are the steps of the scientific method: This excellent formulation is copied from a webpage from American Journal Experts.

  1. Define a question: Constructing a clear and precise problem statement that identifies the main question or goal of the investigation is the first step. The wording must lend itself to experimentation by posing a question that is both testable and measurable.
  2. Gather information and resources: Researching the topic in question to find out what is already known and what types of related questions others are asking is the next step in this process. This background information is vital to gaining a full understanding of the subject and in determining the best design for experiments. 
  3. Form a hypothesis: Composing a concise statement that identifies specific variables and potential results, which can then be tested, is a crucial step that must be completed before any experimentation. An imperfection in the composition of a hypothesis can result in weaknesses to the entire design of an experiment.
  4. Perform the experiments: Testing the hypothesis by performing replicable experiments and collecting resultant data is another fundamental step of the scientific method. By controlling some elements of an experiment while purposely manipulating others, cause and effect relationships are established.
  5. Analyze the data: Interpreting the experimental process and results by recognizing trends in the data is a necessary step for comprehending its meaning and supporting the conclusions. Drawing inferences through this systematic process lends substantive evidence for either supporting or rejecting the hypothesis.
  6. Report the results: Sharing the outcomes of an experiment, through an essay, presentation, graphic, or journal article, is often regarded as a final step in this process. Detailing the project’s design, methods, and results not only promotes transparency and replicability but also adds to the body of knowledge for future research.
  7. Retest the hypothesis: Repeating experiments to see if a hypothesis holds up in all cases is a step that is manifested through varying scenarios. Sometimes a researcher immediately checks their own work or replicates it at a future time, or another researcher will repeat the experiments to further test the hypothesis.

This is clearly not an easy process. There must be transparency at every step so that other researchers can evaluate the methods and repeat the experiments to be sure they get the same results.

How are scientific findings published?

Once results are obtained from a scientific study, a paper describing the results is prepared by the authors. There are usually several authors for any scientific paper. The principal investigator prepares the initial draft and sends it to his/her colleagues. There are usually many drafts and modifications before a final paper is ready to submit. The final paper as agreed upon by the authors is submitted to a peer reviewed scientific journal. The paper is first reviewed by the editor of the journal. If the editor feels there is significant problem with the paper or if he/she thinks it is not appropriate for readers of the journal then the editor can reject the paper. If the paper passes editorial review, then it is sent to at least 3 scientists in the same field for evaluation. These are called peer reviewers. Each reviewer writes a review of the paper, which are sent to the editors of the journal and to the authors of the paper. The reviewers often suggest modifications to the paper. The journal editor may reject the paper based on the reviewers comments, may suggest that modifications be made and the paper be resubmitted, or much more rarely may accept the paper for publication based on positive reviews from the peer reviewers.

As I can attest from my own numerous scientific publications, the acceptance of a paper by a peer reviewed journal is a long and rigorous process. I have also had papers rejected by the editor and after review by peer reviewers.

Very rarely, investigators publish papers based on fraudulent data. This is usually discovered eventually and the editors then publicly retract the paper. This happened to Andrew Wakefield with his publications about a connection between MMR vaccine and autism in the medical journal the Lancet. All of these papers were found to be based on fraudulent data and were retracted. He actually lost his medical license in the UK because of this. Subsequent large very well designed studies showed no connection between vaccines and autism. This is an example of how the scientific method advances knowledge.

Double blind randomized controlled trials

This kind of trial is the gold standard of scientific research. Subjects for the research are recruited and are randomly assigned to either the experimental group or the control group. Great care is taken to be sure that the selection for either group is completely random. The size of the two groups has to be large enough to ensure that any factors that might bias the results should be balanced out in the two groups by the random selection. The intervention being tested (usually a medicine or other treatment) is given to the experimental group and an identical placebo (inactive) pill or intervention is given to the control group. The investigators administering the treatment don’t know whether they are administering the experimental treatment or the placebo, and they also don’t know which people are in the experimental or the control group. That’s why this kind of trial is called “double blind.” The code for which patients got the experimental treatment and which got the placebo treatment is not broken until the end of the trial. Double blind randomized controlled trials give the most reliable bias free results, but they are very expensive to conduct. There are other kinds of trials as well, because not every research question lends itself to a randomized controlled trial. Almost all well designed research is expensive to conduct. In the next section I will write about how research is funded.

How research is funded

Most research is conducted at academic institutions. Funding for research has to cover the salaries of the researchers and their staff, and the research infrastructure of the institution. All of that is very expensive. Almost all research is funded by grants, either federal (National Institutes of Health (NIH) or National Science Foundation (NSF) or private foundations. Most large grants are federal.

Grant application process

Some grants are designed for application by institutions themselves but most grants are applied for by research scientists. Application for a scientific grant is a major involved process. There is a 12 page limit for the main application for an NIH grant, but supplementary documents required like budgets and investigator bio sketches can run to 150 pages. Each federal grant application is evaluated by a study section composed of eminent scientists in the field. Funding is only available for a small number of grants so most grant applications are rejected. If the study section gives the grant a good score the grant may be awarded to the institution or the investigator. Even if the grant is awarded to the investigator, the grant money is administered by the institution. It never goes directly to the investigator. A percentage of each grant is used by the institution to pay for its research infrastructure. The rest is used by the institution to pay the investigator’s salary and to pay for the costs of the research. Private foundations each have their own rules for grant applications, but they are also difficult to get. Grants from private foundations are usually, but not always smaller than federal grants.

Are scientific results truth?

Science is a systematic way to search for truth about how the world works and how we can successfully manipulate it to get desired results. It is a mistake to put too much emphasis on the results of one study. When many studies find the same or similar results, we have increasing confidence that the results represent truth, but never absolute. Scientific results are always to some degree provisional. That does not mean they are not useful.

Benefits of science to society

Rather than enumerate the benefits, here is a link to a University of California website that explains the benefits much better than I can. As the webpage points out, the benefits of science to society are substantial.

Attacks on science and scientists

Because science is a systematic search for knowledge, the findings of some scientific studies are often inconvenient for certain groups or individuals with ideological beliefs. Some scientific findings also threaten some wealthy people whose wealth comes from industries that scientific results threaten. The fossil fuel industry, for example, has created huge wealth for companies and individuals. Climate science has clearly shown that CO2 emissions from fossil fuels is causing rapid global warming that will be catastrophic if CO2 emissions are not drastically reduced. Scientific research has also been instrumental in developing alternative energy sources such as solar, wind, and others that have become economically viable. The fossil fuel industry has invested tremendous amounts of money to attack these findings. They work at creating the idea that these well validated scientific results are controversial. They hire people with academic credentials who question that CO2 emissions are causing climate change. This kind of opposition research is rarely published in peer reviewed journals. Front groups are created as well as PACs to lobby legislators that are funded without revealing the source of those funds. This is so-called dark money, but it has been shown to come primarily from people who get their wealth from fossil fuels. Koch industries is a good example. There also is a large industry that promotes supplements, vitamins, and alternative medicines that have no basis in science. This industry attacks legitimate health research as well as proven preventive treatments, particularly vaccines for children and adults.

These attacks on science are sophisticated, coordinated, and so far unfortunately very effective. In addition prominent scientists have been subpoenaed to appear before hostile congressional committees, harassed and threatened with arrest and/or bodily harm. These things are well documented in the book by Hotez and Mann.

Bottom Line

Science is a systematic way to advance knowledge. As results are validated by multiple well designed studies we find out more about the world and how to manipulate it to our benefit. Publication of scientific results in peer reviewed journals is a difficult and rigorous process. Almost all science is done in academic institutions and is almost entirely funded by federal grants and grants from private foundations. Obtaining grant funding is also a very rigorous process. Grants are administered by institutions and grant money is never given directly to investigators.

Wealthy people who stand to lose money because of scientific findings are conducting a sophisticated, well funded campaign to attack scientific results they don’t like and to discredit, harass, and threaten legitimate scientists.

Lies, Anecdotes and Evidence

Lies

Lies are deliberate falsehoods. The person telling a lie knows the truth, but chooses not to tell it. There are several main reasons for lying. The first, and most common is to avoid responsibility or punishment for something you have done. We are all familiar with the young child who lies for this reason. “I did not break the lamp, Mommy. It just fell off the table.” An alternative strategy to avoid blame or punishment involves blaming someone else. “I did not break the lamp, Mommy. Jaimie did it.” The third reason is to manipulate someone else’s behavior to get them to do something that you want. The classic form of this kind of lie is the telephone scam. The caller asks for money and promises to do something he/she has no intention of doing.

Lying is always destructive, but when lying becomes common in the public sphere, and when lies are told by people who would normally be trusted sources of information, then trust is eroded for everyone, even people who are telling the truth. A substantial portion of the public begins to believe that everyone is lying. This clearly happened through most of 2020 with regard to the pandemic.

Sometimes things are called lies that are not. If someone says something they truly believe to be the truth and are later proved incorrect by subsequent evidence, they are not lying; that is they are not telling a deliberate falsehood. An example of this is that early in the pandemic, Dr. Fauci and other experts at the CDC said that universal mask wearing was not necessary. As evidence accumulated, it became clear that this was incorrect; that universal masking markedly reduced coronavirus transmission. Their initial recommendations were incorrect as proved by later evidence, but they were not lies.

Anecdotes

We have all seen articles such as these online: “Physician dies after receiving COVID vaccine”; “Patient gets infected with COVID-19 two weeks after second COVID vaccination”; “Study shows hyrdoxychloroquine prevents COVID-19.”

These are all examples of anecdotes. Anecdotes are stories, based on personal experience or reports of results based on small non-representative samples. Often these stories are dramatic as shown in the examples above.

Our brains are hard-wired to look for patterns. This was very useful in the evolution of our species, because not recognizing patterns was much more likely to be fatal! Unfortunately, this tendency to look for patterns causes us often to see patterns where there aren’t any. That is why people are predisposed to believe dramatic stories that are based on little or no evidence.

Here are a couple of examples not related to COVID-19:

Flu shots can give you the flu

As a family physician I saw this one every fall. Flu shots are given at the beginning of the season when respiratory viruses are becoming more common. Inevitably, some people would get sick with a viral infection shortly after receiving their flu shot. Flu shots, of course, cannot cause flu. They are made of pieces of viral proteins that cannot infect cells in the body. Such is the power of anecdote though that all my scientific explanations made little difference to people who were convinced that the flu shot caused their viral infection.

Childhood immunizations cause autism

Symptoms of autism generally begin about age two. By that age almost all children have received several immunizations. Once again, this is the power of anecdote. Many parents of children diagnosed with autism are convinced that immunizations caused the autism. This hypothesis has been studied in several very large, well designed studies and thoroughly disproved. Many of these parents remained convinced because of their anecdotal experience that immunizations caused their children’s autism.

Physicians and anecdotes

Physicians are not immune to the power of anecdotes. As a matter of fact many continue to prescribe medicines or treatments that have been shown to be worthless or even harmful. Their excuse is almost always that “in my own experience this treatment works” or “I have never seen these harmful side effects in my patients on this treatment.” I have been guilty of this myself. Years ago a study from Canada showed that a simple exam of an ankle injury could rule out a fracture and save getting x-rays on all of these. The first time I tried it, the exam showed no evidence of a fracture, but the patient ended up having a broken ankle. Despite the fact that this was a perfect example of anecdotal evidence, I got an x-ray of every ankle injury for a long time after that.

Evidence

Evidence, as opposed to anecdotes, comes from systematic studies of large groups of people. Evidence helps answer important questions related to health and disease. There are several forms of evidence that are useful. The kind of evidence depends to some extent on the question. Below are brief descriptions of studies that helps us understand and better treat illnesses such as COVID-19.

Double blind randomized controlled trials

There is only one kind of evidence that gives us confidence that A causes B, or that A does not cause B. It is called a double blind randomized controlled trial. Here is how it works. A large number of people are recruited that are representative of the population who might use the treatment. The subjects are randomized into two groups by the equivalent of flipping a coin for each person. Instead of a coin, a computer does the equivalent of a coin flip. Dividing subjects in this way means that each person has an equal chance to be included in either group. One group will get the experimental treatment and the other group, called the control group will get a placebo that looks just like the treatment medicine but does not have any biologic activity (a sugar pill for example). The subjects don’t know whether they are in the treatment group or the control group. They are “blinded” as to which group they are in. The medicines are labeled with a code, so that not even the investigators know who is getting the treatment and who is getting the placebo. That is the investigators are “blinded” to who is in the treatment group and who is in the control group. That is what double blind means. Neither subjects nor investigators know who is getting the treatment and who is getting the placebo.

What is the reason for all this “blinding”? It is called the placebo effect. In any trial there are a number of people who will get better even when they get a placebo. These are not just psychological effects. When the brain thinks it may be getting a medicine, hormones and neurotransmitters show real physical changes and that makes people feel better. People who are giving the medicines can unwittingly increase the placebo effect, which is why the investigators are “blinded” too. At the end of the trial the code is broken so investigators then know who got the treatment and who got the placebo. If the treatment group has statistically more positive results than the control group, then we know the treatment works. If the control group has about the same improvement as the treatment group, then we know that the treatment does not work.

Anecdotal evidence suggested that hydroxychloroquine might work to treat or prevent COVID-19. A double blind randomized trial though showed no effect on either prevention or treatment of COVID-19.

Anecdotal evidence suggested that high doses of vitamin D might prevent COVID-19 or make it milder. A double blind randomized trial showed no effect of vitamin D on prevention or severity of COVID-19

COVID-19 vaccines were also tested in double blind randomized controlled trials. That is why we are confident that they work very well at preventing infection, hospitalization and death.

Double blind randomized trials are expensive and take a long time to get results. It is not practical to do double blind randomized trials on every scientific question. Also, there are some questions that a double blind randomized controlled trial cannot answer. Sometimes a randomized controlled trial would be unethical. That would be the case if using a placebo group would clearly cause harm to that group.

There are other kinds of evidence that help us decide whether it is worth it to do a double blind randomized control trial or that provide useful evidence when placebo controlled trials are impossible or unethical.

Case-Control Studies

This kind of study is used to answer a question about whether exposure to something causes a disease. The. “exposure” is measured in people who already have the disease (cases) and in a group of similar people who do not have the disease (controls). Unlike randomized controlled trials, case-control studies cannot tell whether the exposure causes the disease. They can only show that the exposure is associated with the disease. There is always the possibility that the difference between the cases and controls is related to some difference between cases and controls other than the exposure being measured. Years ago a case-control study showed an association between coffee drinking and pancreatic cancer. People who drink coffee were much more likely to smoke. It was the smoking that caused the increased risk of pancreatic cancer, not the coffee drinking. Systematic differences between cases and controls other than the exposure being measured are called confounders. Confounders that we know about, such as smoking, can be removed from analysis by statistical techniques. In a case-control study, however, there is always the chance that there are confounders we don’t know about. That is why we can never use a case-control study to say the exposure causes the disease or condition.

Another potential problem with case-control studies is called “recall bias.” Let’s say that the exposure we want to measure is sugar intake and the disease we want to measure is diabetes. We ask people in both the diabetic group. (cases) how much sugar they eat in a month and ask the same question for non-diabetics (controls). People are unlikely to remember accurately the amount of sugar they ate in a month. People may tend to minimize how much sugar they eat. This will obviously make the results of the study much less reliable. People tend to remember (recall) things selectively. That is recall bias.

Population Studies

These studies look at whole populations that live in a particular area. They are similar to case control studies in that They look at an exposure (say intake of saturated fats) with a disease (say heart disease). If consumption of saturated fats is different in whole populations of different countries then the incidence of heart disease in those populations will be measured as well.

The best population studies follow populations over time. Like case-control studies, they can only show association, not causation, but sometimes they can be very useful anyway.

The most famous population study, the seven country study by Ancel Keys followed populations in seven countries over many years. This study was the first to show that animal fat intake was associated with heart disease and that high blood pressure was associated with heart disease and stroke.

The Framingham Study followed the population of Framingham, Massachusetts for many years. In addition to confirming the findings of the seven country study, It showed for the first time the association between diabetes and cigarette smoking with heart disease.

Peer Review

Until the pandemic the main way that results of evidence based studies became known to the public was publication in scientific journals. In order to get a study published in a reputable scientific journal, it first has to be looked at by one of the editors of the journal. The editor decides if the study meets the criteria for that journal, and many papers get rejected at this stage. If the editor thinks the study is done well and fits the criteria for the journal, then the editor sends the paper to at least three experts in the field of the authors of the paper. They critique the paper, make suggestions to improve it, and send their comments to the authors and the editor of the journal. Many more papers get rejected at this stage. Sometimes the authors are asked to re-submit their paper after making changes suggested by the reviewers. This whole process is called peer review. This process assures that for the most part only well done studies make it to publication. The other quality control in scientific studies is replication. One study, no matter how well done, may have missed something. If other peer reviewed studies come up with the same results, then we have increased confidence that the findings are real.

Pre-print servers

The peer review process takes a lot of time. It is usually months, sometimes many months between the completion of a study and its publication in a peer reviewed journal. In the midst of the pandemic that was too long. Physicians who were treating desperately ill patients with a new disease needed to know the results of trials quickly. Scientists could publish their results online as soon as the study was completed and before submission of their studies to peer reviewed journals. The websites that allow them to do this are called pre-print servers. This process allows the results of studies to be available to doctors very rapidly, but the quality control of peer review is missing. Results published on pre-print servers should be considered preliminary. Many of them will not make it through the peer review process when they are submitted to scientific journals.

The Media

Most non-scientists do not read scientific journals or pre-print servers. They find out the results of scientific studies through the media. The media serve an essential function in translating the jargon of scientific papers into language that most people can understand. While there are a few excellent journalists who understand how to evaluate scientific papers, for the most part that is not the case. The media look for things that are splashy and they are just as likely to trumpet preliminary findings from pre-print servers as they are to discuss peer reviewed papers in journals. Even if they discuss the findings of peer-reviewed papers, they often emphasize the positive and do not report the author’s caveats about how carefully to interpret the findings.

I refer you back to my previous post about reliable media sources of medical information

Bottom Line

Lies are deliberate untruths, but when lying comes from sources that we usually can trust, that creates distrust of all sources, even ones that tell the truth.

Anecdotes are compelling stories based on personal experience or reports of small non-representative groups. Our brains are programmed to look for patterns, even when those patterns are figments of our imagination.

Reliable evidence is based on trials of large numbers of people who are representative of the population at risk of disease. Double blind randomized controlled studies are the gold standard of reliable evidence, but other kinds of studies can give good information as well.

Media reports of scientific studies can be useful, but definitely need to be taken with a grain of salt. Some media sources are much better than others.