Subscribe to the OSS Weekly Newsletter!

Want to support the McGill OSS?

Not All Scientific Studies are Useful

Scientific studies should contribute significantly to our knowledge base. Many don’t.

Why do we pursue science? Because we want to know what to eat, what medications to take, what cleaning agents to use, what chemicals to avoid, which personal care products to buy, what sort of exercise to engage in, and how to avoid disease. Basically, we want science to work for us with a goal of guiding us towards a healthy, problem-free long life. Yes, we also do have an innate curiosity for how the world works that prompts fundamental, non-goal oriented research, but here too, the hope is that somehow it will provide some practical benefit. When a couple of researchers in the 1940s were curious about the behaviour of hydrogen atoms in a magnetic field, they had no idea that decades later their findings would lead to magnetic resonance imaging (MRI), one of the greatest advances in medicine.

It is obvious that society has been richly rewarded with the fruits of scientific research. We know how to produce a plentiful food supply, how to deal with infections, how to vaccinate against disease and how to develop computer technology that allows us to call up millions of scientific references in a fraction of a second. The problem is what to do with all that information. How do we separate studies that provide potentially useful data from ones that do not add in any significant way to our knowledge base? In the latter case, a study could be properly conducted and produce reliable results but nevertheless the world would not be worse off had it not been carried out. Research is expensive and funds need to be directed towards efforts that have a chance of improving our lives in some fashion. There are way too many publications these days that should never see the light of day.

Some research is just nonsensical and some is fraudulent. Some are both. Like the paper published in the neuropsychology journal, “Perceptual and Motor Skills” by a scientist from the University of Southern Brittany in France with the intriguing title, “Bust Size and Hitchhiking: a field study.” As one might expect, this study claiming that women with larger breasts are more likely to be picked up when hitchhiking received a great deal of media attention. The same author published a subsequent study that found blond women were more likely to be successful at hitching a ride. One wonders if he is contemplating another trial to see if blond women with large breasts are even more successful. Besides the “do we really need to know this” aspect of these studies, it turns out that the author did not have the data to support the conclusions. Neither did he have the data to support his study alleging that women dressed in red are perceived by men as having stronger sexual intent. It took years, but these nonsensical papers have now been retracted by the journal.

Then there are studies that are not fraudulent, nor poorly designed, but provide no useful information. Such as “Long-Term Dietary Consumption of Grapes Alters Phenotypic Expression in Skeletal Muscle of Aged Male and Female Mice.” Researchers fed mice freeze-dried grape powder, the equivalent of two cups of grapes every day, and investigated changes in the expression of genes associated with muscle health. They found that there were some changes that suggest an improvement in muscle function, but they did not actually measure any muscle function. Of course, mice are not men or women, and nobody is going to eat two cups of grapes a day hoping to increase muscle strength.

Similar mice studies are aplenty. “Lycopene Alleviates Depression‐Like Behavior in Chronic Social Defeat Stress‐Induced Mice by Promoting Synaptic Plasticity” is one. In this case, lycopene, the red pigment found in tomatoes and watermelon, was fed to mice that had been purposefully stressed to such an extent that they lost interest in the sugar-water mixture they normally guzzled. In mice studies this is interpreted as depression. The lycopene supplement renewed their interest. The media breathlessly talked about a “natural cure for depression,” ignoring that the brain of a mouse is not the same as the brain of a human, with some obvious exceptions. Also ignored was that the dosage given to the mice was equivalent to a human consuming 21 tomatoes and 14 cups of watermelon every day. A challenge for even the greatest tomato lovers.

Proper randomized placebo-controlled trials that actually use humans do not necessarily provide worthwhile information either. A study that “investigated the effect of regular mango intake on inflammation and insulin sensitivity in participants with overweight or obesity and chronic low-grade inflammation” was well done by reputable scientists. But what are we to make of the results that prompted headlines such as “Diabetes risk lowered by eating one surprisingly sweet food?” I think, nothing.

The study looked at some blood chemistry differences between overweight subjects who consumed either two frozen mango smoothies or two Italian ice smoothies every day for a month. There were no changes in any markers of inflammation or in blood glucose which is the indicator of diabetes. There was a difference in a marker of insulin sensitivity but that may have been due to the control, the Italian ice smoothie, having a negative effect on insulin sensitivity thereby making the mango smoothie look better. In any case, that is neither here nor there. The same experiment could have been done with apples, oranges or bananas with, in all likelihood, the same result. Indeed, the only conclusion the researchers arrived at is that “the data support consuming mango fruit as part of a dietary pattern to address insulin resistance.” That doesn’t mean that adding two mango smoothies a day to the diet has any benefit. Any suggestion that consuming two mango smoothies a day to reduce the risk of diabetes is unjustified. People at risk for diabetes, as shown by a blood test, need proper nutritional counseling, not mango smoothies. This study was essentially a waste of money. But at least it wasn’t taxpayer money. Guess who funded it. The National Mango Board. What a surprise.


@JoeSchwarcz