Running and Depression: Be careful about spreading the results of research until you understand it.

During the COVID pandemic, a great deal of scientific and medical literature was published on line without going through the normal peer-review process. The justification was that things were happening so fast that scientists and doctors needed quicker access to the information being discovered to be able to implement prevention and treatment strategies.  Many of these were published on sites like MedRxiv (https://www.medrxiv.org/content/about-medrxiv), with little or no peer review. Some of these articles were later shown to be flawed, either in terms of the science of the research methods or in the presentation of the conclusions to be drawn from that research. In a number of cases, these articles, which would likely not have been published in normal (peer-review) circumstances were “taken down”, “unpublished” if you will. This was criticized by many during the pandemic, such as Rapid publications risk the integrity of science in the era of COVID-19, by Bagdasarian, Cross and Fisher.

Except, of course, nothing can truly be “taken down” from the internet. If it is out there, it can be copied, saved, reposted. Certainly this has been shown to be true on social media, where embarrassing posts (e.g., sexual, racist, etc.) have continued with lives of their own even after the original was taken down. In the case of medical and scientific articles, the information is out there, and the study can be referred to and cited by others. And with the thousands and thousand of such articles published on line, only a small percent are ever taken down, even if the methods used in the research are flawed, and the conclusions presented are inaccurate. This was the source of a lot of misinformation during the pandemic, transmitted on social media because it was “in a scientific journal”.

Hopefully we are past that (fingers crossed!), but we are still not without other misleading research being reported. As with almost everything, on line or in print or conveyed by mouth, people are more likely to believe things that reinforce what they already believe (“confirmation bias”), and in general, regarding medical and scientific information, most people are willing to read just the “bottom line” (is “X” good for you? Is “Y” better than “Z” for “A”?), which is usually far too simplistic. This is not limited to laypeople, few of whom have the expertise to actually read and interpret a research study; many doctors (and sadly residents and medical students) read only the Abstract or at best the Conclusion, skipping or skimming over the only important parts of a research paper, the Methods and Results. Reading the Abstract may be adequate to decide whether to read the whole article (“if they actually found what they say they found, do I care?”) but should never be conflated with actually having read it, and thought about it.

And while peer review should ensure that the methods, probably, were legitimate and that therefore the results are accurate, it does not always flag potential problems, particularly with regard to overstating the significance of the results in the conclusion -- or a one-sentence abbreviated conclusion. A recent study (and I pick on it not because it is more egregious than most, but because it was brought to my attention by Medscape, reporting on its presentation at an international conference), is ” Antidepressants or running therapy: Comparing effects on mental and physical health in patients with depression and anxiety disorders”, published in the Journal of Affective Disorders, May 2023. The study, done by a group of researchers in the Netherlands, in fact showed, as Medscape reports, that while both anti-depressant medications and running helped depression, running was better for overall physical health. That is good news if you are a runner, I guess, or if you have been wondering if taking up running would be good for your depression. But if your take-away is something like “See? Natural things – like running – are better for you than drugs!”, there are some details you need to pay attention to. Like who was studied. In any study, it is critical to see if the population studied is similar to you (or your patient, if you are a health care clinician). In this case the average age of participants was about 38. So if you are in that age range, it could apply to you. If you’re 70, the age of the oldest participants, maybe taking up running might not be the best idea.

But a much more important flaw in this study is illustrated by saying “taking up running”: it has to do with the allocation of people to the different treatments, running or medication. In a good study, participants are randomized to one treatment or another. In this study, people chose the treatment they wanted to try, with only those left over allocated to one treatment or another. 83 people chose running with another 13 assigned to it; 36 people chose medication, with another 9 assigned to it. So of the 141 participants, 83, 59%, chose running. Although those already exercising more than once a week were excluded, it is still possible, if not likely, that many of that 59% were, at least in the past, runners, or people who wanted to run. This is what is called a potential confounder, something that is associated with the treatment and the result, and leads to results that are much less convincing than does random allocation.

So does this mean running is not as good or better than drugs for treatment of depression? No (although obviously it requires the physical capability of being able to run). This study provides evidence that in a group of 141 people, average age 38, 58% female, 59% of who chose to be in the running “arm” of the study, it worked as well. But those are important caveats that make the study weaker, and less likely to work for someone who doesn’t fit the profile.

Enough of this study. I chose it because it came to my attention, but also because it was published in a reputable, peer-reviewed journal, and it is not totally worthless – but to draw the conclusion that “running is as good as drugs for depression and better for your overall health” is true for anyone, for you, could be inaccurate.

The impression that one can take away from hearing from someone that “a scientific study showed that…” (and tell me you never hear that!) can be a little, a lot, or completely wrong unless you both read the whole study and have enough background to understand what it does not say as well as what it does. Beyond “was this a well-done study?” and “was it published by reputable scientists in a reputable journal?”, the questions of “to whom are the results most likely to apply?” is also important. You may think yourself healthy and in good shape for your age, but a study of, say, 25 year olds, may not apply to you if you’re 65. The most reliable characteristic is replication of the results by other well-designed studies.

Is there a social justice issue here? I do not know. Probably well-off people have better access to doctors and other professionals, but they also have more choice and are able to choose the ones that they know have the same beliefs (or biases) that they do; for example, exercise is good, drugs are bad, natural is good, etc. My personal (not statistically valid!) experience is that people who are likely to believe something because it shows what they want to be true and to discount studies that show what they don’t want  to be true, come from all socioeconomic, ethnic, and other groups.

But my advice is to take a deep breath, and not believe everything you hear, or read, is 100% true, and certainly not that it is more broadly applicable that whatever the research showed.

Even if you want to.