Translational research vs p-hacking
When is it warranted to act before the evidence is fully established?
The idea of taking abstract research and applying it in practice and community is called translational research.
We are translating research into practice.
We are reducing the time between scientific discoveries and implementation in patient care, and then from small-scale patient care to community-wide applications and public health policy.
It is also sometimes called experimental medicine, or implementation science.
Three stages of Translational Research
Translational research can be thought of in three phases:
Bench (ie, basic science studies)
Bedside (ie, case studies in clinical practice)
Community (ie, policy and procedure)
Given that research at any stage can be bought or fudged, itβs tricky to do this well.
Cutting edge research can be revolutionary, or it can be fake.
Even as new evidence can be fake, established evidence can also have been bought, faked, been misguided, or simply incomplete based on the limitations of instruments at the time.
When paper after paper gets stacked on that shoddy early research, then we can end up with a house of cards such as the idea that the sun, an eternal force in the world, is causing skin cancer, a modern disease.
Was it actually proven that the sun causes more harm than good?
No, it wasnβt. The studies that βprovedβ net-harmful effects from sun were conducted with artificial lights1.
The idea is with artificial lights you can isolate the frequencies.
Eliminating confounding factors is useful for applying the scientific methodβbut the more perfect the study, the less like the real world it is.
Judging the effects of the Sun by the effects of artificial light is akin to judging a fresh apple in season by the effects of imported refined fructose out of season.
Reading research therefore aught to be a careful endeavor.
What makes scientific research trustworthy?
You canβt just go by the headlines.
You canβt just go by the prestige of the publication, the author, the institution.
You canβt go by the date.
You have to actually get in there and get into the experimental design.
You have to care a little bit about science, scientific integrity, and methodology.
At the same time, when it comes to human health, what do you think about the possible need to enact precautions or start implementing things before we have perfect, scientifically validated information? Iβd love to hear your thoughts about this in the comments. Itβs the main point of this post:
How does a p-value validate research results?
When I was studying statistics, one of my teachers suggested deliberately adjusting the sensitivity of the statistical analysis when it comes to human or environmental health.
For example, using a p-value of .10 instead of .05.
A p-value is the probability of getting results like this if no correlation is present.
The standard p-value of .05 means that if no correlation exists, these results would happen less than 5% of the time.
Or, in 95/100 times, these results will not happen due to pure chance.
Setting that value instead at .10 would say these results could only happen due to chance 10% of the tie.
In the higher p-value is used, potentially relevant results will be reported more often than if a lower p-value is used.
So with a p-value of .15, thereβs a 15% chance the results are due to chance, not an actual correlation.
Make sense?
This is also sometimes described as a confidence interval (so, youβd say we are 95% confident these results ARE NOT due to chance).
Raising the p-value increases the likelihood that the results you see are the result of random chanceβwhich is why the standard practice is to only report results with less than 5% likelihood of being caused by chance.
So, what threshold makes sense for a study investigating, say, the likelihood that a product causes autism? Do we want to only see results with less than 5% likelihood of being wrong? 10%? 50%?
Yes, raising the p-value increases the risk of erroneous results (a false positive) than if you had used the standard p-value, but⦠what if you might also save lives?
Itβs all quite tricky given historical policies that led to a preference for βone-drug, one targetβ treatment development, a policy that is currently being dismantled in order to better serve humanity.
And as weβve seen on the safety side, this is a whole different question.
Avoiding new products is relatively less risky than adopting new products.
This is why this is what the current pipeline of new drug to consumer availability looks like:
Understand what p-hacking is, and isnβt
Every once in a while, people try to discredit many of the most popular trends in quantum health (safe sunlight exposure, earthing, electromagnetic hygiene, etc) by saying the evidence to support those habits are βp-hackingβ because the researchers recommend caution or implementation based on less than perfect statistical evidence.
The difference between statistically ethical manipulation of p-values for cautionβs sake and p-hacking is in p-hacking, you actually manipulate the data sets and what results are included/dis-included, rather than changing your statistical model.
Looking outside a standard p-value has legitimate uses in research.
P-hacking is never ethical.
This is why when you read review papers, they always include a flow-chart explaining their process for deciding which evidence was considered, included, and excluded and why.
Statistics is a tool, not a source of Truth, and it is appropriate and wise to use different statistical reasonings for different situations. Or, as I shared recently in You are not a statistic: reason (a tool) without intuition (humanity) is as useless as a flashlight for better seeing the stars.
I hope some of this supports you as you are working your way through this new era where medical research is so freely available.
Updates from Nikko:
Youβre always welcome to email me if you need help finding anything in the website! I know navigation can be a little difficult with a Substack as large as this one. Here are some of the main things you get access to depending on which tier you are subscribed to:
Free: You get notifications about new posts, and a couple bonuses in the welcome email. Here are some of the best posts I have tagged as being free forever.
Paid Monthly or Annual: You can read and watch over 200+ members-only posts that are paywalled in the archive, plus DM coaching/private Q&A for implementing circadian and quantum health at home or in your wellness practice. Here are some (not all!) of the most exciting paywalled posts: Paid Posts.
Quantum (aka Founding Member or Pro/Practitioner Tier): You have access to absolutely everything available here! This includes the Circadian and Quantum Childbearing Year practitioner lessons and my archive of hundreds of curated scientific studies on circadian and quantum health topics, plus some other secret bonuses included in your welcome email. Here is where you can find all the Practitioner Lessons.
Barolet, D., Christiaens, F., & Hamblin, M. R. (2016). Infrared and skin: Friend or foe. Journal of photochemistry and photobiology. B, Biology, 155, 78β85. https://doi.org/10.1016/j.jphotobiol.2015.12.014
Nikko, this is a terrific post. Thanks for making the sometimes obscure world of research just a little more accessible, and emphasizing the need to examine the experimental design and not merely check dates and numbers.