Statistics filters scientific research to find facts in the data. The p-value is one of the most important statistical calculations done on a data set — it tells you how likely your experiment would be to get the same results if run again (or at least I think I’ve got that right). But it does not tell you the magnitude of the effect or the strength of the evidence. It does not tell you if the experiment was run correctly, nor if the other stats were done correctly. It certainly does not tell you how meaningful the results are. The p-value is useful, but it is easy to be deceived by it, and to deceive others using it.
Surprisingly, as this post by Christie Aschwanden claims, most scientist throw around p-values to support their claims yet do not really understanding its meaning and potential abuse. Now that is faith!
I illustrated the four main uses of the word “faith” here. The meaning I am using here is “trust”. And in an interview on that post, one scientist, when asked to define “the p-value” said:
“I know what many people that I have respected have written about [the p-value] and in fact quoted them. Is that a round about enough way to dodge your question.”
And indeed, that is what religious folks do. They listen to folks they trust, the read folks they trust and though they may not really understand the issue themselves, they trust these people. They have faith.
Take home message: “Faith” is useful, but we still need to remain skeptical.