p-hacking & self doubt

worship p-valueAs my confession history illustrates, I love looking for radical truth. I have done it in religion, politics, medicine and more. This hunger for insight has led to my grabbing on to ridiculous ideas. Fortunately my personality is balanced by a huge dose of skepticism. My skepticism has also pulled me out of many of my once treasured delusions.

I make a living in medicine. Prior to my present work, for about five years I was an acupuncture practitioner, and later practiced homeopathy for two years. But my skepticism pulled me out of both of these fields and I have practiced allopathy (“Western” medicine or “Modern” medicine or “Mainstream” medicine) for more than two decades now. Statistical evaluation of medical research is the radical truth that backs the medicine I now practice. I was even a university instructor in statistics and research for a few years. In Modern medical statistics we use a sacred test called “the p-value”.

During all my years in medicine, practicing and teaching, I have been very aware of both the common fraud in research and the common misapplications of statistics. But during the last six months, I have read several articles showing the false-confidence we have in p-values — a foundational test in orthodox statistics.  Here is a post by Deevy Bishop showing how researchers actually fraudulently abuse our confidence in p-values by doing “p-hacking”.  The picture above is from this article on the same issue.

Many of my readers already know this information, but they are highly informed specialists. But most readers may have trouble understanding the math and/or may not be too interested in reading further. So the reason I am writing this is to merely again confess my blindness, my greed for special certainty and my vulnerability to deception.

Take home message: To take action in life, we often need to take ourselves seriously, but disappointingly, less often than we imagine.

1 Comment

Filed under Philosophy & Religion

One response to “p-hacking & self doubt

  1. P-hacking an interesting topic. Some time ago I drafted my notes and ideas on this topic and was going to write a blog post about it. My notes and ideas didn’t get developed into post. But I saved the audio interview (referenced below) that originally inspired me to think more about P-hacking.

    You may be interested in this audio interview with Daniel Lakens on P-Hacking and Other Problems in Psychology Research. Professor Daniel Lakens is from the Eindhoven University of Technology, studies psychology and blogs about research methods and open science.

    Daniel discusses: What’s wrong with the social sciences? and why so many psychology papers can’t be trusted, and what solutions might exist for the problem (including how to fix the skewed incentives in the field).


Please share your opinions!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s