To feign credibility for their holy propaganda, religious professionals love to preface their sales talk by saying something like “in the original language our holy book says …” Read what I wrote here about that deceptive trick.
Such deception is not monopolized only by referring to religious texts. People use science jargon in the same way. People who preface their sales points by saying “studies show” may also be trying to say, “look, this is science, don’t question it.” or “I know my stuff, don’t question me.” So it is not just the religion hucksters who play the game.
I once taught a course on “Medical Research” to graduate students. My course goal was to teach the future physician assistant students to see through the bullshit of drug reps and to not fall prey to smokescreens of “science” talk. I wasn’t trying to teach them to be researchers but to see through the smokescreens of researchers or the hucksters who cherry pick research.
Unfortunately, learning to analytically dissect supposed scientific studies and to see through their shortcomings takes a lot of training. But the first step to being objective is to never believe someone who says “studies show” without demanding much more evidence. And if the science salesman or woman actually happens to hand you the study they are quoting, the first things to examine include:
- the reputation of the source journal
- the research funding source and the vested interests of the researcher
- the number of people studied and the population chosen
- the type of study (qualitative vs. quantitative, epidemiological vs experimental, retrospective vs prospective, etc, etc). Understanding study types is a bit tough. But it allows you to see through the common deceptions:
- Correlation vs Causation deceptions
- Generalization deceptions
- The Ecological Fallacy
So beware: In developed countries, among the highly educated, science is the new Bible and “Studies shows” is the equivalent of “In the original Greek” for Bible pushers. Don’t be hoodwinked by either.
If you want to watch a professional researcher analyze studies, go to Tom Reese’s site “Epiphenom” where he reviews studies on religion done in all sorts of journals.
Questions to readers: Do you have other cautions for beginners to consider when questioning a study?