Find better hearing when you understand research methodology

It seems every day I read about some new study proving that ‘Brand X is good for your heart’ or ‘Product Y slows down aging’.  We’re taught that clinical research is scientific and therefore truthful; however, not all clinical research is equal.  So the question is – should you believe any of it?

The first question to be asked is: who funded the research?  These days a lot of medical research is funded by companies selling something – pharmaceuticals, medical devices/procedures, miracle cures, you name it.  Right off the bat you’ve got bias, and bias is bad for science.  Let’s say a researcher collects some data about a mood-enhancing medication and reports that 60% of the subjects experienced a 20% improvement in their mood while taking the medication.  That sounds pretty good, right?  Well, what they didn’t report was that the remaining 40% of the subjects experienced a 70% decline in their mood – not so good after all.  Like many things in life, research data can be ‘spun’ to paint the picture the researcher (or funding agency) desires.

The next question to be asked is: was the research published in a peer-reviewed journal?  It’s difficult to spin the data when other, unbiased professionals, are deciding whether your research has met the requirements of ‘good science’ and is worthy of publication.

So what are the elements of good scientific research?  The fundamental tenants are: use as many test subjects as possible (a large N), draw subjects from a broad range of groups (age, race, gender, education levels, economic levels, etc.), use both a study group and a control group, remove/control all variables possible, ensure that the individuals collecting and analyzing the data aren’t biased towards a particular outcome (which is to say they have been ‘blinded’), ensure findings meet high levels of statistical significance (95%-99%) before reporting them as valid, and report ALL findings – even those that don’t support the initial hypothesis. 

In addition, for clinical findings to be considered scientific fact they must be repeated – preferably by different researchers.  This takes time and funding, and because there are often profits at stake, early research findings are sometimes rushed out the door.  The fact that subsequent studies often contradict one another is proof of this.

The last question to be asked is: what’s the publication citation?  It’s astonishing how many studies are reported in the media with absolutely no reference to the publication citation.  Before I place my professional faith in any clinical research I want to read and assess it myself.

So the next time you read that Product Z can make you 50 years younger, or cure tinnitus be skeptical – it’s just good science.

Authored by Dr. Matthew MacDonald