TumblrFeed

Curate, connect, and discover

Trust Science - Blog Posts

1 year ago

Being able to read journal articles critically is also such an important skill. I’ve been reading journal articles since late high school, but it’s only been in grad school that I’ve learned how to approach them critically. Fortunately there are a few questions you can ask about any study that will help you decide how much you trust the results even if you know nothing of the field.

What journal was the article published in? Some journals are far more reputable than others. Usually a quick Google search can inform you of if a journal is considered reputable.

Do the authors list any conflicts of interest? Conflicts of interest aren’t an immediate red flag, but if the author has a strong incentive to only publish certain results then I’ll definitely be taking a second or third look at the study.

Who funded the study? If Coca Cola funded a study that says drinking one soda a day is beneficial to your energy levels, I’m not trusting that implicitly.

Is it peer reviewed? Peer review can fail, but this is the quickest test for if a study is good or not.

What are the limitations of the study? This plays into the assumptions the author made. Were the experiments only done on white men (often the first standard in medical research though it’s getting better)?

Similarly do the authors list their limitations? The best articles will have a short section on limitations or a paragraph in the discussion about limitations. I am always slightly wary when no limitations are listed.

What is the sample size in the study? This number will usually be found in methods. The bigger the number, the better. However, there are a lot of standards for what the minimum sample size should be. In small animal research, you’re usually looking at a few dozen mice or rats. In larger animal research, you may be looking at less than ten animals (pigs, horses, cows). In human trials (also known as clinical research), it tends to be dependent on what the study is on. Knee replacements probably 15 people or so. Spinal cord trauma would be more like 5 people. (Social science will also have different minimum sample sizes but I’m not familiar enough to give estimates. In general subjective surveys require a lot of people. More objective testing done by researchers will have less people involved)

How many citations does the study have? This one can be a little more hit or miss. An article published a year or two ago may be great and have no citations. While an article published fifty years ago may have a hundred citations but have incorrect information (in this case it’s usually that methods have improved and new information was discovered instead of poor research quality). Niche topics may also be hardly cited despite being good articles.

There are other questions you can ask like “Can I follow the methods?” “Does the interpretation of their results logically follow from their results?”Etc. but those tend to be harder the less familiar you are with a field. And if you’re reading about a study in a news article like CNN, Apple News, etc. there are different tricks to determining how much you trust them (I tend to look for hyperbole and rhetorical devices. One time I found a news article saying physicists had figured out faster than light travel. They were referencing a theoretical mathematics paper that stated using several assumptions hyperluminal travel is mathematically possible)

What I learn from Science & Technology Studies is that you shouldn't blindly trust science because there's a fair amount of fuckery (mostly unintentional but sometimes not) going on in the background, but you also shouldn't *not* trust science in the way that most people who don't trust science don't trust science.

Anyways, hope that helps!


Tags
Loading...
End of content
No more pages to load
Explore Tumblr Blog
Search Through Tumblr Tags