Photo by Katya Austin on Unsplash
Effective Use of Sources (Source Material Part 5)
Okay, you've determined the author cites some good research from a good journal. How do you decide if the source has been used correctly or if there is something purposefully or accidentally misleading about the way the the author has used their sources? Sometimes authors cite great research, but draw inferences they have no business drawing. For instance, popular press articles will sometimes cite correlational research to make causal claims, which can be extremely misleading (researchers are, by the way, not immune from this mistake). Sometimes causal inference can seem legit to the uncritical reader, but ends up being a huge leap from what the the research says.
Another common problem is the cherry picking of supportive research. If the author is writing to make an argument, it's often easy to go find research in support of the view (see also confirmation bias discussed in Metacognition module). This is what anti-vaccination folks do when they cite the discredited work of Andrew Wakefield that (falsely!) links childhood vaccines and autism. In this case, there is one exceedingly poor source that could be viewed to support the link and much work that contradicts it. You want to determine whether you can be justifiably confident that the author has done their due diligence and researched the topic without bias (see also People & Context module). Ask whether the author is accurately representing the current state of understanding in relevant fields of study. Do independent researchers concur with the findings of the cited research? Is the information corroborated by multiple sources or just a single source (i.e., has the finding been replicated)? Did the author bother looking at the full literature, or are they just pulling from what supports what they want to believe and spread?
It's also possible that popular press articles embellish aspects of the research, like exaggerating the certainty of the findings, overextending applications of the findings to real life, or leaving out noted boundary conditions and limitations noted by the researchers. Look to the original source—is the piece you initially read being transparent about the usefulness and quality of the work, including limitations (e.g., can causality be inferred or not?), flaws (e.g., were enough participants collected?), and alternate explanations (e.g., can the findings just as easily be explained in ways that differ from the explanation presented in the piece?).
Learning Check