Photo by Martin Adams on Unsplash
Kinds of Source Material (Source Material Part 2)
When the author is writing directly about science or making other claims about how the world works, it's good to see that the author is using sources from an academic literature (e.g., original peer-reviewed journal articles from a reputable field of study). How do you know if the sources cited are reputable, peer-reviewed journal articles? This library site at Wilfred Laurier University and this one at University of Toronto will give you brief tutorials on peer-review.
If you're reading a piece with a scientific or historical component, it's important that the author has done legitimate and complete research and has shared their research as fully as possible with readers. Ask whether they've cited a reputable and applicable peer-reviewed academic literature, and are not just hiding behind the shine of one scientific article they've found.
As the ease of self-publishing online increases, things are becoming a little messier with regard to identifying reputable peer-reviewed sources. Academics sometimes share unpublished work that is not yet peer-reviewed (these often come in the form of preprints). The current ease of sharing work provides a great tool for sharing current and ongoing work among academics, but can sometimes be confusing to journalists and the general public. As a shortcut, if it comes in a Word document or looks like a PDF version of a Word document, it's not published (or, alternatively, it's an alternate form of the work that's been published elsewhere). In any case, if it looks like a preliminary draft it may well be, and this should noted prominently—generally on the front page—by the authors. Check whether the authors have indicated somewhere on the first few pages whether it's unpublished work or a pre-print of soon to be published work. If the authors are being transparent, that information should be there. If you can't find that information, keep your skepticism high.
It's also noteworthy that if you find one or two sources linked in a piece that aren't high quality, this doesn't necessarily make the work problematic. It's all about context—consider what the author is using them for. The author might primarily base their work on the science literature, but also use another kind of source for an anecdote or as a basis for comment on current events.
Furthermore, much that you might read online, however, won't be reporting on science. It might, perhaps, be reporting on or offering opinions about current events. These kinds of pieces might depend primarily on other web sources like blogs, podcasts, news articles, things people have said on Twitter, or interviews they conducted. This makes sense—for current events there isn't an academic literature to back up the claims.
There are a lot of grey areas in opinion pieces and news stories where authors might feel they have licence to assume things they shouldn't. For example, an article may be a critique of a current trend observed on social media. In such a case, popular media articles, blogs, and other posts would likely be the primary sources for anecdotes. The problem is that if we really want evidence for the social trend, we need more than just anecdotes—we need data and competent analysis of that data by social scientists. And that's where reputable peer-reviewed sources are helpful. Without evidence in the data, the author may be assuming a trend where there is none, based on a few extreme or confirmatory cases (i.e., cases that support the authors pre-existing hypothesis).
Articles that use individuals' opinions as sources of support for a belief or a position or action that ought to be taken should raise concerns. Quoting people from the general public about their views on policy is neither a good gauge of public opinion nor a good basis for an argument in favour of the policy (visit fallacies in Arguments & Truth Claims section, and in particular consider argument from anecdote). If quotes from non-experts are the basis for an argument or claim, the article might not be a trustworthy source of information.
Interviews with experts are generally better than interviews with the general public, but also need further investigation as they may be based largely on opinion, research that's in it's early stages, or research that is complete but has never been peer-reviewed and published.
In summary, take the citation of unpublished or non-peer-reviewed work with a grain of salt, and think hard about whether a popular press article that is basing its claims on blog or social media posts is worth your time.
Learning Check