“It is so hard to reject an opinion shared by our peers that too often we don’t even try to evaluate claims based on their merits. We let our group do our thinking for us.”
– Sloman & Fernbach (2017), The Knowledge Illusion: Why We Never Think Alone
Most of us are aware that social media is riddled with nonsense from charlatans, trolls, bots, fake news sites, and people who simply don’t know any better. What we often don’t realize, though, is that being on the lookout for and disregarding such sources isn’t a sufficient guard against misinformation. The sources we find trustworthy, too—even those we might have good reason to trust, like known experts and professional organizations—may also sometimes share misleading or false content. The pitfalls of social media as an information source are such that we ought to maintain a critical eye even toward the contributions of people and institutions we hold in high regard.
This post aims to illustrate the importance of being attuned to misinformation even from sources one might generally trust. I delve into the example of a recent inaccuracy shared on Twitter by the Society for Personality and Social Psychology (SPSP).
Photo by Brett Jordan on Unsplash
SPSP's website says it is “the world’s largest organization of social and personality psychologists. With over 7,500 members, SPSP strives to advance the science, teaching, and application of social and personality psychology.” As a social psychologist, I’ve been a member of SPSP in the past and, along with almost 17,000 others, currently follow @SPSPnews on Twitter for notifications about recent psychology findings. In short, SPSP is a valued organization for a good-sized group of academics (and others who might elect to follow on social media).
On October 29, 2021, @SPSPnews tweeted the following:
This tweet tells us of research finding that people in more individualistic countries (places like Canada, the U.S., & Netherlands where individual people’s goals & needs tend to be prioritized) are less interested in getting vaccinated against Covid-19 compared to those in more collectivistic countries (e.g., Nepal, Pakistan, & Columbia where group goals & the greater good are emphasized). The heading in the link makes a more causal—if somewhat vague—claim, suggesting that individualism "is responsible for spreading Covid-19" (rather than being merely correlated with interest in getting vaccinated).
There are enough clues—or, or let's just say, red flags—in this tweet to generate doubt and suggest that more reading is required before we take this claim seriously. First, a lofty causal claim concerning recent, and probably unreplicated, social science research should pique our skepticism (e.g., the social world is hard stuff to study well). In any case, given that country of residence can't feasibly be manipulated in an experiment (an experiment is the go-to method for examining causation), it’s unlikely that this research even deals with data that allows for causal inferences.
Second, there’s a disjoint in the correlational language of the quote and the causal language in the text accompanying the link. Something's almost certainly amiss there.
Third, there are clues that the quote in the body of the tweet oversimplifies things. In the link’s text, there’s reference to another variable, “feelings of personal invincibility from Covid-19”. How does that fit into things? The tweet doesn't seem to be telling us the whole story.
Finally, the linked article is found at Inverse, which is not a peer-reviewed academic journal. That is, the quote about the research appears to come from a secondary source rather than the peer-reviewed research article where, presumably, the research has been published. Thus, there’s an added link in the chain of communication and, therefore, more room for error. And more reason to be skeptical.
Given this litany of concerns, I wondered whether there was, in fact, any evidence backing up this tweet’s content.
What does the research really find about vaccine intentions?
In drawing from the secondary source—an article by Nick Keppler at Inverse—instead of the original peer-reviewed research article, the tweet ends up seriously miscommunicating the research. The original work by James Leonhardt and colleagues (published at PLOS ONE) does not, in fact, hypothesize or find evidence that people in more individualistic countries are less interested in getting vaccinated against Covid-19 than people in collectivistic countries.
The actual hypotheses and findings diverge quite a bit and are a bit more complex, involving an interaction between country-level individualism-collectivism and people’s perceptions of how serious it would be for them to get Covid-19. The authors state two hypotheses regarding people's interest in getting vaccinated (there are further hypotheses, but these are the one's pertaining to vaccination):
First, the authors posit that perceived invincibility (operationalized as viewing hypothetical personal infection with Covid-19 as less serious) will be negatively related to willingness to be vaccinated. That is, people who see infection as less serious will also be less willing to get a vaccine.
Second, country-level individualism-collectivism will moderate the effect of perceived invincibility on willingness to be vaccinated, such that the negative relationship between perceived invincibility and willingness will be stronger in more individualistic countries (vs. collectivistic countries).
Note two important things distinguishing the above hypotheses from the information offered in SPSP's tweet (including the shared link). First, there are no causal claims (these data don’t allow for causal inferences!). Second, there’s no hypothesis predicting a direct effect of individualism-collectivism on interest in getting vaccinated.
So, what does the research find? First, and in line with the first hypothesis above, there was an effect of the perceived invincibility measure—greater impervious to Covid-19 related to lower willingness to be vaccinated. Of critical relevance to the SPSP tweet, the researchers found no effect of country-level individualism-collectivism on willingness to be vaccinated. That's a far cry from the quote shared by SPSP that “people living in places that value individualism also had diminished interest in getting vaccinated”.
Where individualism-collectivism comes into play is that it moderated the effect of perceived invincibility. “While invincibility has an overall negative effect on vaccination intention, the effect is particularly pronounced among participants in cultures with low cultural collectivism” (Leonhardt et al., 2021).
(Note: on close inspection, one would find that the main effect for individualism-collectivism on willingness to get vaccinated is close to the .05 significance cutoff (p = .06). For several reasons, that can't be used as evidence. First, scientists using null hypothesis significance testing agree, for better or worse, on the .05 cut-off for significance—finding anything above it means that the null hypothesis should be retained (i.e., no effect). Second, the authors don't hypothesize an effect of country-level individualism-collectivism on desire to get inoculated (they hypothesize individualism-collectivism solely as a moderator). That’s critical—the p-value is mostly meaningless in the absence of a relevant hypothesis. Third, observed main effects are qualified by the interaction between individualism-collectivism and perceived invincibility. The interaction renders any effect of individualism-collectivism at best less meaningful and at worst uninterpretable. It’s important to remember that the authors of the original paper mostly steer clear of discussing the effect of country level individualism-collectivism)
In short, contrary to Inverse article (and the tweet), the cited research simply doesn’t find evidence that people in more individualistic countries are less interested in getting a Covid-19 vaccine.
The title of the Inverse article, “One quintessentially American Trait is Responsible for Spreading Covid-19 – Study”, is therefore quite misleading. Going further than suggesting a relationship between individualism-collectivism and interest in vaccination, it claims that individualism causes the spread of Covid-19. Again, there's no evidence in this research of a relationship—let alone a causal relationship!—between individualism and spread of Covid-19. There is, in fact, nothing in the research that suggests individualism is causing greater spread of the disease (only data on people’s self-reported willingness to get vaccinated).
Of note, although further discussion of the research’s shortcomings is beyond the scope of this post, I do have some reservations about the research article, too, including the operationalization of the key individual-level independent variable (perceived invincibility), some causal inferences where I don't think they belong, and what I think are some hasty and loose recommendations for public policy communications. In any case, it does bear mentioning that there’s room for significant critique regarding each bit of media at play: the tweet (@SPSPnews), the secondary source (article at Inverse), and the original peer-reviewed article (Leonhardt et al.).
It’s clear that SPSP should have gone to the primary source—the research article—rather than quote the article at Inverse. While SPSP, or rather the person responsible for tweeting for the organization, makes a fairly large error here, part of the issue is that Twitter incentivizing speed, simplicity, and provocative ideas. That is, the medium influences the message. Nevertheless, it seems clear that those behind SPSP’s Twitter account took the easy route this time, sharing an inaccurate interpretation of the research as reported by a secondary source rather than carefully reviewing and summarizing the true findings conveyed in the original research article.
What should we do with this?
The main point of this post is not to criticize SPSP (though, as seen above, I think there's a legitimate criticism to be made), but rather to show that, when on social media, we need to be extra careful with the information that comes our way—even when it comes from widely trusted sources.
There are many things we can do to help us determine whether we should alter our beliefs based on, or share with others, a social media post—some are quick shortcuts and others require some careful work. Here are just a few select things to think about when using social media as an information source:
We should be reasonably skeptical about the information we encounter on social media, regardless of whether we trust the source. It may not be a bad idea to start with the baseline assumption that there’s a good chance posts contain inaccuracies. The nature of social media can lead users to miscommunicate—limited word counts can result in oversimplification, quick posting can cause errors, and desire to get clicks and likes can lead to unjustifiably provocative or negative posts. While we can be more confident in trustworthy sources, on social media even these generally need to be fact checked before our beliefs should be moved. If you don’t have time to fact-check or don't have good reason to be confident in the content, take every post with a grain of salt and don’t share.
A good overarching approach—albeit a difficult one to know we're applying effectively—might be to adopt an accuracy motivation, the drive to get things right, rather than other motivations that often guide our online behaviour, such as desire to impress others or to find evidence that supports one’s beliefs. In practice, that means before believing or retweeting a claim try to figure out whether there’s good reason to accept it. If you’re invested in accuracy, you’re probably going to be less likely to believe and retweet misinformation.
There are digital tools to keep on hand to inform how much one ought to trust a source. One example is Media Bias/Fact Check, which checks the extent of factual reporting and political bias for various media sources. According to Media Bias/Fact Check, Inverse (where the article quoted by @SPSPnews is found), offers a high degree of factual reporting, but it has: “slight to moderate liberal bias. They often publish factual information that utilizes loaded words (wording that attempts to influence an audience by using appeal to emotion or stereotypes) to favor liberal causes. These sources are generally trustworthy for information, but may require further investigation.” There are clues there that, while we need not dismiss content from the site, articles can sometimes be misleading (as we found above).
Continually update your trust of the account or account holder. If you determine that the information in a social media post is somewhat misleading, consider the degree to which you should change your opinion of not just the post, but the account holder. Remember, though, that when tweets are associated with an organization, they may not necessarily reflect the normative views of that body. In fact, it’s entirely possible that a single person or small group of people within the organization is responsible for the post. That person or group may not have expertise on the specific topic, may not fully represent the views of the organization, and may be prone to all the mental barriers we see among personal accounts on social media. Recurrent inaccuracies, then, should move our views on the account to a greater extent than views of the overall organization.
Be wary of quotes. Quotes in tweets could suggest—but aren’t diagnostic of—minimal thought/reading by the person making the post or that they’re not willing to take responsibility for the post. If there's a quote, dig into its source.
Go all the way to the primary source if what you want to do is get the most accurate reading. Things can get confusing when there’s a trail of even just three bits of media, as in our example above—there may be problems with the social media post, the secondary source, and/or the primary source. To be efficient, once you have the original source, put the tweet and the secondary source aside.
Comments