Liz Neeley and I discussed this in our 2014 paper.

In terms of social media outreach, or outreach in general, the impact on a scientist’s career remains largely unquantified and quite possibly indirect. “Many faculty members identified their primary job responsibilities as research and post-secondary teaching. They felt that outreach participation hindered their ability to fulfill those responsibilities and might be an ineffective use of their skills and time, and that it was not a valid use of their research funding”. In the survey by Ecklund et al., 31% of scientists felt that research university systems value research productivity, as indexed by grants and published papers, over everything else, including outreach. With this prioritization structure in place outreach may be perceived as unrelated to a scientist’s academic pursuits.

Perhaps because of both the ease of quantification and the impact is hypothesized to be direct, one specific question continues to generate considerable attention.   If a paper receives a significant number of social media mentions does it also receive a significant number of citations? If this correlation exist then this would support an argument that Tweeting, Facebooking, etc. about your scientific papers. This science communication would increase the exposure of your paper, including to scientists, eventually leading to more citations of that paper.  In this were true the impact of science communication would be direct and impact a metric that is used to evaluate scientists.

One of the largest studies on this topic, in analysis of 1.4 million documents published in PubMed and Web of Science published from 2010 to 2012, Haustein et al. found no correlation between a paper or a journals citation count and Twitter mentions.  However, multiple studies since do find a link between Tweets and citations rates including the papers of Peoples et al. and de Winter.

A new paper by Finch et al.  finds a link between social media mentions an citations also exists in the orthinology literature.  The authors set up the question nicely in the introduction

Weak positive correlations between social media mentions and future citations [5,8–10] suggest that online activity may anticipate or drive the traditional measure of scholarly ‘impact’. Online activity also promotes engagement with academic research, scholarly or otherwise, increasing article views and PDF downloads of PLoS ONE articles, for example [11,12]. Thus, altmetrics, and the online activity they represent, have the potential to complement, pre-empt and boost future citation rates, and are increasingly used by institutions and funders to measure the attention garnered by the research they support [13].

The findings? For a subset of 878 articles published in 2014, the group found that an increase in social media mentions, as indexed by the Altmetric Score, from 1 to 20 resulted in 112% increase in citation count from 2.6 to 5.5 citations per article.

So drop what you’re doing and start Tweeting about your most recent paper RIGHT NOW! 

But wait…

All of these studies show a correlation and not causation.  Simply put, scientific papers with a lot of social media mentions also have lots of citations.  One hypothesis would be that communicating your science broadly increases its exposure and increases the probability of citation.  And it appears that often those advocating for science communication repeat this narrative despite there currently being no support for this hypothesis.  Why is there no support?

Because the correlation between social media mentions and citations could be equally explained by other hypotheses.

So an equally likely explanation for this correlation is that papers that are popular garner both numerous social media mentions and eventually numerous citations.

Tom Webb also makes an outstanding point about the authors of such studies.

The authors of this most recent study note this overall causation and correlation dilemma as well.

Instead, our results suggest that altmetrics might provide an initial and immediate indicator of a research article’s future scholarly impact, particularly for articles published in more specialist journals…The correlative nature of this and other studies makes it difficult to establish any causal relationship between online activity and future citations

So what now?  First stop arguing, as many did on Twitter today (examples here), that Tweeting about your paper is good thing because it will ultimately generate more citations.  The jury is still out on this and will be until a study specifically is designed to test for the causation and not the correlation.

The post Will Tweeting About Your Research Paper Get You More Citations? Meh. first appeared on Deep Sea News.

 “Impacts of biodiversity loss on ecosystem services” by Boris Worm and others [7]…demonstrated through a large body of evidence that biodiversity loss greatly reduces the ecosystem services that we obtain from the oceans, and also contained an analysis projecting “the global collapse of all taxa currently fished by the mid–21st century (based on the extrapolation of regression in Fig. 3A to 100% in the year 2048)”. This projection of global seafood collapse by 2048 was highlighted in their associated press release [8]. The press release resulted in prominent coverage of the projection in major news outlets, and provoked a reaction from some fisheries researchers [8] and 10 rebuttals [9]–[18], including three rebuttals by Ray Hilborn [who argued fisheries papers published in Science and Nature were selected for their publicity value, despite being scientifically flawed]  [10]–[12].

Despite the controversy, or perhaps because of it, Worm et al. [7] is a highly cited paper (799 citations at 20 January 2013 in the Web of Science), and was among the top 10 most cited fisheries papers in 2012 [21]. Past citation analysis shows that the rebuttals to the 2048 projection in Worm et al. [7] had virtually no effect on citation rates or the impressions that people gained about this paper [22]. (Branch 2013)

Trevor Branch, having long been captivated by fisheries research and having deep knowledge of this Worm et al. controversy, sat back in his chair, pented his fingers, and had an epiphany: “I’ve got it! We should analyze citation patterns for Worm et al. (2006)!” (a written dramatization of possibly real events). So he did. And the results are pretty awesome.

Sidenote: some of his methods were pretty badass.

Where gender [of a paper author] was not obvious it was inferred from the website Baby Name Guesser (http://www.gpeters.com/names/baby-names.​php) which returns the ratio of male to female babies born with a particular name.

One of Branch’s main questions was, “Did the framing of the press release result in a greater frequency of citations mentioning the 2048 projection [mentioned in Worm et al 2006]?” – the answer was a resounding YES.

Thus, there is some evidence that increasing the prominence of the 2048 projection in the press release resulted in citing papers referring to this projection more often than expected.

However, Branch found that scientists who had a deeper understanding of the overall fisheries controversy were less likely to fall prey to the media hype:

Papers that mentioned the 2048 projection had characteristics that suggest unfamiliarity with the controversy surrounding this projection, namely papers with few authors, published in journals with low impact factors, in fields far removed from ecology and fisheries, and sharing no coauthors with the Worm et al. paper. For such papers, it is easy to see how the authors, editors, and reviewers could be unaware of the controversy over the 2048 projection.

These factors suggest that the more knowledgeable the authors of citing papers were about the controversy over the 2048 projection, the less likely they were to refer to it. A noteworthy finding was that if the original authors were also involved in the citing papers, they rarely (1 of 55 papers, 2%) mentioned the 2048 projection. Thus the original authors have emphasized the broader concerns about biodiversity loss, rather than the 2048 projection, as the key result of their study.

You can see these citation trends in the lovely Figure 2:

So that leave me asking…are us scientists just as prone to believe soundbites promoted by press releases and the media, particularly for hyped-up papers outside our own discipline? In the case of Worm et al. (2006), the message that “all fisheries will collapse by 2048” certainly seemed to spread like an urban legend in the peer-reviewed literature.

Reference: Branch TA (2013) Citation Patterns of a Controversial and High-Impact Paper: Worm et al. (2006) “Impacts of Biodiversity Loss on Ocean Ecosystem Services”. PLoS ONE 8(2): e56723. doi:10.1371/journal.pone.0056723

The post Media hype gets you more citations? Well, it did for this fisheries paper. first appeared on Deep Sea News.