Dennis Donathan, a Research Assistant and Data Scientist for the Public Knowledge Project, discusses a recent study he conducted on Reddit users’ engagement with research outputs. His research showed that although Reddit is not typically the focus of altmetrics research, the platform can act as an informative and valuable tool in altmetrics research. In addition, his research demonstrated the platform’s value as a tool to assess public engagement with research and communicate research in a more meaningful and contextual way.
I became interested in Reddit as an import tool for scientific communication thanks to my advisor and collaborator Timothy Bowman, while finishing my Master of Library and Information Sciences degree. We were interested in looking at engagement with academic-objects, such as research outputs and researchers themselves, and the nature of those engagements on Reddit.
First, let’s get some background on Reddit through an academic engagement lens. While combing through the literature I found it lacking regarding Reddit; some studies would mention it for comparative purposes when analyzing some facet of Altmetric.com data, and their results always showed Reddit to be underwhelming. Only a few studies looked at Reddit and its relation to academic-objects. Specifically, one article by Hara et al. (2019) found that researchers who went on Reddit to host Ask Me Anything threads found the interactions to be overwhelmingly positive; this strong reaction provoked us to probe further into studying whether “Reddit can really deliver when it comes to impactful interactions with academic-objects.”
One of the largest ideological drivers behind our research was the 2019 paper by Diaz-Faes et al., The Second Generation of Altmetrics. This paper represents an ideological shift in altmetrics away from simply counting events (e.g. counting the number of times a link to a paper is tweeted) to focusing on a broader range of activities that happen on social media. Juan Pablo Alperin suggested that we should be “looking at the social aspect of social media,” (emphasis mine) and that we should seek ways to engage directly with communities of interest regarding our research (2019). Therefore, in our study we wanted to explore beyond the characteristics of the academic-objects shared, and instead focus on the number of engagements and the nature of those engagements, such as the sentiment expressed in a post or comment with a link to an academic-object.
Reddit can really deliver when it comes to impactful interactions with academic-objects.
To achieve this we looked at Reddit boards or subforms called subreddits (typically stylized /r/subreddit_name_here), that are typically focused around a singular topic. They can be broad like /r/science or niche in the case of /r/whatsInYourCart. To make the case that Reddit currently occupies a promising space in science communication, we looked at a subreddit that is not directly related to science or a scientific discipline, /r/australia. Throughout our research we found several interesting and promising elements of engagement on Reddit; we focused on the length of comments and how long-form responses differ from shorter microblogging style responses. Short-form responses, such as those on microblogging platforms like Twitter, tend to express more neutral sentiment, likely due to the character limit. For example, previous research has shown that tweets to academic-objects often act as publication alert services for academics rather than meaningful engagement with the academic-objects on Twitter (Tenopir et al., 2013).
To understand why we choose /r/australia, Table 1 shows the top 10 subreddits by number of Altmetric events, which in the case of Reddit are posts that contain a direct link to an academic output (Altmetric.com, 2019).
Subreddit | Members | Altmetric Events |
science | 22394701 | 23626 |
todayilearned | 3116920 | 5262 |
drugnerds | 81826 | 3322 |
nootropics | 17478 | 3295 |
machinelearning | 14511 | 2842 |
psychology | 527900 | 1887 |
futurology | 13968603 | 1812 |
australia | 40465 | 1726 |
environment | 95778 | 1651 |
physics | 68663 | 1510 |
The Australia subreddit does not share many commonalities with the other subreddits found in Table 1, specifically we can categorize subreddits into three distinct groups: academic-focused (e.g. science), academic-adjacent (e.g. drugnerds), and non-academic (e.g. todayilearned, australia). The subreddit /r/todayilearned, as the name suggests, is a place where users can post information that they have recently learned, which lends itself to having many academic outputs directly linked there. Exploring the Australia subreddit provides a unique opportunity to see how a community that is not directly related to an academic discipline interacts with academic outputs. To get at the soul of this venture, I pulled only the first-level comments (n=8628) to /r/australia posts marked as “Altmetric events” (n=1726), or those posts that mention and link to academic outputs; replies to other users’ comments (i.e., replies to replies) were not included in the data set for analysis.The comments were categorized into two groups: those with more than 280 characters, and those with 280 or fewer characters. This grouping is based on the 280 character limit for tweets. Figure 1 displays the percentage of comments within each comment group.
Nuanced topics and ideas can be more fully expressed without the concern of the message being “lost in translation.”
A considerable number of comments exceed the 280 character limit (29.36%), opening a window into potential insights, interactions, and lively discourse on how the public (and academics (see Section ‘3.6 Who Tweets Scholarly Output?’ of the linked book chapter for context)) socially discuss academic topics. Nuanced topics and ideas can be more fully expressed without the concern of the message being “lost in translation.” It’s important to note that there is a sizable quantity of comments utilizing longer responses. These are comments that, should they have been made on a microblogging platform like Twitter, would have either been severely condensed or broken into multiple responses. On Reddit, users can submit a singular long-form response to articulate their response to the fullest degree. Additionally, we observed that these long-form comments receive a higher submission score on average, Reddit’s crowd-sourced rating system for scoring comments and posts via upvotes and downvotes (see Figure 2).
Using the Valence Aware Dictionary for Sentiment Reasoning (VADER), I ran a sentiment analysis on the comments. Perhaps the most useful finding is that the comments that exceed 280 characters are less likely to express neutral sentiments (see Figure 3). While this isn’t surprising, the ability to use more characters and invoke greater articulation allows for more freedom of expression, resulting in a wider range of sentiments, whereas using fewer characters tends to force a more neutral tone when discussing academic-objects. The comments on Reddit reflect this: longer comments are found more frequently at the extremes of the sentiment spectrum, whereas shorter comments are found more frequently towards the middle.
Reddit represents a unique opportunity for us to evolve our social media presence. While microblogging sites have proven to be very useful for academics, it seems that we have also grown to understand their limitations. Simultaneously, the priorities in research assessment and evaluation are also changing. A focus on “societal impact,” an almost dizzyingly vague term, has been steadily growing. There is no singular metric that can capture societal impact, so the best recourse is to use multiple tools. Microblogging sites have value and do a wonderful job for what they are suited for, but as we push toward having more meaningful conversations and engagements in online spaces about research and scholarship, Reddit presents itself as a valuable tool for evaluating and demonstrating meaningful public engagement with academic-objects. Our research shows that Reddit users eagerly and willingly engage with academic-objects, and we can see areas in which they provide distinctly different expressions of sentiment. This study provides a small window into the value of Reddit as a viable data source for assessing public and societal engagement with research and scholarship. As always, further research is needed.
So why not poke around, search for posts or subreddits related to your discipline or even your own published academic-objects, and see what is being said. For example, if you do not have access to the subscription Altmetric Explorer database, you can use the free Altmetric Bookmarklet to find attention to your publications and perhaps find specific attention from Reddit. Maybe even look at your regional-based subreddits; you never know when a topic related to your discipline will pop up and an opportunity to share in a meaningful nuanced discussion may arise.
Dennis Donathan II is an MLIS graduate from Wayne State University currently working for the Public Knowledge Project as a Research Assistant/Data Scientist. Dennis is primarily interested in scientometrics and altmetrics as it relates to open science, science communication, and community engagement.
Unless it states other wise, the content of the Bibliomagician is licensed under a Creative Commons Attribution 4.0 International License.