In our global survey on innovations in scholarly communication, we asked researchers (and people supporting researchers, such as librarians and publishers) what tools they use (or, in the case of people supporting researchers, what tools they advise) for a large number of activities across the research cycle. The results of over 20,000 respondents, publicly available for anyone analyze, can give detailed information on tool usage for specific activities, and on what tools are preferentially used together in the research workflow. It’s also possible to look at results for different disciplines, research roles, career stages and countries specifically.
But we don’t even have to dive into the data at the level of individual tools to see interesting patterns. Focusing on the number of people that answered specific questions, and on the number of tools people indicate they use (regardless of which tools that are) already reveals a lot about research practices in different subsets of our (largely self-selected) sample population.
On methodologyFor each question on tool usage, we offered seven preselected choices that could be clicked (multiple answers allowed), and an ‘and also others’ answer option that, when clicked, invited people to manually enter any other tools they might use for that specific research activity (see Figure 1).
Figure 1 – Example question with answer options
We did not include a ‘none’ option, but at the beginning of the survey stated that people were free to skip any question they felt did not apply to them or did not want to answer. Nonetheless, many people still answered ‘none’ (or any variation thereof) as their ‘other’ options.
Since methodologically, we cannot make a distinction between people who skipped a question or who answered ‘none’, we removed all ‘none’ answers from the survey result. We also adjusted the number of respondents that clicked the ‘and also other’ option to only reflect those that indicated they used at least one tool for the specific research activity (excluding all ‘nones’ and empty answers).
Figure 2 – Response rate per survey question (researchers and librarians)
What simple question response levels already can tell us
The differences in response levels to the various questions are quite marked, ranging from barely 15% to almost 100%. It is likely that two effects are at play here. First, some activities are relevant for all respondents, e.g. writing and searching information, while others like sharing (lab) notebooks are specific to certain fields, explaining lower response levels. Second, there are some activities that are not yet carried out by many or for which respondents choose not to use any tool. This may be the case with sharing posters and presentations and with peer review outside that organized by journals.
Then there are also notable differences between researchers and librarians. Researchers expectedly more often indicate tool usage to publish and librarians are a bit more active in using or advocating tools for reference management. Perhaps more interestingly, it is clear that librarians are “pushing” tools that support sharing and openness of research.
The differences in response levels to the various questions are quite marked, ranging from barely 15% to almost 100%. Some activities seem relevant for all respondents, e.g. writing and searching information. Others, like sharing (lab) notebooks, sharing posters and presentations and peer review outside that organized by journals, may be carried out by . fewer people, or may be done without using specific tools.
Then there are also notable differences between researchers and librarians. As expected, researchers more often indicate tool usage to publish and librarians are a bit more active in using or advocating tools for reference management and selection of journals to publish in. Perhaps more interestingly, it is clear that librarians are “pushing” tools that support sharing and openness of research.
Disciplinary differences
When we look at not just the overall number of respondents per activity, but break that number down for the various disciplines covered (Figure 3), more patterns emerge. Some of these are expected, some more surprising.
Figure 3 – Response rate per survey question across disciplines (researchers only)
As expected, almost all respondents, irrespective of discipline, indicate using tools for searching, getting access, writing and reading. Researchers in Arts & Humanities and Law report lower usage of analysis tools than those in other disciplines, and sharing data & code is predominantly done in Engineering & Technology (including computer sciences). The fact that Arts & Humanities and Law also score lower on tool usage for journal selection, publishing and measuring impact than other disciplines might be due to a combination of publication culture and the (related) fact that available tools for these activities are predominantly aimed at journal articles, not books.
Among the more surprising results are perhaps the lower scores for reference management for Arts & Humanities and Law (again, this could be partly due to publication culture, but most reference management systems enable citation of books as well as journals). Scores for sharing notebooks and protocols were low overall, where we would have expected this activity to occur somewhat more often in the sciences (perhaps especially life sciences). Researchers in Social Sciences & Economics and in Arts & Humanities relatively often use tools to archive & share posters and presentations and to do outreach (phrased in the survey as: “tell about your research outside academia”), and interestingly enough, so do researchers in Engineering & Technology (including computer science). Finally, peer review outside that done by journals is most often done in Medicine, which is perhaps not that surprising given that many developments in open peer review are pioneered in biomedical journals such as The BMJ and BioMedCentral journals.
How many apps do you have on your smartphone*? On your tablet? Do you expect the number of online tools you use in your daily life as a researcher to be more or less than that?
Looking at the total number of tools that respondents to our survey indicate they use in their research workflow (including any ‘other’ tools mentioned, but excluding all ‘nones’, as described above), it turns out the average number of tools reported per person is 22 (Figure 4). The frequency distribution curve is somewhat skewed as there is a longer tail of people using higher numbers of tools (mean = 22.3; median = 21.0).
Figure 4 – Frequency distribution of total number of tools used per person (20,663 respondents)
We also wondered whether the number of tools a researcher uses varies with career stage (e.g. do early career researchers use more tools than senior professors?).
Figure 5 shows the mean values of the number of tools mentioned by researchers, broken down by career stage. We used year of first publication as a proxy for career stage, as it is a more or less objective measure across research cultures and countries, and less likely to invoke ‘refuse to answer’ then asking for age might have been.
Figure 5 – Number of tools used across career stage (using year of first publication as proxy)
There is an increase in the number of tools used going from early to mid career stages; peaking for researchers who published their first paper 10-15 years ago. Conversely, more senior researchers seem to use less tools, with the number of tools decreasing most for researchers who first published over 25 years ago. The differences are fairly small, however, and it remains to be seen whether they can be proven to be significant. There also might be differences across disciplines in these observed trends, depending on publication culture within disciplines. We have not further explored this yet.
It will be interesting to correlate career stage not only with the number of tools used, but also with type of tools: do more senior researchers use more traditional tools that they have been accustomed to using throughout their career, while younger researchers gravitate more to innovative or experimental tools that have only recently become available? By combining results from our survey with information collected in our database of tools, these are the type of questions that can be explored.
Innovations in Scholarly Communication 