Towards a Plan S gap analysis? (2) Gold open access journals in WoS and DOAJ

(NB this post is accompanied by a another post, on open access potential across disciplines, in the light of Plan S)

In our previous blogpost, we explored open access (OA)  potential (in terms of journals and publications) across disciplines, with an eye towards Plan S. For that exercise, we looked at a particular subset of journals, namely those included in Web of Science. We fully acknowledge this practical decision leads to limitation and bias in the results. In particular this concerns a bias against:

• recently launched journals
• non-traditional journal types
• smaller journals not (yet) meeting the technical requirements of WoS
• journals in languages other than English
• journals from non-Western regions

To further explore this bias, and give context to the interpretation of results derived from looking at full gold OA journals in Web of Science only, we analyzed the inclusion of DOAJ journals in WoS per major discipline.

We also looked at the proportion of DOAJ journals (and articles/reviews therein) in different parts of the Web of Science Core Collection that we used: either in the Science Citation Index Expanded (SCIE) / Social Sciences Citation Index (SSCI) /Arts & Humanities Citation Index (AHCI), or in the Emerging Sources Citation Index (ESCI).

The Emerging Sources Citation Index contains a range of journals not (yet) indexed in the other citation indexes, including journals in emerging scientific fields and regional journals. It uses the same quality criteria for inclusion as the other citation indexes, notably: journals should be peer reviewed, follow ethical publishing practices, meet Web of Science’s technical requirements, and have English language bibliographic information. Journals also have to publish actively with current issues and articles posted regularly. Citation impact and a strict publication schedule is not a criterion for inclusion of journals in ESCI, which means that also newer journals can be part of ESCI. Journals in ESCI and the AHCI do not have a Clarivate impact factor.

Method
We compared the number of DOAJ journals in Web of Science to the total number of journals in DOAJ per discipline. For this, we made a mapping  of the LCC-classification used in DOAJ to the major disciplines used in Web of Science, combining Physical Sciences and Technology into one to get four major disciplines.

For a number of (sub)disciplines, we identified the number of full gold journals in Web of Science Core Collection, as well as the number of publications from 2017 (articles & reviews) in those journals. We also looked what proportion of these journals (and the publications therein) are listed in ESCI as opposed to SCIE/SSCI/AHCI. For subdisciplines in Web of Science, we identified 10 research areas in each major discipline with the highest number of articles & reviews in 2017. Web of Science makes use of data from Unpaywall for OA classification at article-level.

All data underlying this analysis are available on Zenodo: https://doi.org/10.5281/zenodo.1979937

Results

Looking at the total number of journals in DOAJ and the proportion thereof included in Web of Science (Fig 1, Table 1) shows that Web of Science covers only 32% of journals in DOAJ, and 66% of those are covered in ESCI. For Social Sciences and Humanities, the proportion of DOAJ journals included in WoS is only 20%, and >80% of these journals are covered in ESCI, not SSCI/AHCI. This means that only looking at WoS leaves out 60-80% of DOAJ journals (depending on discipline), and only looking at the ‘traditional’ citation indexes SCIE/SSCI/AHCI restricts this even further.

Fig 1. Coverage of DOAJ journals in WoS

Table 1. Coverage of DOAJ journals in WoS (percentages)

We then compared the the proportion of DOAJ journals covered in SCIE/SSCI/AHCI versus ESCI, to the proportion of publications in those journals in the two sets of citation indexes (Fig 3). This reveals that for Physical Sciences & Technology and for Life Sciences & Medicine, the majority of full gold OA articles in WoS is published in journals included in SCIE, indicating that journals in ESCI might predominantly be smaller, lower volume journals. For Social Sciences and for Humanities, however, journals in ESCI account for the majority of gold OA articles in WoS. This means that due to WoS indexing practices, a large proportion of gold OA articles in these disciplines is excluded when considering only what’s covered in SSCI and AHCI.

Fig 2. Gold OA journals and publications in WoS

The overall patterns observed for the major disciplines can be explored more in detail when looking at subdisciplines (Fig 3). Here, some interesting differences between subdisciplines within a major discipline emerge.

• In Physical Sciences and Technology, three subdisciplines (Engineering, Mathematics and Computer Sciences) have a large proportion of full OA journals that is covered in ESCI rather than SCIE, and especially for Engineering, these account for a sizeable part of full gold OA articles in that subdiscipline.
• In Life Sciences and Biomedicine,  General and Internal Medicine seems to be an exception with both the largest proportion  of full OA journals in ESCI as well as the largest share of full gold OA publications coming from these journals. In contrast, in Cell Biology, virtually all full gold OA publications are from journals included in SCIE.
• In Social Sciences, only in Psychology a majority of full gold OA publications is in journals covered in SSCI, even though for this discipline, as for all other in Social Sciences, the large majority of full gold OA journals is part of ESCI, not SSCI.
• In Arts & Humanities the pattern seems to be consistent across subdisciplines, perhaps with the exception of Religion, which seems to have a relatively large proportion of articles in AHCI journals, and Architecture, where virtually all journals (and thus, publications) are in ESCI, not AHCI.

Fig 3. Full gold OA journals and publications in Web of Science, per subdiscipline

Looking beyond traditional citation indexes

Our results clearly show that in all disciplines, the traditional citation indexes in WoS (SCIE, SSCI and AHCI) cover only a minority of existing full gold OA journals. Looking at publication behaviour, journals included in ESCI account for a large number of gold OA publications in many (sub)disciplines, especially in Social Sciences and Humanities. Especially in terms of an analysis of availability of full OA publication venues in the context of Plan S, it will be interesting to look closer at titles included in both SCIE/SSCI/AHCI and  ESCI per (sub)discipline and assess the relevance of these titles to different groups of researchers within that discipline (for instance by looking at publication volume, language, content from cOAlitions S or EU countries, readership/citations from cOAlition S or EU countries). Looking at publication venues beyond traditional citation indexes fits well with the ambition of Plan S funders to move away from evaluation based on journal prestige as measured by impact factors. It should also be kept in mind that ESCI marks but a small extension of coverage of full gold OA journals, compared to the large part of DOAJ journals that are not covered by WoS at all.

Encore: Plan S criteria for gold OA journals

So far, we have looked at coverage of all DOAJ journals, irrespective of whether they meet specific criteria of Plan S for publication in full OA journals and platforms, including copyright retention and CC-BY license*.

Analyzing data available through DOAJ (supplemented with our mapping to WoS major disciplines) shows that currently, 28% of DOAJ journals complies with these two criteria (Fig 4). That proportion is somewhat higher for Physical Sciences & Technology and Life Sciences & Medicine, and lower for Social Sciences & Humanities. It should be noted that when a journal allows multiple licenses (e.g. CC-BY and CC-BY-NC-ND), DOAJ includes only the most strict license in its journal list download. Therefore, the percentages shown here for compliant licensing are likely an underestimation. Furthermore, we want to emphasize that this analysis reflects the current situation, and thereby could also be thought of as pointing towards the potential of available full OA venues if publishers adapt their policies on copyright retention and licensing to align with criteria set out in Plan S.

Fig 4. Copyright criteria (CC-BY and copyright retention) of DOAJ journals

*The current implementation guidance also indicated that CC-BY-SA and CC0 would be acceptable. These have not been included in our analysis (yet).