Russian Science: Its Place in the World
Is publication of a paper in a journal indexed in Web of Science or Scopus always an evidence of a high quality of research? How can we increase Russian scientists’ share in the global flow of academic publications? These and other issues were discussed during the online debate hosted by STRF.ru. Konstantin Fursov, head of HSE ISSEK’s R&D performance analysis sector, took part in it as a leading expert. Other participants included students and teachers of the “Governance of Science, Technology and Innovation” Master’s programme.
Konstantin Fursov's answers to the discussion participants’ questions:
1. Is publication of a paper in a journal indexed in Web of Science or Scopus always an evidence of a high quality of research and the specific researcher’s personal contribution?
KF: Publication of a paper in a journal indexed in a database provides no guarantee of its quality; a proof of that is the fact that fraudulent publications are regularly discovered in these databases. Of course, Web of Science’s and Scopus’s analytical services conduct rigorous evaluation of journals before including them in the databases, but that doesn’t guarantee that high quality of every single paper is always maintained.
As to productivity of particular researchers, their presence in WoS or Scopus can only serve as one evaluation criterion. A good example is annual Thomson Reuters’s attempts to predict Nobel Committee’s decisions. Only very few researchers from the potential candidates list get the prize, while actual laureates may not be included in the list (based on bibliometric data) at all.
2. When is a scientific journal’s impact factor relevant, and when isn’t it?
KF: To put it simply, impact factor is an indicator of a papers’ citation level in scientific journals. Therefore, one might suppose that a paper published in a journal with a higher impact factor is likely to have a higher quality level, but this doesn’t at all mean that we won’t find good papers in journals with lower rankings. For a number of reasons, in particular simplicity of calculation and “objectivity”, impact factor is used almost universally, mainly to identify leading publications. When you use this indicator to evaluate research productivity it’s important to remember that its meaning differs in various research areas, so it can’t be used as an all-purpose yardstick for all journals alike.
3. Would it be right to provide material motivation to increase scientists’ publication activity?
KF: Material motivation is not illegal, doesn’t contradict professional ethical norms, and can be seen as a way to recognise the value of research. However, such motivation may provoke scientists to behave unfairly. In China, where during the recent decades scientists’ presence on the global research scene was actively promoted, there were numerous examples of various manipulative strategies employed to “take advantage of the system”. These included publications in pseudo-scientific journals, selling co-authorships, and fraud with expert evaluation.
Another – opposite – example of introducing material motivation is provided by Australia, where publication activity figures were incorporated in the decision-making mechanism for grants allocation. This resulted in explosive growth of publication activity, but the quality of the papers deteriorated. So any kind of motivation should have a clear purpose, and an adequate format.
4. How can the share of Russian scientific publications in the global flow be increased?
KF:I’d rather speak about strengthening Russian science’s positions in the world. A country’s share in the overall global number of publications is just one indicator, which depends on scientific productivity of other countries, national journals’ presence in databases, completeness of indexed information, and other factors.
If we are to answer this question “idealistically”, being aware that the 2.44% figure set in the president’s May decree is not an end in itself, increasing Russia’s status on the global scientific arena requires, in addition to more funding, developing various institutions, including those responsible for strengthening international cooperation and integrating Russian science into the global research environment.
5. In 2015, Thomson Reuters announced inclusion of the Russian Science Citation Index (RSCI) in the Web of Science platform now comprising more than 600 scientific journals. How will this project affect Russian science?
KF: Selecting journals for inclusion in the RSCI is probably the most serious and formidable attempt to identify Russian scientific journals of a relatively high level. If institutions responsible for the evaluation of scientists agree with this approach, we may expect shifts in researchers’ publication behaviour and in journals’ standards – because the RSCI list sets higher standards than any Higher Attestation Commission’s list ever did. A journal’s inclusion in the RSCI can open opportunities for making the next step – towards top-level international league. In a way it’s a challenge for Russian scientific publications.
As to the inclusion of journals in the Web of Science platform, that hardly opens new opportunities for Russian scientists – apart form a certain improvement of their “visibility”. This index is not a part of the Core Collection, so it’s unlikely to significantly improve the country’s place in the global rating.
Next come comments by Konstantin Fursov and other participants of the discussion – HSE ISSEK staff members
A question by Anna Gorbatova, PhD (economics), STRF.ru special correspondent:Konstantin, do the Web of Science and Scopus provide a full picture of Russian scientists’ publication activity? Are there any other sources one should use?
KF: In terms of measuring the “visibility” of Russian scientists on the global scientific arena, they’re quite sufficient. Though the WoS and Scopus databases are not the only sources of information about publication activity and have various drawbacks, primarily due to their in-built inclination towards natural sciences (the WoS) and English-language publications (that goes for both of them, though the Russian-language segment is definitely better represented in Scopus), they still remain the best information resources. Also, they’re well-structured (unlike, e.g., Google Scholar) and offer several useful analytical tools for quick calculation of major indicators.
Apart from the WoS and Scopus which are very expensive (and that’s another drawback), there are numerous other specialised resources offering information about publications on specific research areas; these are used for analytical purposes. The best known include PubMed (an open access version of the commercial MEDLINE database) on medical science, SciFinder on chemistry, Aerospace & High Technology Database on space exploration. But it should be noted that in all of them the Russian segment, is represented approximately on a par with the WoS or Scopus.
But speaking about more comprehensive measurement of Russian scientists’ publication activity, the Russian Science Citation Index could be a valuable source, together with (potentially) the Map of Russian Science. Why? Because the “analytical functionality” of both these systems is currently rather low. The Map of Science is still operating in the pilot mode only. So these sources so far are suitable maybe only for finding and viewing information about particular authors and organisations.
A question by Alina Kadyrova, student of the HSE ISSEK Master’s programme “Governance of Science, Technology and Innovation”: I have a question: if journals’ impact factor is not, according to Konstantin, a reliable indicator of a papers’ “quality”, should it be considered in making decisions about material compensation to staff members? As far as I understand, compensation criteria are the same for the whole university, not research area-specific.
KF: It makes perfect sense using impact factor as one of the criteria (together with, e.g., overall publication activity of the researcher), because it can serve as a proxy indicator of a paper’s quality. Therefore it’s important to take into account at least the research area the journal belongs to, and not use the same criterion for all. E.g. the leading sociological journal, Annual Review of Sociology (IF = 4.1), is hardly likely to ever reach the level of the medical journal Lancet (IF = 45.2). As to criteria for determining material compensation, they’re quite different all over the place; universities shape their own compensation and motivation policies. Just to give an example: instead of impact factor, HSE takes into account quartiles in which various journals belong.
A question by Ivan Bolokhov, postgraduate student of the Moscow State University’s Political Science Faculty: Dear experts, for quite some while now China has been actively encouraging its scientists to publish in English, in Chinese and international journals alike. That resulted in significant increase of citation level. Do you think this practice would be useful in Russia? After all, practically nobody reads papers in Russia, and by following suit we could significantly increase Russian scientists’ presence on the international scientific arena.
Comment by Ekaterina Dyachenko, research fellow at the ISSEK Laboratory for Economics of Innovation:Papers by Russian scientists included in theWeb of Science database have indeed mostly been written in English in recent years. But our scientists publish several times more papers in Russian-language journals not indexed in Web of Science. Even if we take the relatively small range of journals recently included in the Russian Science Citation Index, they annually publish 1.5–2 times more papers than the number of papers by Russian authors included in Web of Science. And mostly it’s Russian-language papers.
The study ”Changes in Publication Languages and Citation Practices and Their Effect on the Scientific Impact of Russian Science (1993–2010)” by Olesia Kirchik and her Canadian colleagues shows how significantly citation of papers by Russian authors published in Russian and in English differs. So the claim that most of the Russian science remains little known to the rest of the world is not entirely unfounded.
However, tools for increasing this visibility must be well thought-out. Just introducing a significant material motivation to publish in English would have negative consequences, too.
The full discussion is available at STRF.ru portal