It’s Not All About the Journal Impact Factor

To the Editor:

We read with great interest the recent Panorama article that discussed the rise in the Journal Impact Factor (JIF) of rheumatology journals and the potential effect of the coronavirus disease 19 (COVID-19) pandemic in this rise.¹ Although there has definitely been a rise in the JIF of journals following the COVID-19 pandemic, there are concerns about some aspects of the paper that we wish to discuss in this letter.²

First, the JIF is a copyrighted index of Clarivate. Therefore, it would have been more appropriate to have it referred to by its full name, ie, the JIF rather than as the Impact Factor.³ Second, it is unclear whether the authors refer to the 2-year JIF or the 5-year JIF in their analysis. Third, it is uncertain which versions of the JIF are being referred to in the analysis. Clarivate publishes the Journal Citation Report (JCR) of the previous year. For example, the JCR 2021 was published midyear of 2022.³ When the authors refer to the JIFs from 2015 to 2020, it is not clear whether these refer to the JIFs published in the JCR from 2016 to 2021 or the JCR from 2015 to 2020. If the latter is the case (2015-2020), then no effect should be expected on the JIF as this would report the citation analysis up to December 2019 (when the COVID-19 pandemic was just beginning). Increments in the JIF are partly a reflection of the ever-increasing volume of scientific publications.⁴ This has not been clearly acknowledged in the paper by Mendoza-Lopez et al.¹ Finally, as discussed hereafter, the present article does not take into consideration the fallacies of the JIF as a journal-level metric.⁵

The JIF, a journal-level metric, is simply a ratio of the number of citations received by the journal to the number of citable items (articles or reviews) in the period of calculation (2 years or 5 years). Though this metric denotes nothing about the quality of an individual article in the journal, it is extensively misused by authors and academic reviewers alike. Therefore, the statement of Mendoza-Lopez et al that prospective authors should consider both the JIF and the rise in the JIF of journals before submitting their papers for consideration of publication may not be valid.¹ The JIF is notoriously prone to fluctuations, as shown by the historical rise in the JIF of Acta Crystallographica Section A from 2.0 to 49.9 in 2009 (the second highest at that time), and then back to ~2 from 2011 onward.⁵ The San Francisco Declaration on Research Assessment clearly states that published research should be evaluated on its own merits rather than on the basis of arbitrary metrics such as the JIF of the journal where it has been published.⁶ An indirect indicator of the quality of an individual paper could be the number of citations it receives over a period of time, or the attention that the article has garnered on social media platforms or from users, such as with the Altmetric score.⁷ However, even this has limitations, as some of the highest-scoring articles on Altmetrics in 2020 were actually retracted publications that gained attention for their notoriety rather than their quality.⁵ It is essential for authors as well as for agencies that review scientific output, such as universities or funding agencies, to bear in mind that mere numbers derived from journal-level metrics or author-level metrics cannot substitute or serve as a shortcut for research assessment.⁸ Research assessment should be based on the actual quality and rigor of such research and its relevance at large to the scientific community and to humanity at large.⁹

• Copyright © 2024 by the Journal of Rheumatology