6 What is Impact Factor, and why is it so important?

The impact factor of a journal relates to the number of times each publication from the journal gets cited in the two years preceding the date of the Impact Factor (IF) (Equation (6.1)). Thus, if you are thinking of publishing in a journal that has an IF of 1 you might expect that in the two years following the publication of your article you may get one citation. But as this is an average for all publications appearing in this journal, it is not necessarily true for your paper. As discussed elsewhere you might be very good at publicising your work and have it extensively cited. One or two extensively cited papers might even change the impact factor of the journal if it doesn’t have so many publications per year. If on the other hand you are thinking of publishing your article in a journal that has an IF of 5, you might expect that your article will be cited five times more than if you published in the first journal (IF = 1).

It’s a relatively simple calculation as seen in equation (6.1):

\[\begin{equation*} \frac{\mbox{The sum of all citations in journal X for year Y }}{(\mbox{No. pubs in journal X for year Y-1}) + (\mbox{No. pubs in journal X for year Y-2})} \tag{6.1} \end{equation*}\]

Because all citations for year Y are needed before the IF can be calculated for each journal, IF for the preceding two years is typically not released until June of Y+1.

Impact Factors are published by a number of different literature databases. For example, for the impact factor calculated by the Web of Science, if your journal is not even listed in the Web of Science then it will not have any Impact Factor. The Web of Science is continually policing the quality of the journals, and this means from time to time journals are excluded. This tends to happen at the lower end of the Impact Factor scale. But recently it happened to some very well known journals and there was a big stink (Pinto et al., 2021). You can read more about this here.

Note that there are conflicts of interest with publishing Impact Factors. For example, a publisher, Elsevier, owns Scopus and can decide whether or not a publication can get an Impact Factor. Similarly, the new scholarly database, Dimensions, is owned by Holtzbrinck who also own Springer Nature. Databases that are used by many of our employers in their means of evaluating our effectiveness are owned by for-profit companies. This is certainly cause for concern. There is a group of people who are trying to replace Impact Factor with a group of other metrics, so perhaps by the time you read this Impact Factor will no longer be relevant. If you’re interested read more here.

6.1 From a simple score to a way of life

When IF was originally devised by Eugene Garfield in 1955, it wasn’t supposed to govern the lives of academics, it was simply intended to be a way of deciding which journals to include in the Science Citation Index (Garfield, 1999). It then became useful for librarians to help them decide which journals to keep and which to ditch under ever constrained budgets (caused by publishers’ ever increasing prices). But along the way, this very simple index is now considered by many people to be a measure of quality, prestige and even academic success (Garfield, 1999). Many people have highlighted how wrong these beliefs are, but the growing trouble is that not only have many academics been misled, but so have administrators responsible for hiring and promotions.

“Like nuclear energy, the impact factor has become a mixed blessing.”

—Eugene Garfield (1999)

In a paper by McKiernan et al. (2019), they found that IF features in the guidelines of many university panels responsible for the fate of academics jobs and therefore lives. Worryingly, many of these institutions don’t actually talk about what IF measures. Instead they equate it with values and qualities that it certainly does not represent. Thus, you may find that your career is influenced by a simple metric that almost all who use it don’t actually understand. The undue influence on lives of scientists that IF has led directly to the San Francisco Declaration on Research Assessment (known as DORA) to which many institutions and publishers have signed up. You should read this very simple declaration and find out whether your institution is a signatory. If they are, then remember to hold them to the DORA principles during any assessment that you undergo.

Goodheart’s Law: “When a measure becomes a target, it ceases to be a good measure.”

—Marilyn Strathern (1997)

Charles Goodheart originally came up with a term that specifically related to economics, but this was generalised beyond economics by Marilyn Strathern (1997). It is hard to emphasise just how far publishing metrics have now come to embody Goodheart’s Law (Fire & Guestrin, 2019). The Impact Factor now dominates many aspects of life for Early Career Researchers, where the pay-off for a high impact publication might make the difference between having a job or leaving science altogether. The pressure is so high that it leads to bad behaviour, misconduct and fraud.

6.2 Five year Impact Factor

Many journals report their ‘Five Year Impact Factor’ in addition the the standard two year timeframe (as seen in equation (6.1)). This is because many disciplines, such as biological sciences, don’t have maximum impact of articles within the first two years of publication as do subjects like medicine and physics. Papers with immediate impact can be equated with sensational breaking news stories that instantly grab headlines, making money for those news outlets. These ‘hot papers’, ‘jump-decay’ or ‘fast breaking papers’ (i.e. left skewed distribution) will inflate the IF of the journal, which, if sustained, will earn the publisher more money. Most papers in biological sciences have a ‘delayed peak’ (i.e. low frequency and long tails), consistently gathering small numbers of citations over long periods of time. Papers that suddenly become popular after a long time with no or very few citations are termed ‘sleeping beauties’ (i.e. right skewed, see Bornmann & Marx, 2012). The last in the group of four citation types includes those that have a constant, usually low, number of citations over time. Together, each of these four patterns can be described by three variables that describe the citation behaviour of all papers: fitness, immediacy and longevity (see Fortunato et al., 2018). Using fitness, immediacy and longevity of citations to papers within a field can help to normalise citation metrics between different areas of science.

Although the five year Impact Factor might be much more appropriate, most people ignore this metric. While this metric might be more appropriate, because of the bad ways in which people have used impact factor, it is probably better to push back against this metric as you should against others.

6.3 What can you do if you publish in a journal with high IF?

Very high ranking journals for Impact Factor are publications like Cell, Nature and Science. This is because these publications are read by a very great number of people, and so are widely cited. Articles that get published in them receive a lot of attention from the press and media. This results in the prestige that a hiring institution might be looking for. If your academics publish in this journal, your institution may well receive lots of positive publicity. In some countries, notably China, there may be a cash incentive towards publishing in a journal with a high IF (Quan, Chen & Shu, 2017).

These highest ranking journals also demonstrate that IF is not linked to quality. Increasing numbers of disciplines are seen to be rejecting the reliability of many papers published in highly prestigious journals (Brembs, 2018). Moreover, there are higher levels of fraud reported in higher ranking journals see part IV. Only ~25% of articles published in these journals contribute to their very high impact factors (Editorial, 2005). Taken together, we should question the worth of these highest ranking journals to the scientific project.

One frightening trend in biological sciences is that the higher the Impact Factor the more the journal will charge you to publish in it (Gray, 2020; Mekonnen et al., 2021). At the time of writing in November 2020 Nature has just announced that they will charge USD 9500 to publish OA in their highest-ranked journal (see Part II). These sums are an order of magnitude greater than journals like these (with rejection rates >90%) should reasonably charge (Grossmann & Brembs, 2021). This is more money than it cost to publish in any other journal at the moment, and will be greater than the cost of many research projects, or even salaries of Early Career Researchers in some parts of the world.

6.4 Why is IF so important?

Academics are measured by their productivity but also about the quality of their output. Because there are so many different academic disciplines the bean counters who administer us need some way of ranking academics against each other. This is why they use the Impact Factor of the journals in which their academics publish in order to determine the quality of their output. Even though there are other metrics of the actual quality of an academic, most administrators continue to cling to IF and their beliefs of what it stands for.

Some countries reward their academics if they publish in high ranking journals. This can result in a salary bonus (Quan, Chen & Shu, 2017). It may also help with promotion, getting tenure or even just getting an interview for a job (Schimanski & Alperin, 2018; McKiernan et al., 2019). If you’re going to publish and you want a career in academia then you need to be aware of Impact Factors and what they mean to different stakeholders.

Many people will complain that their particular sub-discipline has a range of very low ranking journals with low impact factors. Others complain that journals with high impact factors tend to be edited by an old boys club that facilitates the members. In some cases like Proceedings of the National Academy of Sciences of the United States of America (PNAS) this is certainly true. Member sponsored manuscripts are also handled by the submitting member, meaning that these editors are able to ensure sweetheart reviews should they wish. This has led to some very questionable work being published in one of the world’s most prestigious scientific journals because the authors had good connections (Fainra & Gibbons, 2022).

6.5 Editors try to increase IF

It’s important to remember that editors care about Impact Factor (see Ioannidis & Thombs, 2019). There are several reasons for this. Firstly, the impact factor of the journal can be used (by the publisher or society) as a simple measure of how well the editor is doing. Secondly, the higher the impact factor of the journal, the number of submissions of manuscripts increases so that the editor can select ones they perceive to be of higher ‘quality’. Being the editor of a journal with a low (or no) impact factor can result in receiving fewer, more mediocre manuscripts. Editors can only choose their content from what is submitted. Poor manuscripts take up much more time than good ones: more rounds of review, more disagreements among reviewers and more time spent making editorial decisions. Thus, by increasing the IF of the journal that you edit, you are likely to increase both the number of submissions (allowing you to reject poorer ones) and retain better ones.

All this means that if editors believe that your paper will not garner the same or more citations in two years as the current Impact Factor of their journal, they may desk reject your submission. This is just one of the ways in which editors are known to manipulate impact factors for their journals. Established ways (Metze, 2010; Martin, 2016) of editors increasing impact factor for their journals include:

1. Ask authors to cite publications from their journal published within the last 2 years.
2. Ask reviewers to suggest publications from their journal published within the last two years to authors on which their review is conducted.
3. Encourage the submission of papers from laboratories with high output and citation rates.
4. Reject papers that are likely to have no citations. This effectively reduces the size of the denominator in the above equation.
5. Publishing issues in January means they have a maximum period of the year to get cited. This is now being inflated to having issues published online well ahead of the January date all the time gathering citations.
6. Encourage review articles which themselves garner more citations.
7. Editorials that cite every paper in the journal. This tactic is frequently used in special issues.

Evidence of this unscrupulous wrangling of Impact Factor within the Biological Sciences has been found when looking at the differentiation of inflation rates between journals. Wilson (2007) found that for-profit journals managed to increase their IF faster than society journals within the same period.

6.5.1 Negotiating your IF

As the number of citations from your published content is divided by the number of papers, one way of improving the IF of a journal is to reduce the number of papers that are counted towards the denominator in the calculation made by the citations database. It has been known for some time that those journals with the highest IF negotiate the removal of all of their editorial and news content from their denominators, making the number of publications much smaller and hence the IF larger (Garfield, 1999; Adam, 2002). On the other hand, if any of these news articles or commentaries get citations, these are included in the addition to the numerator. Thus our favourite high impact journals, Nature and Science, can publish very citable news at the front of their magazine, but negotiate with the commercially minded database owners about exactly which of their content counts towards their IF (Brembs, Button & Munafò, 2013). Later, we will see evidence of the financial leverage that these negotiations can reward these top tier journals (see Part IV).

6.6 Push back against IF

Just like any metric, Impact Factor is liable for abuse. You need to be aware of how IF is used and abused by many people in the academic community. You also need to be aware of what the rewards are for these individuals. Our problem with Impact Factor is not really the way in which it is manipulated by individuals to achieve their own ends. Instead, we should be worried about the way in which it leads the scientific community towards bad science and dishonesty. People who have benefited from using IF to measure their careers are likely to object if their institutions abandon it (e.g. Chawla, 2021), even though they are already signatories of DORA. Retaining IF benefits these senior academics, their closed practices and the publishing industry who use this metric (that they own and police themselves) to direct money from taxpayers earmarked for research into their own pockets (more on this in Part IV).

If you must calculate IF, one very simple way that you can push back against industry calculated IF is to calculate IF scores for your own papers, and show how they relate to the IF of the journal that you publish in. In this way, you are simply comparing your actual citations in the years (2 or more as appropriate) after your paper is published with the mean for the journal. There is an even chance that you generally get more citations than the mean for the journal, and you can convincingly show that your citations are consistently higher than the journal IF. For this to be true, you might need to help your work get cited, and that’s the subject of another section.