Scientific Insurgents Say "Journal Impact Factors" Distort Science
An international declaration was published today calling for the scientific community to eliminate the role of Journal Impact Factor (JIF) in evaluating research for funding, hiring, promotion, and institutional effectiveness.
The call for new standards gives voice to more than 150 prominent scientists and 75 scientific groups, including journal editors, publishers, and scholars from around the world, who are hoping to improve the ways in which the outputs of scientific research are evaluated. What began as an "insurrection" against the JIF, as one publisher put it, has turned into a wider reconsideration of scientific assessment.
The declaration is timed to coincide with editorials published in journals the world over, including an endorsement from Bruce Alberts, Editor-in-Chief of Science magazine in the journal's May 17th issue. Other endorsers of the declaration include the editors of Journal of Cell Biology (JCB), Traffic, Genetics, eLife, Journal of Cell Science, Aging Cell, Molecular Biology of the Cell (MBoC), BioArchitecture, The EMBO Journal, Journal of Cell Science, Journal of Surfactants & Detergents, Cell Structure & Functions, Lipids, Genes, Journal of the Electrochemical Society, and Development, among others.
The San Francisco Declaration on Research Assessment, or DORA, can be read and signed at its official website, where a full list of signatories can be found as well. DORA was framed by a group of journal editors, publishers, and others convened by the American Society for Cell Biology (ASCB) last December in San Francisco, at the society's annual meeting. The group agreed that JIF has become an obsession in science. JIF ranks scholarly journals by the average number of citations their articles attract in a set period.
The problem with JIF is that it warps the scientific process, including how research is conducted, reported, and funded, the group said. The DORA statement published today includes 18 recommendations for change in the scientific culture, at the level of the individual scientist, publishers, institutions, funding agencies, and bibliometric services. Instead of focusing on the JIF value, DORA would have scientists and organizations pay closer attention to the content of research, and matching that content with the appropriate audience, regardless of publication value.
Although a variety of metrics exist that rank scientific publications, the "two-year" JIF is the oldest and most influential. The two-year JIF is the average number of citations received in a year per paper published in a given journal, during the two preceding years. This means that the earliest a new journal can have a JIF is the end of its third full year of publication.
Originally, the JIF, created by Eugene Garfield, was used in the 1950s as a subscription-buying tool for academic and medical librarians. It was published by Garfield's Institute for Scientific Information (ISI). Today, the JIF appears once per year in Journal Citation Reports, as part of the Thomson Reuters (ISI) Web of Knowledge. The JIF is widely misused, and treated as powerful proxy for scientific value when it shouldn't be, said the DORA framers. The JIF has become especially powerful in China, India, and other nations emerging as global research powers.
"It's maddening," said David Drubin, Editor-in-Chief of ASCB's journal Molecular Biology of the Cell (MBoC). "This is a metric that really drives a lot of traffic. You really see it most clearly when you travel to foreign countries and I especially see it with my foreign postdocs. They only want to publish in journals with high impact factors."
Another problem with JIF is that because it is based on the mean number of citations per article, rather than the median, a blockbuster article can have a disproportionate affect on a journal's rating, said Bernd Pulverer, Chief Editor of the EMBO Journal.
"My favorite example is the first paper on the sequencing of the human genome," Pulverer said. "This paper, which has been cited just under 10,000 times to date, single-handedly increased Nature's JIF for a couple of years."
"The Journal Impact Factor (JIF) was developed to help librarians make subscription decisions, but it's become a proxy for the quality of research," said Stefano Bertuzzi, ASCB Executive Director. "Researchers are now judged by where they publish not by what they publish. This is no longer a question of selling subscriptions. The 'high-impact' obsession is warping our scientific judgment, damaging careers, and wasting time and valuable work."
The declaration's 18 recommended changes are intended to create "a change in the culture where people will choose the journals that they publish in not on the prestige but on the fit," said Drubin. "Is the format correct? Is the audience correct? Does the editorial board have the appropriate expertise?" A difficult change, Drubin said, but a necessary one, vital to the integrity of scientific self-assessment.
As for his role in the JIF insurrection, Drubin said, "For me, it was just a matter of when enough is enough."