3 Red Flags That Reveal a Dangerous Journal

Researchers have identified three warning signs that a journal might be involved in suspicious publishing activity. Applying those indicators to journal data, they found potential problems among dozens of titles indexed by Scopus, one of the major academic databases. The researchers stress that the indicators are signals for further investigation, not proof that a journal has engaged in misconduct.

The three warning signs are a doubling of a journal’s annual publication output, at least 70% of its articles coming from a single country, and at least 15% of its articles being flagged by the free online tool Questionable Paper Screener.

Using journal titles extracted from the freely available OpenAlex database, the researchers identified 3,753 journals that met at least one of these criteria. They cross-referenced the titles with journals indexed by Scopus and found that 5% had been delisted. The findings were published in a preprint on October 20¹.

Researchers and research-integrity experts say the signals can help identify journals that warrant closer examination, but they do not establish that a title is problematic. They are “rudimentary flags,” said Achar Agrawal, a co-author of the preprint and a freelance data scientist in Raipur, India. The findings also illustrate the challenges faced by bibliographic databases as questionable publishing practices become more common, he added.

Why Scopus matters

Bibliographic databases such as Scopus play an important role in the global research system because they provide data used in many rankings and assessments of individual researchers and institutions. Publication in Scopus, which is operated by Dutch publisher Elsevier, is generally viewed as an indication that a journal is trustworthy.

Scopus regularly reviews the journals on its list. The database uses tools to look for signs of suspicious activity that could indicate a journal is producing low-quality research or failing to follow accepted academic standards. These signs include “changes in publication volume and journal scope, unusual patterns in author and author collaboration, unusual citation behavior, and signs that the journal is becoming vulnerable to questionable content,” an Elsevier spokesperson told Nature.

For journals that receive reports, Scopus “carefully and individually evaluates all available information, including considering the journal’s own response,” the spokesperson added. Journal entries that do not meet the threshold for a specific metric are removed. Scopus delisted 156 journals last year and 100 so far this year.

An Elsevier spokesperson said that “no single signal or fixed threshold determines the outcome.” Scopus’ Content Selection Committee said that signals should prompt an investigation and that, when necessary, journals would be submitted for re-evaluation.

Signs of trouble

The study found that approximately 3,500 journals that had not been delisted collectively published more than 500,000 articles during the years in which they triggered at least one signal. Nearly 300 journals triggered two signals, while 28 triggered all three.

Anna Abarkina, a research-integrity researcher and social scientist at the Free University of Berlin, said that each indicator warrants further investigation but is not, by itself, a “clear decision for delisting.” “There are many reasons why a journal might double in publication volume,” she said, including changes made as part of a new journal’s or publisher’s business plan. However, Abarkina said that the presence of all three red flags suggests suspicious activity that is difficult to justify.

Álvaro Cabezas Clavijo, a researcher in research evaluation at La Rioja International University in Logroño, Spain, also said that the three indicators should be interpreted cautiously. For example, in a small journal, a large proportion of papers from one country may not be a “sign of questionable practices.”

Source: www.nature.com