Credit: FabrikaSimf/Shutterstock
Fabricated academic citations have become a defining problem of the artificial-intelligence era, but new research suggests they could eventually disappear. An analysis of 50 behaviors associated with large language models (LLMs) found that many have weakened or vanished as companies release more capable models. On average, behaviors that disappeared did so about 16 months after they were first described in a published paper. The analysis, described in a blog post, has not been peer-reviewed.
Researchers say that modern LLMs rarely, if ever, attribute information to non-existent sources. Other tendencies, including overthinking simple maths problems and complying with illogical requests, also appear to fade as models improve.
“As users of these systems, we don’t need to have a fixed view of what AI can and cannot do,” says co-author Owen Queen, an AI researcher at Stanford University in California.
But Maxime Topaz, a healthcare-AI researcher at Columbia University in New York City, cautions that a behavior’s decline in newer models does not mean its effects will disappear soon. Topaz found1 that the number of hallucinated citations in academic papers has increased significantly since 2023. The rise is likely linked to the growing use of LLMs by researchers, many of whom are not using the latest models.
A characteristic lifespan
Queen and his colleagues identified 50 LLM behaviors described and widely cited in publications between the end of 2022, when some of the first LLMs became widely available, and the end of 2025. They gave 20 LLMs released from mid-2024 to the end of 2026 the same prompts that had first elicited each behavior in the original studies. This allowed them to track whether each effect strengthened or weakened over time.
For example, the researchers asked the models to count the number of r’s in “strawberry”, a task that LLMs once found difficult but can now perform more reliably.
Almost half of the behaviors tested had largely disappeared in the newer models, leading the researchers to consider them “expired”. The expired behaviors lasted an average of approximately 16 months after publication, although some vanished much sooner. Models stopped overthinking simple maths problems just a month and a half after that behavior was reported.
Other characteristics persisted. For example, models continued to stereotype Black women by portraying them unfairly as resilient, and this behavior lasted for more than three years.
Most of the behaviors tested became less pronounced over time, but three intensified. Today’s models praise users’ preferred answers more often than ever. They also show a stronger preference for their own writing over writing produced by other LLMs or by people. In addition, they are now even worse at generating genuinely random numbers than earlier models were.
No hallucinations?
Source: www.nature.com


