They do this in various ways, like establishing paper pipelines, collecting rents on labs and committees, focusing on money layer, using their profiles and citation count to help with acceptance, etc. You talk to them and they can’t explain their papers beyond a superficial introduction.
They collect huge citations, travel and give talk on the winner horses, collect credit, which feeds back into this scheme.
I wonder if Google could invent a new metric to expose them (weak ratio of first authorship, etc).
Please feel free to disagree with me! I am keen to hear more anecdotes to get more datapoints.
it would be funny if by accelerating the enterprise it actually forced an effort to correct the trajectory.
It's really, _really_ high time we dispensed with the idea that this is "AI". Nobody said they're not useful, but "AI" they are not.
To me this effect doesn’t seem to reflect on AI very much, it seems to reflect on humans. Like maybe this is more evidence of the Babble Hypothesis and the incentives in research than AI, no?
It would have been useful to check whether less original work was already getting more citations before AI adoption. That could reflect broader trends and network effects: heavily cited research areas attract more authors optimizing for citations, so high-productivity researchers end up clustering on the same topics.
We tend to think that obvious potential is the same as realized potential, for new technology.
For any specific context, there are generally innumerable smaller adaptations and capability thresholds that have to be crossed. And the price for that journey is often temporary loss off overt productivity.
By definition, creativity cannot be automated, and AI is a fantastic automation machine. It can explore thinking paths at a rate humans cannot match. But creativity is bringing the unthinkable into the thinkable, and that requires sensory experience [1]. Specifically, new definitions and symbols which never existed before. Imagine the concept vector space, and expanding that with new independent dimensions. Is that even possible ? When you look at history the answer is yes !. And each time there was an independent dimension added, it was an act of genius. It is an instructive exercise to name these moments in history where an independent dimension was added to human thought. Some examples in math would be the invention of a number, and in politics could be the idea of democracy. By contrast, LLMs are trapped in the vector space they are trained on, and they lack the feedback loop with sensory experience to be able to create and validate theories.
[1] https://philsci-archive.pitt.edu/28024/1/Scientific_Inventio...
'It’s not about the architecture per se,' Evans says. 'It’s about the incentives.'"
But also just thinking about your point for one second: in your mind, how else would they argue for the conclusion if not by checking the trend over time? Like what is the precise implication here?
When we solve problems we usually follow a heuristically guided energy efficient path. We just prune a lot of possibilities based on our existing knowledge and experience.
Creativity happens when we consciously (or not) go off the beaten path and explore. Most of those explorations are dead ends. But some will yield unexpected connections, patterns etc that we call “creativity” .
An AI system could also go on those kinds of explorations. Today they aren’t it because we are not asking them to.
A lot of the time people state the kind of fundamental limitations of LLMs very confidently when it feels like it is too early for people to really know. Like we are already well past the point where where LLMs are just pre trains on the internet with some RLHF for chatbot… Most of the effort is spent on elaborate reinforcement learning.
Is it unconceivable that future generations of LLMs could be RL’d to use einsteins visual method for theories [1] with the right tooling and geometry representations? Or just something random like that.
[1]. https://www.visualscribing.com/blog/2019-11-11-einstein-on-v...
> The emergence of physically consistent World Models offers a pathway to a synthetic laboratory. By enabling agents to run counterfactual simulations—to experience the physical consequences of a thought experiment—we may finally mechanize the feedback loop between intuition and logic.
I see it as an overfitting problem. Fundamentally, the topic here seems to be that citation indices and similar metrics are actually flawed indicators, and obsessing over them is just Goodhart's law in action. Ultimately, the argument is that the entire design of those metrics is wrong. To be precise, it was a good metric at first, but now that the scale has changed, it's become bad. This is common in programming too—things that are correct in the beginning but become problematic as they grow larger.
From an individual researcher's perspective, it's rational. You get more citations, your career accelerates. Everyone knows this. Paper counts aren't everything. Citation counts aren't everything. Journal impact factors aren't everything. You shouldn't only play it safe. But everything is tied to those metrics anyway.
Most researchers who give me work are fully aware of these facts. But are they going to change anything? Funding is still distributed based on those metrics.
Max Planck said, 'Science advances one funeral at a time.' Science doesn't progress purely through reasoned argument. The authority of the older generation, research funding networks, journals, and school-specific evaluation criteria all move together.
And honestly, I think discoveries will keep happening—probably quite rapidly. Because AI doesn't have the factional conflicts or interpersonal issues that humans do. It's very good at connecting papers across schools of thought without bias. In other words, the current human system is flawed at consolidating research, but I think AI is actually strong in this area. I expect AI-driven discoveries will continue for some time. The people who ride this wave will clearly be the winners.
Everyone knows things are broken, but no one is trying to fix them. I always think human society is inefficient. I read this post, but I'm more curious about who will actually lead the improvement effort.
The aim of many scientists is discovery, publishing is a side chore to survive and to get funding. Automate paperwork and you get more time for discovering.
I think the flattening of progress is the most interesting dimension to the article. For an example a useful biological product discovery with a nonlinear path to get to there, look at the Taq polymerase (https://en.wikipedia.org/wiki/Taq_polymerase). Without some NSF funded exploratory ecological research by Tom Brock in Yellowstone Hot Springs to test the theoretical limit of life at high temperatures (https://en.wikipedia.org/wiki/Thermus_aquaticus) we never get to the Taq polymerase, we never get reliable/robust PCR (https://en.wikipedia.org/wiki/Polymerase_chain_reaction), which is now a gold standard method in both clinical and environmental testing! It is rather improbable to think that large language models would associate those domain connections across the topic (molecular biotechnology + ecology + microbial physiology). I also did some exploratory work with text embedding models people might use for RAG and challenged them with an open source scientific MCA question dataset, generalist embedders performed worse vs. domain specific embedders trained on scientific corpora (doesn't surprise me at all). However, if everything regresses to the median of the universe of possible knowledge, it seems like scientific leaning frontier models would get locked into this asymptotic flattening before turning cashflow positive for model vendors OR they become so locked down that only big pharma, state actors, or big ag can afford the API rates and vetting process.
I like LLM's but this writing style is like eating the same dish 4 times a day.
the last decade of US politics demonstrates just how powerful willingness to produce put strips all other critical skills.
AI exacerbates this and exposes fundamental human heuristic frailty.
I was also in academics myself up to the Master's level (research track), and personally had to deal with the politics of getting support for what I wanted to work on; that experience helped to discourage me from going on to a PhD, as I'd rather have proper leeway to work on what I really prefer and take avenues I find interesting.
AIs do things no human has done before millions of times a day.
https://news.ycombinator.com/item?id=48863490
LLMs don't just 'average' their data.
All the factional conflicts are in there, and there are also plenty of reports of people getting weird / toxic / passive aggressive responses from AI.
Because the model is trained with everything, you can in principle get anything out of it. You want to get an answer based on all the right things, while keeping all the wrong things suppressed. But it's easy to get something less than ideal, due to the specifics of training, harnesses, context, prompts etc.
Well, these AI are never going to die in any real sense, so expect them to make orthodoxy more sticky, not less.
LLMs are aggressively trained to reproduce facts and consequently struggle to reject orthodoxy. There isn't any reason they can't, in principal, make big new discoveries just by getting lucky, which is sort of also how humans do it, but its ok to acknowledge that current AIs aren't so good at certain things.
I have a hard time believing that all novel concepts yet to be discovered are contained within that space, though.
I presume you are an expert in some field. Think carefully about the boundary of the field and all the subtlety and complexity of that boundary and all the oversimplification you do to communicate that stuff to lay people. AI is, in some large sense, directed at all lay people, not experts, and even if we wanted to direct it at experts, at the edges of knowledge, there really isn't a lot of training data for that. Mathematics is a sort of exception because it has very clear validation criteria which makes RF particularly easy for it.
many children have an unlimited capacity to ask "why?". many adults are the same
if the abilities of AI are finite, then we will continue to have burning curiosity, questions to ask, and discoveries to make
This seems like some variant of "why don't you short the market and become rich". It doesn't work like that.
Should be interesting to see what happens to the programming profession when there isn't anyone around anymore who actually knows programming.
When the child is able to go to YouTube and find a tutorial rather than having to puzzle it out, yes, it absolute does. We've seen this for decades now.
Once the Pythagorean theorem was proposed, many different proofs have been identified. In art, once a new style is created it's often straightforward for others to replicate. In physics, the idea of Relativity was what enabled the design of experiments to demonstrate its correctness. Proposing the idea is what's essential.
The first type happens when you are enthusiastically engaged in a topic, which LLMs will likely enhance.
The second type happens as a by-product of solving a, perhaps deeply uncomfortably, difficult problem. This is what people are talking about when they say LLMs will hamper human cognition. Instead of sitting there for an hour and struggling, people will instead reflexively give in and ask an LLM to solve it for them.
I think in most cases, understanding is the point. we don't expect students to derive general relativity before doing astrophysics. re-invention is only a tool for understanding
The flip side is even more interesting. There’s a great number of electrical engineers or even with significant physics backgrounds who don’t really understand how electricity actually works, but they can still solve useful problems. By understanding I mean they can describe what underlying physical phenomena reactance represents etc.
AI is turning scientists into publishing machines—and quietly funneling them into the same crowded corners of research.
That’s the conclusion of an analysis of more than 40 million academic papers, which found that scientists who use AI tools in their research publish more papers, accumulate more citations, and reach leadership roles sooner than peers who don’t.
But there’s a catch. As individual scholars soar through the academic ranks, science as a whole shrinks its curiosity. AI-heavy research covers less topical ground, clusters around the same data-rich problems, and sparks less follow-on engagement between studies.
The findings highlight a tension between personal career advancement and collective scientific progress, as tools such as ChatGPT and AlphaFold seem to reward speed and scale—but not surprise.
“You have this conflict between individual incentives and science as a whole,” says James Evans, a sociologist at the University of Chicago who led the study.
And as more researchers pile onto the same scientific bandwagons, some experts worry about a feedback loop of conformity and declining originality. “This is very problematic,” says Luís Nunes Amaral, a physicist who studies complex systems at Northwestern University. “We are digging the same hole deeper and deeper.”
Evans and his colleagues published the findings 14 January in the journal Nature.
For Evans, the tension between efficiency and exploration is familiar terrain. He has spent more than a decade using massive publication and citation datasets to quantify how ideas spread, stall, and sometimes converge.
In 2008, he showed that the shift to online publishing and search made scientists more likely to read and cite the same highly visible papers, accelerating the dissemination of new ideas but narrowing the range of ideas in circulation. Later work detailed how career incentives quietly steer scientists toward safer, more crowded questions rather than riskier, original ones.
Other studies tracked how large fields tend to slow their rate of conceptual innovation over time, even as the volume of papers explodes. And more recently, Evans has begun turning the same quantitative lens on AI itself, examining how algorithms reshape collective attention, discovery, and the organization of knowledge.
That earlier work often carried a note of warning: The same tools and incentives that make science more efficient can also compress the space of ideas scientists collectively explore. The new analysis now suggests that AI may be pushing this dynamic into overdrive.
To quantify the effect, Evans and collaborators from the Beijing National Research Center for Information Science and Technology trained a natural language processing model to identify AI-augmented research across six natural science disciplines.
Their dataset included 41.3 million English-language papers published between 1980 and 2025 in biology, chemistry, physics, medicine, materials science, and geology. They excluded fields such as computer science and mathematics that focus on developing AI methods themselves.
The researchers traced the careers of individual scientists, examined how their papers accumulated attention, and zoomed out to consider how entire fields clustered or dispersed intellectually over time. They compared roughly 311,000 papers that incorporated AI in some way—through the use of neural networks or large language models, for example—with millions of others that did not.
AI adoption boosts individual scientific impact, with AI-using researchers consistently earning more citations than those who do not use AI.Veda C. Storey
The results revealed a striking trade-off. Scientists who adopt AI gain productivity and visibility: On average, they publish three times as many papers, receive nearly five times as many citations, and become team leaders a year or two earlier than those who do not.
But when those papers are mapped in a high-dimensional “knowledge space,” AI-heavy research occupies a smaller intellectual footprint, clusters more tightly around popular, data-rich problems, and generates weaker networks of follow-on engagement between studies.
The pattern held across decades of AI development, spanning early machine learning, the rise of deep learning, and the current wave of generative AI. “If anything,” Evans notes, “it’s intensifying.”
Intellectual narrowing isn’t the only unintended consequence either. With automated tools making it easier to mass-produce manuscripts and conference submissions, journal editors and meeting organizers have witnessed a surge in low-quality and fraudulent papers or presentations, often produced at industrial scale.
“We’ve become so obsessed with the number of papers [that scientists publish] that we are not thinking about what it is that we are researching—and in what ways that contributes to a better understanding of reality, of health, and of the natural world,” says Nunes Amaral, who detailed the phenomenon of AI-fueled research paper mills last year.
Aside from recent publishing distortions, Evans’s analysis suggests that AI is largely automating the most tractable parts of science rather than expanding its frontiers.
Models trained on abundant existing data excel at optimizing well-defined problems: predicting protein structures, classifying images, extracting patterns from massive datasets. Some systems have also begun to propose new hypotheses and directions of inquiry—a glimpse of what some now call an “AI co-scientist.”
But unless they are deliberately designed and incentivized to do so, such systems—and the scientists who rely on them—are unlikely to venture into poorly mapped territories where data are scarce and questions are messier, Evans says. The danger is not that science slows down, but that it becomes more homogeneous. Individual labs may race ahead, while the collective enterprise risks converging on the same problems, methods, and answers—a high-speed version of the same narrowing Evans first documented when search engines replaced library stacks.
“This is a really scary paper to think about in terms of how the second- and third-order effects of using AI in science play out,” says Catherine Shea, a social psychologist who studies organizational behavior at Carnegie Mellon University’s Tepper School of Business in Pittsburgh.
“Certain types of questions are more amenable to AI tools,” she notes. And in an academic environment in which papers are the main currency of success, researchers naturally gravitate toward the problems that are easiest for these tools to crank through and turn into publishable results. “It just becomes this self-reinforcing loop over time,” Shea says.
Whether this trend persists may depend on how the next generation of AI tools is built and deployed across scientific workflows.
In a paper published last month, Bowen Zhou and his colleagues at the Shanghai Artificial Intelligence Laboratory in China argued that the application of AI in science remains fragmented, with data, computation, and hypothesis-generation tools often deployed in a siloed and task-specific fashion, limiting knowledge transfer and blunting transformative discovery. But when those elements are integrated, AI-for-science systems help expand scientific discovery, says Zhou, a machine-learning researcher who previously served as chief scientist of the IBM Watson Group.
Perhaps, says Evans. But he doesn’t think that the problem is baked into the algorithmic design of AI. More than technical integration, he argues, what may matter most is overhauling the reward structures that shape what scientists choose to work on in the first place.
“It’s not about the architecture per se,” Evans says. “It’s about the incentives.”
Now, says Evans, the challenge is to deliberately redirect how AI is used and rewarded in science: “In some sense, we haven’t fundamentally invested in the real value proposition of AI for science, which is asking what it might allow us to do that we haven’t done before.”
“I’m an AI optimist,” he adds. “My hope is that this [paper] will be a provocation to using AI in different ways”—ways that expand the kinds of questions scientists are willing to pursue, rather than simply accelerating work on the most tractable ones. “This is the grand challenge if we want to be growing new fields.”
This article appears in the March 2026 print issue as “AI Helps Scientists but Hurts Science.”