There really is no point, as long as you verify the content matches your intent and edit out anything poorly written.

Frankly, I've read plenty of papers by native English-speakers over the years that'd strongly benefit from being rewritten by an LLM too...

If we imagine a set of all human ideas that these models have access to, then the set of possible discoveries would be something like the superset of all possible combinations of those ideas. I think all LLM discoveries are bounded by that space.

Looking at the recent OpenAI math discoveries, that seems to be pretty much what happened. Existing ideas were used as building blocks, the model found a valuable combination, and the result was something new that had real value.

If fake papers weren't already a big problem before AI and the fields had already been policing themselves adequately, if this was already a functioning high-trust domain, maybe we could ignore this a bit more, but the fields already manifestly had problems. People taking advantage of that are reasonably more likely to use AI. The pressures to publish or perish provide the voltage and the AIs are a rather convenient path-to-ground.

I agree in some sense that if a truth is published, it doesn't matter if the AI or a human published it. However there are perfectly reasonable reasons to be concerned that AI usage is correlated to not publishing truths, especially in a world where merely being human-generated was already not an adequate check against that.

In other words, it "sounds smart" without necessarily having anything to back it up. In even more critical terms, it's very good at bullshitting.

Unfortunately for us, the scientific community current relies on a certain amount of trust. (To do otherwise is very expensive! see: bitcoin). When you introduce a known-bullshitter to write your papers, every human in the loop effectively has to defend against an adversarial attack. Not just the readers at home, or the peer reviewers, but even the author needs to be wary that the facts and arguments coming out of the LLM are true and meaningful.

Personally, I've been a minor contributor to several high-profile papers. I don't know how every field does it, but in my experience, the corresponding author (generally the PI or other senior scientist), is responsible for the accuracy of the paper. They ultimately have to trust the people who did the work that the facts are true. Introducing LLMs into the mix make it more difficult for them to identify and review sections they are unsure of. (An honest person will typically write at a confidence level reflecting their certainty. LLMs do not do this in any reliable way.)

I've also found that LLMs frequently use metaphors that are unhelpful, or used out of context in a field that isn't familiar with them. This makes understanding the text more work, for no good reason. Introducing terms or definitions with low relevance reads as impressive at first glance, but avoiding the standard terminology in the field just adds confusion. (As an analogy, imagine if you were reading a CS paper that, for no particular reason, devoted a section to a new data structure called an "akimbo tree," which after much untangling, you realized was a reinvention of a randomized splay tree.)

What about the papers that graduate to proper publication?

Arxiv is full of pre-prints that anyone can upload.

To me it sounds like 1. Either your tool is just not that good and reliable as you thought, 2. AI is trained on human written articles, so some of that human written content informed the now established “AI slop”.

There are people who shipped “slop” before AI.

Have you tried doing that or even read the article?

The article says that their detector flags 0.4% of pre-AI papers as AI-written.

If I paste the first page from this paper (https://www.fourmilab.ch/etexts/einstein/specrel/specrel.pdf) in https://unslop.run/app, I get a 0% chance that it was AI-written.

A possible conclusion for this could be: If the majority of CS papers is AI written, let's just accept this reality universally and stop worrying about it altogether.

Pre-LLMs, a paper with no spelling or grammar errors showed that somebody had put effort into writing and editing it. If they cared about the presentation, they probably also cared about the content. LLMs routinely produce nonsense that looks superficially like high-quality work.

There are far too many papers to read all of them. LLM slop is evidence that something is probably low quality. As the saying goes, "if you can't be bothered writing it, I can't be bothered reading." The rare outliers will get enough citations and recommendations to overcome this filter.

My problem therefore is: we are seeing more and more papers written with tools that are known to make up facts, citations, and even entire papers. And the number of papers has increased, too. I therefore see it less as "people are being more productive" and more "people are releasing bad science much faster than we can keep up with".

Sure, some people are artists - but most aren't.

The difficulty with this is then: How do you get a clean post 2023 dataset? I have no straightforward idea for this. You can't use other AI detectors to build it because then you'd never outperform them.

FYI this is all relatively new so there might be lots of issues and iterations coming.

The funny thing is that "slop" was defined by the writing habits of AI model, which we have learned to pick upon and recognize.

The "It's not X, it's Y", the rhetorical questions and other patterns would have been the tools of a skilled writer, and those people writing "like AI" before AI most likely would have been recognized as such.

This is a sin.

"More scientific" is not some merely stylistic thing that faithfully preserves the original meaning of what you wrote. The precise details of each paragraph matters a lot in terms of what and how it communicates. The fact that these details do matter means that, according to my accounting, it is not entirely your paper.

Also, I find striving for "scientific" to be a pretty undesirable thing. Why should papers read like that? What is the benefit? The best papers (in terms of their writing and communication) are unpretentious and conversational. I'm pretty sure I'd prefer your "basic sentences", especially if they were wholly yours. (I understand that there are also external forces at play here as you mentioned.)

I see plenty of anecdotal evidence that models have been trained fantastically well—and getting better—at writing to trigger the right neurons in the human population to produce “This is interesting/informative/correct” responses in bulk.

Could their ability to produce those responses run far ahead of their ability to actually achieve the last in reality? Sure seems plausible, and then where are we?

That’s a big if. We all know that’s not what’s happening.

Reviewers were also totally unengaged. Of 20 reviews I read (from my reviewers or from reviewers on the same papers), maybe 2 were mediocre, and the rest were crap (though likely not AI).

The notion that science will somehow benefit from this is about as stupid an idea as you can have. Science relies on skepticism. AIs are not skeptical, and many folks are submitting papers because they stand to gain something, not because they are motivated to do good research or develop new understanding. Fields are being inundated with garbage that is maximally indistinguishable from real work (that's the training objective for LLMs). This in turn maximizes the cost of identifying bad work.

This is the same enshittification process that we see everywhere else. You get spam phone calls because there is no reason for a spammer not to call you. "Researchers" are submitting spam papers because there is no cost to doing so with some possible gain. Absent intervention, this eventually drives the community value of the network to zero (or potentially negative, if friction costs to switching are high).

if 50% of the work is nonsense, then there's a serious concern that we can't move forward at all.

Perhaps the most darkly amusing consequence of this particular mania is that by poisoning the majority of our information environment with hallucinated slop, we have likely crippled the next several generations of machine-learning techniques before they're even invented! Small, locally-hostable LLMs will rattle along spewing spam long after the broader "genai bubble" pops, and building clean training datasets will permanently be more difficult and expensive.

I am almost certain he was "hallucinating" the results. This was in the 2010s

There are well known issues in academic publishing, though I imagine it has become much noisier like open source

That's a big if. ArXiv is not peer reviewed and LLMs basically interpolate and extrapolate text, which makes them essentially fluff generators. Even in the most charitable interpretation, LLMs enable those with nothing to say to say nothing while meeting surface-level style guides.

Other problems include: Signal to Noise Ratio going through the roof.

AI hallucinates and makes up stuff 100% percent of the time. Never been a fan of that word for this.

Again, I fail to see the problem here that isn't solved by careful reading WHICH IS WHAT PEOPLE SHOULD BE DOING ANYWAY. I would like to see room for AI disclosure, maybe a statement of "this is how much AI I used."

But this blanket X% of this looks like AI? Again, so what?

Yes, I feel like there's room to improve things, I just strongly doubt that "using AI to detect AI" is a particularly useful thing to do here.

Trust is very important to human progress.

Pangram agrees: https://www.pangram.com/history/3de33376-94e3-404d-bbb0-751a...

> The article doesn't seem to mention consideration of AI for polishing human work.

Because it isn't a consideration. You are what they are looking for.

This was my big fear before we saw price increases. Now I'm pinning all my hopes on AI being too expensive to justify further big corporate pushes. (Sigh.) I love having new tools, but I hate being pushed to use ______ tool to meet some managerial metric.

I can't speak for the entire scientific research community, but I can say that for basic biomedical research (not clinical research), LLMs are mostly ignored. They simply don't have the ability to parse most raw wet lab data.

This obviously has not stopped the creation of a legion of startups, new Asst Profs, etc, claiming that they are using "AI" to crack new frontiers. In practice, the most effective of these use machine learning, rather than LLMs.

Maybe for very short phrases, but otherwise I disagree. Phrasing very quickly runs into a combinatorial explosion. In the words of Noam Chomsky, "Virtually every sentence that a person utters or understands is a brand-new combination of words, appearing for the first time in the history of the universe."

In my opinion, the difficulty in LLM/human text discrimination isn't that a person might coincidentally write exactly the same text as an LLM would, but rather that 1) LLMs aren't hard locked to a single phrasing (so this is a tougher problem than matching to a single static document, e.g. plagiarism detection) and 2) text has relatively low information density, so you need quite a bit of it to gather enough data to run a statistical test with a reasonably narrow confidence interval.

That's a huge assumption, and one that goes against the whole notion of using LLMs to generate text. AI slop is by far the norm.

But I suspect that a lot of academic's feelings about it are informed by what others have told them and how they've been trained, rather than by what's actually permissible in the publishing system.

Huh?

But that's easy to tell?

It's better then incompetents code, worse then a motivated average dev... But good enough hence the real question is value aka time& money invested/quality.

That's much harder to tell, and I currently think opus/fable generated code is decent enough to be safely in favor - at least on subscription

Can you unpack that a bit? It produces measurably, meaningfully inferior code everywhere I see it in use.

> "leadership encourages this because from what they can tell, there is no downside"

As said 'leadership' I find this a bit puzzling. I'm seeing strong, quantifiable evidence of increasing churn, increasing incident count, and length of downtime from the date of our biggest push into GenAI, and I'm organizing efforts on my teams to mitigate those issues and actively reduce GenAI adoption.

If you mean my c-suite, you're mostly correct although they are already rumbling about seeing zero or negative ROI on GenAI investments.

> Anyone not using LLMs all day is just not going to be as prolific

Agreed, but prolific != productive.

Hah. I've been working with ChatGPT-5.6 Sol and a lot of guidance to port a particular geometry from 2800 lines of Python-scripted Rhino3D (basically a custom parametric CAD kernel in there, plus use of it) to Python-scripted OCCT/FreeCAD (an existing CAD kernel) and it's up to 36,000 lines and only about half way there. And I've been setting goals and targets for duplication minimization, code size reduction, etc. The results are fine for my purposes, but if I put a positive value on "voluminous" I'd be super impressed; with my actual negative value assignment on lines of code, this is more "absolute crap but still useful to me."

I think "absolute crap but still useful to me" is a pretty high value and not worth neglecting, but I'm definitely scared by leadership who sees the toilet overflowng and assumes that means that value has been maximized.

There. AI-polished sentence.

Not sure if its just me, lately I have started feeling pretty offensive about the increased usage of the word. Its management not leadership by any means.

> "leadership encourages this because from what they can tell, there is no downside"

For most people in management its easier to pick the current set of slangs/abbreviation's, general trend and go with it. Understanding the details would take time, raise questions and no one in management has time or political capital to spend on it.

That's well put.

> But good enough hence the real question is value aka time& money invested/quality.

There's time invested SO FAR and time that will have to be invested to maintain it. In my experience, even with Fable, it's not there yet. It's the reason why it's easy to vibe code an app from scratch, but at some point when complexity significantly increases, the codebase becomes a mess.

Humans write slop too, you know. Just saying.

If I extrapolate this example to my professional life, this code now manages millions of dollars, a single mistake can wipe it all out, it has to be maintained by 5 other engineers and understood by 5 other domain experts.