That is, it's nice to make a pretty stand-alone image, but without tools to maintain consistency and place them in context you can't make a project that is more than just one image, or one video, or a scattered and disconnected sequence of pieces.

Maybe everyone has a different dose of skepticism. Personally I'm not even looking at results for models that were released after the benchmark, for all this tells us, they might as well be one-trick ponies that only do well in the benchmark.

It might be too much work, but one possible "correct" approach for this kind of benchmark would to periodically release new benchmarks with new tests (that are broadly in the same categories) and only include models that predate each benchmark.

https://mordenstar.com/blog/edits-with-nanobanana

In particular, NB Pro successfully assembled a jigsaw puzzle it had never seen before, generated semi-accurate 3D topographical extrapolations, and even swapped a window out for a mirror.

Question: Now that the steamboats have been invented, how do I keep my clipper business afloat ?

Answer: Good riddance to the broken idea of IP, Schumpeter's Gale is around the corner, time for a new business model.

Back in reality, you can get in line to sue. Since they have more money than you, you can't really win though.

So it goes.

also in the tweet

> GPT Image 1.5 is **ing crazy

and

> holy shit lol

what's impressive if you don't know if it's right or not (as the other comment pointed out, it is not right)

My own main use cases are entirely textual: Programming, Wiki, and Mathematics.

I almost never use image generation for anything. However its objectively extremely popular.

This has strong parallels for me to when snapchat filters became super popular. I know lots of people loved editing and filtering pictures but I always left everything as auto mode, in fact I'd turn off a lot of the default beauty filters. It just never appealed to me.

I don't have any captcha systems in place, but I wonder if it might be worth putting up at least a few nominal roadblocks (such as Anubis [1]) to at least slow down the scrapers.

A few weeks ago I actually added some new, more challenging tests to the GenAI Text-to-Image section of the site (the “angelic forge” and “overcrowded flat earth”) just to keep pace with the latest SOTA models.

In the next few weeks, I’ll be adding some new benchmarks to the Image Editing section as well~~

[1] - https://anubis.techaro.lol

Also: SUPER ANNOYING. It seems every time you give it a modification prompt it erases the whole conversation leading up to the new pic? Like.. all the old edits vanish??

I added "shaky amateur badly composed crappy smartphone photo of ____" to the start of my prompts to make them look more natural.

Counterpoint from someone on the Musk site: https://x.com/flowersslop/status/2001007971292332520

One thing that gpt-image-1 does exceptionally well that Nano Banana (Pro) can't is previz-to-render. This is actually an incredibly useful capability.

The Nano Banana models take the low-fidelity previz elements/stand-ins and unfortunately keep the elements in place without attempting to "upscale" them. The model tries to preserve every mistake and detail verbatim.

Gpt-image-1, on the other hand, understands the layout and blocking of the scene, the pose of human characters, and will literally repair and upscale everything.

Here's a few examples:

- 3D + Posing + Blocking: https://youtu.be/QYVgNNJP6Vc

- Again, but with more set re-use: https://youtu.be/QMyueowqfhg

- Gaussian splats: https://youtu.be/iD999naQq9A

- Gaussians again: https://youtu.be/IxmjzRm1xHI

We need models that can do what gpt-image-1 does above, but that have higher quality, better stylistic control, faster speed, and that can take style references (eg. glossy Midjourney images).

Nano Banana team: please grow these capabilities.

Adobe is testing and building some really cool capabilities:

- Relighting scenes: https://youtu.be/YqAAFX1XXY8?si=DG6ODYZXInb0Ckvc&t=211

- Image -> 3D editing: https://youtu.be/BLxFn_BFB5c?si=GJg12gU5gFU9ZpVc&t=185 (payoff is at 3:54)

- Image -> Gaussian -> Gaussian editing: https://youtu.be/z3lHAahgpRk?si=XwSouqEJUFhC44TP&t=285

- 3D -> image with semantic tags: https://youtu.be/z275i_6jDPc?si=2HaatjXOEk3lHeW-&t=443

I'm trying to build the exact same things that they are, except as open source / source available local desktop tools that we can own. Gives me an outlet to write Rust, too.

Air gap. If you don’t want content to be used without your permission, it never leaves your computer. This is the only protection that works.

If you want others to see your content, however, you have to accept some degree of trade off with it being misappropriated. Blatant cases can be addressed the same as they always were, but a model overfitting to your original work poses an interesting question for which I’m not aware of any legal precedents having been set yet.

I can tell you with 100% certainty they are not. For example, Crash doesn't have a backside for his torso. You could definitely make a model that uses these as textures, but you'd really have to force it and a lot of it would be stretched or look weird. If you want to go this approach, it would make a lot more sense to make a model, unwrap it, and use the wireframe UV map as input.

Here's the original Crash model: https://models.spriters-resource.com/pc_computer/crashbandic... , its actual texture is nothing like the generated one, because the real one was designed for efficiency.

In late stage capitalism you pay for fake photos with someone. You have chat gpt write about how you dated for a summer, and have it end with them leaving for grad school to explain why you aren't together.

Eventually we'll all just pay to live in the matrix. When your credit card is declined you'll be logged out, to awaken in a shared studio apartment. To eat your rations.

At the least, it's not present in these new images.

They forgot to calibrate the cameras, so everything had a green tint.

Meanwhile all the other teams had a billion macbeth charts lying around just in case.

And I say "subtle" - but because that model would always "regenerate" an image when editing, it would introduce more and more of this yellow tint with each tweak or edit. Which has a way of making a "subtle" bias anything but.

Updated the mirror test to use the NB Pro version.

You were fine until this bit.

You got to play the copyright game when the big corps were on your side.

Now they're on the other side. Deal with it and get over it.

I edited the original "Lord of War" poster with a reference image of Jensen and replaced bullets with GPU dies, silicon wafers and electronic components.

https://x.com/singhkays/status/2001080165435113791

So I don't think there's even a question of whether or not newer models are going to be maximizing for benchmarks - they 100% are. The skepticism would be in how it's done. If something's not being run locally, then there's an endless array of ways to cheat - like dynamically loading certain LoRAs in response to certain queries, with some LoRAs trained precisely to maximize benchmark performance. Basically taking a page out of the car company playbook in response to emissions testing.

But I think maximizing the general model itself to perform well on benchmarks isn't really unethical or cheating at all. All you're really doing there is 'outsourcing' part of your quality control tests. But it simultaneously greatly devalues any benchmark, because that benchmark is now the goal.

There seems to be no other way (apart from air-gapping everything, as others say).

Generate a novel previz scene programatically in Blender or some 3D engine, then task the image model with rendering it in a style (or to style transfer to a given image, eg. something novel and unseen from Midjourney). Another test would be to replace stand in mannequins with identities of characters in reference images and make sure the poses and set blocking match.

Throw in a 250 object asset pack and some skeletal meshes that can conform to novel poses, and you've got a fairly robust test framework.

Furthermore, anything that succeeds from the previz rendering task can then be fed into another company's model and given a normal editing task, making it doubly useful for two entirely separate benchmarks. That is, successful previz generations can be reused as image edit test cases - and you a priori know the subject matter without needing to label a bunch of images or run a VLM, so you can create a large set of unseen tests.

[1] https://imgur.com/gallery/previz-to-image-gpt-image-1-x8t1ij...

Add an anonymizing scheme (blind signatures or group signatures), done.

Comparing NB Pro, GPT Image 1, and GPT Image 1.5

https://genai-showdown.specr.net/?models=o4,nbp,g15

What a prompt and image.

Ludicrously unnecessary nitpick for "Remove all the brown pieces of candy from the glass bowl":

> Gemini 2.5 Flash - 18 attempts - No matter what we tried, Gemini 2.5 Flash always seemed to just generate an entirely new assortment of candies rather than just removing the brown ones.

The way I read the prompt, it demands that the candies should change arrangement. You didn't say "change the brown candies to a different color", you said "remove them". You can infer from the few brown ones that you can see that there are even more underneath - surely if you removed them all (even just by magically disappearing them) then the others would tumble down into a new location? The level of the candies is lower than before you started, which is what you'd expect if you remove some. Maybe it's just coincidence, but maybe this really was its reasoning. (It did unnecessarily remove the red candy from the hand though.)

I don't think any of the "passes" did as well as this, including Gemini 3.0 Pro Image. Qwen-Image-Edit did at least literally remove one of the three visible brown candies, but just recolored the other two.

hopefully this leads to greater importance of seeing things with your own wetware.

A professional artist wouldn't know what you want.

You didn't even specify an art style. 1970s sci-fi novel cover isn't a style. You'll find vastly different art styles from the 70s. If you're disappointed, it's because you're doing a shitty job describing what's in your head. If your prompt isn't at least a paragraph, you're going to just get random generic results.

Most of Crash in the first game was not textured; just vertex colours. Only the fur on his back and his shoelaces were textures at all.

That would be misrepresentation. Even Stallman isn't OK with that. You can take one of his opinion pieces and publish it as your own. Or you can attach his name to it.

However, if you're editing it and releasing it under his name, clearly you're simply lying, and nobody is OK with that. People have the right to be recognized as authors of things they did author (if they so desire) and they have a right to NOT be associated with things they didn't.

> At the end of the day it’s very gross when we are exploited without our knowledge or permission so rich groups can get richer.

The second part is the entirety of the problem. If I'm "exploited" in a way where I can't even notice it, and I'm not worse off for it, how is it even exploitation? But people amassing great power is a problem no matter if they do it with "legitimate" means or not.

Where switching will be easier is with casual chat users plus API consumers that are already using substandard models for cost efficiency. But there will also always be a market for state of art quality.

https://arstechnica.com/ai/2025/12/disney-invests-1-billion-...

https://mordenstar.com/other/z-image-turbo

And if that still doesn't get you there, hash the image inputs to detect if its one of these test photos and then run your special test-passer algo.

As Gemini has gained competitiveness (higher confidence in its output, better reputation), its prices have steadily risen

https://www.engadget.com/ai/google-pulls-ai-generated-videos...

The next step is to take this beyond AI generations and to license rights to characters and IP on social media directly.

The next salvo will be where YouTube has to take down all major IP-related content if they don't pay a licensing fee. Regardless of how it was created. Movie reviews, fan animations, video game let's plays.

I've got a strong feeling that day is coming soon.

Meanwhile, the next generation of great artists is already at work down the street from you. Some kids you've never heard of, playing around in a basement or garage you've probably driven past a hundred times. They're learning to make the most of the tools at hand, just like the old masters did. Except the tools at hand this time are little short of godlike.

It's an exciting time. If you wanted things to stay the same, you shouldn't have gone into technology or art.

SWE: "Seriously? import PIL \ read file \ == (c + 10%, m = m, y = y, k = k) \ save file done!"

Exec: "Yeah, and first blogger get's a hold of image #1 they generate, starts saying 'Hey! This thing's been color corrected w/o AI! lol lame'"

Or not, no idea. i've not understood the choice either, besides very intelligent AI-driven auto-touch up for lighting/color correction has been a thing for a while. It's just, for those I end up finding an answer for, maybe 25% of head scratcher decisions do end of having a reasonable, if non intuitive answer for. Here? haven't been able to figure one yet though, or find a reason/mention by someone who appears to have an inside line on it.

Look again at Gemini's output, it looks like an actual book cover, it looks like an illustration that could be found on a book.

It takes on board corrections (albeit hilariously literaly).

Look at GPT image's output, it doesn't look anything like a book cover, and when prompted to say it got it wrong, just doubles down on what it was doing.

Stallman has his opinions on software, I have my opinions on my visual work. I don’t get really how that applies here or why that settles this matter.

In addition to giving models multiple attempts to generate an image, we also write several variations of each prompt. This helps prevent models from getting stuck on particular keywords or phrases, which can happen depending on their training data. For example, while “hippity hop” is a relatively common name for the ball-riding toy, it’s also known as a “space hopper.” In some cases, we may even elaborate and provide the model with a dictionary-style definition of more esoteric terms.

This is why providing an “X Attempts” metric is so important. It serves as a rough measure of how “steerable” a given model is - or put another way how much we had to fight with the model in order for it to consistently follow the prompt’s directives.

You will note that the Minimum Passing Criteria allows for a color change in order to pass the prompt but with the rapid improvements in generative models, I may revise this test to be stricter, only allowing "Removal" to be considered as pass as opposed to a simple color swap.

If you look at the ones that passed (Flux.2 Pro, Gemini 2.5 Flash, Reve), you'll see that they did not add/subtract/move any of the pockmarks from the original image.

That's such a bad straw man I wonder if you're really supporting the position you claim to be supporting. Maybe you're just trying to give it a bad name.

Your opinion isn't on visual work, but visual property. You don't demand to be paid for your work - your labor. Rather you traded that for the dream of being paid rent on a capital object, in perpetuity (or close enough). Artists lost to the power-mongers when we bit at that bait.

I don’t really know where all the hostility came from in this conversation but I think it’s best if we move on.

GPT Image bombed notably worse than the others, not the original picture itself, but the complete lack of recognition of my feedback that it hadn't got it right, it just doubled down on the image it had generated.