Added!

Both the submission and your link took me way too long to see what's the issue here.

What were they even thinking? Don't they care about this? Is their AI generating all their charts now and they don't even bother to review it?

> can't put together a decent powerpoint.

probably AI generated

That one is so obviously wrong that it makes me wonder if someone mislabelled the chart, but perhaps I'm being too optimistic.

I pasted the image of the chart into ChatGPT-5 and prompted it with

>there seems to be a mistake in this chart ... can you find what it is?

Here is what it told me:

> Yes — the likely mistake is in the first set of bars (“Coding deception”). The pink bar for GPT-5 (with thinking) is labeled 50.0%, while the white bar for OpenAI o3 is labeled 47.4% — but visually, the white bar is drawn shorter than the pink bar, even though its percentage is slightly lower.

So they definitely should have had ChatGPT review their own slides.

This half makes sense to me - 'deception' is an undesirable quality in an llm, so less of it is 'better/more' from their audiences perspective.

However, I can't think of a sensible way to actually translate that to a bar chart where you're comparing it to other things that don't have the same 'less is more' quality (the general fuckery with graphs not starting at 0 aside - how do you even decide '0' when the number goes up as it approaches it), and what they've done seems like total nonsense.

Saved. Thanks for that belly laugh.

That's not really a fair comparison. Dog food has nutritive value.

I can’t believe I’ve never heard this one before. So apt.

When such technology comes out, we'll find out.

> But there was one catch: during its release, an error was made on a chart.

that should be a tell that other things may be rigged to look better than they are

There is a correlation between good communication and good outcomes. This is bad communication from the people that could maybe get us good outcomes.

After all of this hype this is the best they can do? This is the forefront company (arguable) of the forefront tech and no one can review slides before being shipped out? I think the reason why this has resonated with people is that it gives a "vibe" of not giving a shit, they'll ship whatever next slop generator they want and they expect people to gladly lap it up. Either that or they're using their own dog food and the result is this mess. Do the stats even matter anymore? Is that what they're banking on?

We're not talking about said hypothetical technology. We're talking about LLMs.

Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and we've asked you many times to stop.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

The engineers, sure. Product team... well, we've seen the past 2-3 years that AI isn't necessarily based on quality and accuracy. They are also at the top of their game in terms of how to optimize revenue.

Just because you're the best, doesn't mean you're any good

> Both the submission and your link took me way too long to see what's the issue here.

Mission accomplished for them then.

Presumably it corresponds to Table 8 from this doc: https://cdn.openai.com/pdf/8124a3ce-ab78-4f06-96eb-49ea29ffb...

If that’s the case, it’s mislabelled and should have read “17%” which would better the visual.

It makes Apple's charts look rigorous and transparent

There is a smell of desperation around OpenAI, so I wouldn't be surprised if this level of hypevibing came from the top.

The other chart on that slide was actually to scale. My suspicion is that it was super rushed to the deadline for this presentation and they maybe didn't use excel or anything automatic for the charts, so they look better, and they missed the detail due to time pressure.

> The 69.1 column has the same height as the 30.8 column. My guess is they just duplicated the 30.8 column and forgot to adjust the height to the number

Why, unless specifically for the purpose of making it possible to do inaccurate and misleading inconsistencies off this type, would you make charts for a professional presentation by a mechanism that involved separately manually creating the bars and the labels in the first place? I mean, maybe, if you were doing something artistic with the style that wasn't supported in charting software you might, but these are the most basic generic bar charts except for the inconsistencies.

Green for good on stacked bar charts is the new hotness.

Is this the moment the bubble pops (at least for OpenAI)?

GPT-5 has to be one of the most underwhelming releases to date, and that's fresh on the heels of the "gift" of GPT-OSS.

The hottest news out of OpenAI lately is who Mark Zuckerberg has added to Meta's "Superintelligence" roster.

I think the stock market has just proven time and time again that a large proportion of investors (and VCs) do basically no due diligence or critical thinking about what they're throwing money at, and businesses actually making profit hasn't mattered for a long time - which was the only thing tethering their value to the actual concrete stuff they're building. If you can hype it well your share price goes up, and even the investors that do do due proper diligence can see that and so they're all in too.

By and large people do not have the integrity to even care that numbers are obviously being fudged, and they know that the market is going to respond positively to blustering and bald faced lies. It's a self reinforcing cycle.

I don't think the PR people at OpenAI are at the top of their field.

People at the top of their field can be deeply sloppy at times.

My unjustified and unscientific opinion is that AI makes you stupid.

That's based solely on my own personal vibes after regularly using LLMs for a while. I became less willing to and capable of thinking critically and carefully.

Since everyone assumes GPT hallucinated these charts, the truth must be that they're 100% pure, organic, unadulterated human fuckups.

Maybe they asked GPT-5 to update slides.

There's only one error, bar height for o3. Somehow height uses value from 4o, which seems like some sort of copy paste error.

EDIT: I was looking just at the first chart. I didn't see there's more below.

I mean this is the industry standard. For example every time Nvidia dumps a new GPU into the ether, they do the same thing. Apple with M series CPUs. They even go a step further and compare a few generations back.

Eyeballing it, that bar looks to be around 15% in height. Typing "50" instead of "15" is a plausible typo. Albeit, one you might expect from a high-schooler giving a class presentation, not in a flagship launch by one of the most hyped startups in history.

Just remember, everyone involved with these presentations is getting a guaranteed $1.5 million bonus. Then cry a little.

The most plausible explanation IMHO is that they are moving at a million miles per hour, and someone forgot to replace some placeholder graphics.

Yea I guess I won’t immediately prescribe malice but SHEESH. One of the most anticipated product launches in years and this kind of junk made it through to the public deck. Really pretty inexcusable.

GPT-5 models are actually great models for the API, the nano model is finally good enough to handle complex structured responses and it's even cheaper than GPT-4.1-nano.

The gift of GPT-OSS that is actually Phi

It’s vibes all the way down :)

GPT-5 is probably going to be a meaningful improvement for most of my non-technical family members who like ChatGPT but have never used anything other than 4o. In fact, most of the people I know who use ChatGPT pay no attention to the model being used, except for the developers I know. This update is going to be a big deal for them.

For me, it's just another nice incremental improvement. Nothing special, but who doesn't like smarter better models? The drop in hallucination rates also seems meaningful for real-world usage.

Oh trust me I know. I worked at Palantir well before it was public and had firsthand experience of Alex Karp. He would draw incomprehensible stick figure box diagrams on a whiteboard for F100 CEOs, ramble some nonsensical jargon, and somehow close a multimillion dollar pilot. The guy is better at faking it than high-end escorts. It doesn't surprise me that this has fooled degens around the world, from Wall Street to r/wallstreetbets. Incredibly, even Damadoran has thrown in the towel and opened a position, while still admitting he has no idea what they do.

Honestly? They might be.

No. AI is a tool to make ourselves look stupid. Suggesting that it makes people stupid suggest that they are even looking at the output.

It's been fixed on the OpenAI website.

> 'deception' is an undesirable quality in an llm, so less of it is 'better/more' from their audiences perspective

So if that ^ is why 50.0 is lower than 47.4 ... but why is then 86.7 not lower than 9.0? Or 4.8 not lower than 2.1

if their AI is so good, why didn't they use it and get good results?

I mean it in the kindest way, but scientists might be the sloppiest group I've worked with (on average, at least). They do amazing work, but they're willing to hack it together in the craziest ways sometimes. Which is great in a way. They're very resourceful and focused on the science, not necessarily the presentation or housekeeping. That's fine.

AI being used to completely off load thinking is a total misuse of the technology.

But at the same time that this technology can seemingly be misused and cause really psychological harm is kind of a new thing it feels like. Right? Like there are reports of AI Psychosis, don't know how real it is, but if it's real I don't know any other tool that's really produced that kind of side effect.

It also scares me how good they are in appealing and social engineering. They have made me feel good about poor judgment and bad decision at least twice (which I noticed later on, still in time). New, strict system prompt and they give the opposite opinion and recommend against their previous suggestion. They are so good at arguing that they can justify almost anything and make you believe that this is what you should do unless you are among the 1% experts in the topic.

Doesn’t matter. Either way is bad

Does it work that well if you don’t tell it there is a mistake though?

Move fast and break investor confidence

But how could have they used ChatGPT-5 if they were working on the blog post announcing it?

>but visually, the white bar is drawn shorter than the pink bar, even though its percentage is slightly lower.

But the white bar is not shorter in the picture.

How is 50 instead of 15 a plausible typo? A zero is on the opposite end of the keyboard than a 1.

why would the placeholder graphics be the exact same but with incorrect heights on the bars?

Yeah this is somewhat stuttery on an M2 mac.

It is actually incredible how they managed to find an even more unscientific definition than "can perform a majority of economically useful tasks." At least that definition requires a little thought to recognize it has problems[1]. $100bn in profits is just cartoonishly dumb, like you asked a high schooler to come up with a definition.

[1] If a computer can perform the task its economic usefulness drops to near zero, and new economically useful tasks which computers can't do will take its place.

clearly the error is in the number, most likely the actual value is 5.0 instead of 50.0 which matches the bar height and also the other single digit GPT-5 results for metrics on the same chart

Because there share your mistake and are also sure, so instead of using basic tools that work they use AI tools that fail at the basics

"I asked GPT-5 without thinking and it said..."

I’m at a large firm of ~1000 employees, only about 25% are devs and everyone is being told they should be using LLMs/AI Agents or they are going to be behind. Even to say, if you don’t adopt these tools, we can find someone who will.

At least you're truthful when you say it's completely made up.

Here is the corrected version:

Imagine a revolutionary technology comes out that has the potential to increase quality of life, longevity and health, productivity and the standard of living, or lead to never before seen economic prosperity, discover new science, explain things about the universe, or simply give lonely people a positive outlet.

But there was one catch: during its release, an error was made on a chart. It turns out that it did not lead to the massively over exaggerated benefits that were promised and that it merely represents a minor incremental improvement over its predecessor and that it will be overshadowed in a matter of months by another release from a competitor.

Where should this community focus its attention?

Let's ignore for the moment that we're talking about a word generator that relies on an infinite amount of pirated data input to "learn" anything. Let's also ignore that the primary goal of "AGI" for the people pushing it is to replace workers en masse and to enrich themselves, and not any naive notion of progress or whatever.

So this miraculous technology that can do everything, cure diseases, reverse human aging, absolve us of our sins etc. can't accurately make a bar chart? Something kids learn in 5th grade mathematics? (At least I did, mileage might vary there)

I mean, if your whole business is producing an endless stream of incorrect output and calling it good enough, why would you care about accuracy here? The whole ethos of the LLM evangelist, essentially, is "bad stuff is good, actually".

First, it is LLMs. Second, we can focus on both the technology and the error.

> Miraculously, this technology is free to use, available to anyone with an internet connection.

If something is free but not open source, you are the product.

(@dang doesn't guarantee that we'll see anything - you need to email hn@ycombinator.com for that)

Users flagged it. We can only guess why users flag things, but in this case there had been so much coverage of GPT-5 on the frontpage, plus the chart gaffe was being extensively in those threads, that they probably found this post some combination of repetitive and unsubstantive.

It ended up spending 14 hours on the frontpage anyhow, which is quite a lot, especially for one of those single-purpose sites that people spin up for joke/drama purposes. Those are a great internet tradition but not always the best fit for HN (https://news.ycombinator.com/newsguidelines.html).

That would still be basic fail, you don't label a chart, you enter data, the pre-AGI computer program does the rest - draws the bars and slows labels that match the data

This isn't scientist sloppy, this is salesperson sloppy. Very different.

> They are so good at arguing that they can justify almost anything

This honestly just sounds like distilled intelligence. Because a huge pitfall for very intelligent people is that they're really good at convincing themselves of really bad ideas.

That but commoditized en masse to all of humanity will undoubtedly produce tragic results. What an exciting future...

> They are so good at arguing that they can justify almost anything

To sharpen the point a bit, I don't think it's genius "arguing" or logical jujitsu, but some simpler factors:

1. The experience has reached a threshold where we start to anthropomorphize the other end as a person interacting with us.

2. If there were a person, they'd be totally invested in serving you, with nearly unlimited amounts of personal time, attention, and focus given to your questions and requests.

3. The (illusory) entity is intrinsically shameless and appears ever-confident.

Taken together, we start judging the fictional character like a human, and what kind of human would burn hours of their life tirelessly responding and consoling me for no personal gain, never tiring, breaking-character, or expressing any cognitive dissonance? *gasp* They're my friend now and I should trust them. Keeping my guard up is so tiring anyway, so I'm sure anything wrong is either an honest mistake or some kind of misunderstanding on my part, right?

TLDR: It's not not mentat-intelligence or even eloquence, but rather stuff that overlaps with culty indoctrination tricks and con[fidence]-man tactics.

> Did you miss the picture below where the bar for 50% is lower than the bar for 47.4%

That one was added later, if I interpret the attribution at the bottom correctly. And I'm also pretty sure it wasn't there when I first saw it pop up.

> There are too many WebGL effects on startup websites that were built to run on a M4 Max.

Tale as old as time. When the retina display macs first came out, we say web design suddenly no longer optimized for 1080p or less displays (and at the time, 1376x768 was the default resolution for windows laptops).

As much suffering as it'd be, I swear we'd end up with better software if we stopped giving devs top of the line machines and just issued whatever budget laptop is on sale at the local best buy on any given day.

Should be fixed now.

No, i mean what is the context. Who created this originally? Where is the link to openai or whoever creating this chart, or context behind the misinformation if any. I check the comments and stories about chatgpt5 and there is no reference to this, so im at a loss.

Ok, I see there was a bug on the site and it wasn't scrolling on iOS. They fixed that now, although the background context is still unclear, and none of the links in the site seem to explain it.

Speak for yourself!

Why not using LLMs? That would be like employing hundreds of farmers and making sure they don't use a tractor and do everything by hand instead?

LLMs can be a huge performance boost, when used wisely (i.e. not just blindly using whatever they spit out)

Their field is pretty much selling sloppiness-as-a-service, tho.

I'm genuinely a bit concerned that LLM true believers are beginning to, at some level, adopt the attitude that correctness _simply does not matter_, not only in the output that spews from their robot gods, but _in general_.

I think the bar is quite low, but AI can absolutely generate "decent" powerpoints

They probably did use it and those are the charts it produced.

Maybe they did?

Either way is bad. Intentionally human made and approved is worse than machine generated and not reviewed. Malicious versus sloppy.

At my work every dev had two machines, which was great. The test machine is cattle, you don't install GCC on it, you reflash it whenever you need, and you test on it routinely. And it's also the cheapest model a customer might have. Then your dev machine is a beast with your kitten packages installed on it.

Not on a numpad. I heard rumors some actually use it ^^

Yep. It sounds more like a dictation error as “fifteen” and “fifty” sound similar. No idea why this should matter in the slide production process though.

funny isn't it - makes me feel like it's kind of over-fitted to try and be logical now, so when it's trying to express a contradiction it actually can't

We can talk a lot about how a tool should be used and how best to use it correctly - and those discussions can be valuable. But we also need to step back and consider how the tool is actually being used, and the real effects we observe.

At a certain point you might need to ask what the toolmakers can do differently, rather than only blaming the users.

Develop on a super computer, test on $200 laptop - not really any suffering that way.

I wouldn't go that far, but maybe split the difference at a modern i3 or the lowest spec Mac from last year.

It would be awesome if Apple or someone else could have an in-OS slider to drop the specs down to that of other chips. It'd probably be a lot of work to make it seamless, but being able to click a button and make an M4 Max look like an M4 would be awesome for testing.

Communication is a big part of science, so it's not great that scientists fail in this area

"ChatGPT, this slide deck feels a bit luke warm, help me make a better impression"

I could completely believe someone who is all-in on the tech, working in marketing, and not really that familiar with the failure modes, using a prompt like this and just missing the bad edit.

thats hilarious actually. gives credence to the gpt theory haha

To keep a fast feedback loop, build on the fast machine, deploy, test on the slow one.

Tbh even the absolute lowest spec Mx macs are insanely powerful, probably best to test on a low end x86 laptop.

No no no.. go one better for the Mac. It should be whichever device/s which are next to be made legacy from Apple’s 7 year support window. That way you’re actually catering to the lowest common denominator.

I only ever do. But I rephrased my post to make my meaning clearer. Nice discernment there.

Yep, fixed.

How hard would it be for the model to be like: "Okay, there's a bar chart in the picture, the left bar is 350 px in size and the right bar is 120 px. Meanwhile, the labeled values are X and Y, which doesn't seem to match those relative sizes due to this math I ran thanks to all the cool deterministic tools I have."

Apparently quite a bit.

This was a big COVID-era lesson; that places like the CDC and NIH and whatnot really need a well-trained PR wing for things like Presidential press conferences, to communicate to the public.

These charts were from the GPT-5 release stream from OpenAI. The second image is a direct screenshot:

https://www.youtube.com/watch?v=0Uu_VJeVVfo&t=1840s

they didn't get good results

But in this case, the bar was too high.

It's kinda crazy to witness, you can see in the main GPT-5 release thread that there are people excusing things like the bot being blatantly wrong about Bernoulli's Principle in regards to airplane flight. I wish I could find it again but it's thousands of comments, one of the comments is literally "It doesn't matter that it's wrong, it's still impressive". Keep in mind we're discussing a situation where a student asks the AI about how planes fly! It's literally teaching people a disproven myth!

This is wonderful. Like, the chart shows the models capability. Not the numbers in the chart, not the data presented, but the actual chart itself.

Yes, that second image was initially hidden on iOS due to the scrolling bug in the site (now fixed).

So they spotted what seems to be an unintentional error in a chart in a youtube video, and created a completely different chart with random errors to make a point, while due to their own coding error the (somewhat obtuse) explanation wasn't even visible on mobile devices.

Not sure why this was voted to the top of the first page of HN, although I can surmise.

Most people are not, in fact, wise

Is there an MCP or API to do that? Can it take a template and layout and produce coherent sentences in a consistent format?

just let it create plaintext files like latex

But the way they learn to be wise in the context of using LLMs is to try using them and fail, just like all learning experiences. Companies insisting on the use of these tools seems logical to me when the assumption is that they will, once learned, be better than previous methods of working, but only with practice.

Quarto!