You know, now that I'm thinking about it, I'm beginning to wonder if poor data privacy could have some negative effects.

Baby's gotta get some cash somewhere.

Are beta waves a sign that my mind is racing and wide awake, or are they the reason?

Perhaps the author is not a coward, but is giving the company time to respond and commit to a fix for the benefit of other owners who could suffer harm.

What makes you think this is the one?

The other side of owning equipment like this is it still could be useful for some for personal and private use.

https://xcancel.com/beneater/status/2012988790709928305

Very, but there are already tons of them at lots of different price, quality, openness levels. A lot of manufacturers have their own protocols; there are also quasi/standards like Lab Streaming Layer for connecting to a hodgepodge of devices.

This particular data?

Probably not so useful. While it’s easy to get something out of an EEG set, it takes some work to get good quality data that’s not riddled with noise (mains hum, muscle artifacts, blinks, etc). Plus, brain waves on their own aren’t particularly interesting—-it’s seeing how they change in response to some external or internal event that tells us about the brain.

The K, of course, stands for Ka-ching!

Yesterday I watched it try and work around some filesystem permission restrictions, it tried a lot of things I would never have thought of, and it was eventually successful. I was kinda goading it though.

As for the reverse engineering, the author claims that all it took was dumping the strings from the Dart binary to see what was being sent to the bluetooth device. It's plausible, and I would give them the benefit of the doubt here.

The operator is still a factor.

Google for a list of all the exceptions to HIPPA. There are a lot of things that _seem_ like they should be covered by HIPPA but are not...

I guess that’s not a huge problem, though, since all users are presumably at least anonymous.

If that's the case then they should have deferred this whole blog post.

It's good that they were responsive in the disclosure, but it's still a mark of sloppiness that this was done in the first place, and I'd like to know so I can avoid them.

I said a guess, not absolute.

That hasn't, universally, been my experience. Sometimes the code is fine. Sometimes it is functional, but organized poorly, or does things in a very unusual way that is hard to understand. And sometimes it produces code that might work sometimes but misses important edge cases and isn't robust at all, or does things in an incredibly slow way.

> They have no problem writing tedious guards against edge cases that humans brush off.

The flip side of that is that instead of coming up with a good design that doesn't have as many edge cases, it will write verbose code that handles many different cases in similar, but not quite the same ways.

> They also keep comments up to date and obsess over tests.

Sure but they will often make comments or tests that aren't actually useful, or modify tests to succeed instead of fixing the code.

One significant danger of LLMs is that the quality of the output is higly variable and unpredictable.

That's ok, if you have someone knowledgeable reviewing and correcting it. But if you blindly trust it, because it produced decent results a few times, you'll probably be sorry.

It gave up, removed the code it had written directly accessing the correct property, and replaced it with a new function that did a BFS to walk through every single field in the API response object while applying a regex "looksLikeHttpsUrl" and hoping the first valid URL that had https:// would be the correct key to use.

On the contrary, the shift from pretraining driving most gains to RL driving most gains is pressuring these models resort to new hacks and shortcuts that are increasingly novel and disturbing!

So, will they? Probably. Can you trust the kind of LLM that you would use to do a better job than the cheapest firm? Absolutely.

Identify the kickstarter product talked around in this blog post: (link)

To think some blackhat hasn't already did that is frankly laughable. What I did was like the lowest of low-bars these days.

LLMs are just surfacing the fact that assessing and managing risk is an acquired, difficult-to-learn skill. Most people don't know what they don't know and fail to think about what might happen if they do something (correctly or otherwise) before they do it, let alone what they'd do if something goes wrong.

It’s wasteful not to save and learn from those.

The lack of detail makes me suspect the truth of most of the story.

These blog posts now making the rounds on HN are the usual reverse engineering stories, but made a lot more compelling simply because they involve using AI.

Never mind that the AI part isn't doing any heavy lifting and probably just as tedious as not using AI in the first place. I am confused why the author mentions it so prominently. Past authors would not have been so dramatic and just waved their hands that they had some trial and error before finding out how the app is built. The focus would have been on the lack of auth and the funny stuff they did before reporting it to the devs.

It’s quite literally why the internet is so insecure, because at many points all along the way, “hey, should we design and architect for security?” is/was met with “no, we have people to impress and careers to advance with parlor tricks to secure more funding; besides, security is hard and we don’t actually know what we are doing, so tow the line or you’ll be removed.”

Found that in seconds. EEG, electrical stimulation, heat, audio, etc. Claims a 20 hour battery.

As to the Claude interactions, like others I am suspicious and it seems overly idealized and simplified. Claude can't search for BT devices, but you could hook it up with an MCP that does that. You can hook it up with a decompiler MCP. And on and on. But it's more involved than this story details.

Not everybody gets it.

We often treat doxxing the same way, prohibiting posting of easily discovered information.

So yeah, a product exists that claims to be a sleep mask with these features. Maybe someone could even sleep while wearing that thing, as long as they sleep on their back and don't move around too much. I remain skeptical that it actually does the things it claims and has the battery life it claims. This is kickstarter after all. Regardless, this would qualify as the device in question for the article. Or at least inspiration for it.

Without evidence such as wireshark logs, programs, protocol documentation, I'm not convinced that any of this actually _happened_.

> I stumbled upon these vulnerabilities on one of the coldest days of this winter in Vancouver. An attacker using them could have disabled all Mysa-connected heaters in the America/Vancouver timezone in the middle of the night. That would include the heat in the room where my 7-month-old son sleeps.

It's used in a enormous number of IoT devices.

The "IoT gateway" service from AWS supports MQTT and a whole lot of IoT devices are tethered to this service specifically.

The LLM got it to “working” state, but the people operating it didn’t understand what it was doing. They just prompt until it looks like it works and then ship it.

If we applied this similar analogy to a e.coli infection of foods, your recommendation amounts to "If we say the company name, the company would be shamed and lose money and people might abuse the food".

People need to know this device is NOT SAFE on your network, paired to your phone, or anything. And that requires direct and public notification.

(Also, "We're not happy until you're not happy.")

The parents are saying they'd rather vibe code themselves than trust an unproven engineering firm that does(n't) vibe code.

You could cut the statement short here, and it would still be a reasonable position to take these days.

LLMs are still complex, sharp tools - despite their simple appearance and proteststions of both biggest fans and haters alike, the dominating factor for effectiveness of an LLM tool on a problem is still whether or not you're holding it wrong.