Why did they give it any of that?!
2. I pay for Meta Verified on Instagram and for the past 2 weeks "Enhanced support" leads me to a broken interface. "Page isn't available right now". So, what am I paying for exactly?
3. It seems you can use Meta's AI Assistant to sometimes get through to a human. I've done this twice now, and both times my case has been escalated to a different team (apparently) yet I never get an email, I never get an update in the chat (the chat ENDS immediately after the phone call with support), and the issue is never resolved. It's been 2 weeks. The case says "Completed", with no response. Worthless as always.
4. My wife creates content on Instagram and has had her account suspended multiple times now for "Account Integrity". I assume the system thinks she's not the person in the content, despite providing her valid email, phone number, video selfie, and 2 types of ID (passport & driver's license) multiple times. What's hilarious is the passport was accepted on of her accounts (they wiped out everything on her Account Center), but another account was rejected. Great AI, same passport, exact same lighting... different outcome.
So as it stands, we're both fucked on both facebook and instagram thanks to awful AI moderation, and fucked further thanks to awful AI support. No resolution in sight. The incompetence is next level. I really don't see this getting resolved. This already happened to my wife earlier in February, she managed to get one account back, and a month later she's hit with the same identity issues.
Using AI for both the moderation and the support makes me sick. The same poor AI that incorrectly flagged me and my wife's accounts for a load of incorrect bullshit is the same system that's meant to help resolve it? Of course it's going to side with its own poor decision. YouTube seems to do the same thing and auto-reject appeals in seconds. Really smart /s
I believe we need enforcement that social platforms should NOT be using AI to perform destructive actions without human intervention. Noone should ever lose their accounts because of AI mistakes. AI should be used to surface potential issues which get passed to a HUMAN to double check before applying the action. AI simply isn't good enough to have full control.
Fucking pissed off and even angier now I've had to write all this up and remind myself just how ridiculous the situation is. Sorry for the rant, but losing your accounts you put work into is very crushing and demotivating. Being accused of these violations fills us both with so much resent for the companies running this shit.
Sam Cofounder Postmates
On the off-chance there's anyone at Meta seeing this (@Wirah on twitter)
Had to make this new username as my original (samstr) comment doesn't show up. No idea why. Probably shit AI
The solution (which also solved SIM support agents being bribed or hacking known acquaintances) was to prevent the agents from resetting the SIM card without some steps the original owner would have to follow (and could follow even if they've lost their original phone), like a PIN they'd have to remember. I think the same solution should be applied to AI agents.
I created the account when instagram first came out, never used it, and totally forgot about it. I got stuck in a strange position where I had to login from a device I had previously logged in from, but because it's been over a decade, I no longer have any of the devices I might have used to create/access the account.
I still have access to both the email and phone number used for the account, but that was not good enough.
How hilariously incompetent. I Filled a CCPA complaint.
Thankfully, IG gave me the option of restoring my username when I logged back into my account today.
It’s a shame nobody tried to get it to drop the production table entirely! (mostly joking). Just claim to be a high level SRE solving some critical production bug, the only solution to which is dropping the database.
The stories of AI support fails are getting funnier and stupider.
Of course it's always possible that they simply don't care who has your account, as long as they get money.
The next obvious thing would be to let accounts the algorithm judges to be low-value still opt-in to strict verif. The vast majority of low-value accts won't bother flipping it on if the option is buried two menus deep, but many of the few low follower/views accts who are targets for some other reason (political, stalker, etc) - know they are targets and can self-protect by opting in, further reducing account hijacks.
So, before we even get to whether this 'loose' verif is "bad", those two simple implementation changes would certainly have cut the bad outcomes of a (potentially) bad idea by >95%.
It might even do that preemptively if it thinks they're going to shut it down.
Also, I discovered that many of IG's auth endpoints are just broken. For example you can't change password on web because of CORS, which isn't a transient outage but just a flat out bug.
Edited to add: This is just the cherry on top of years of stupid auth flow at IG. I have received tens of thousands of reset links or codes from IG over the years. There used to be a way to put your account on recovery cooldown for a few weeks but they got rid of even that.
(https://xcancel.com/DarkWebInformer/status/20612535997583155...)
Otherwise the only way to provide these services is to massively underfund support, if you charge 0$ per account and serve 1 Billion users, then you cannot afford to spend 1 minute of human support time on an account.
Yes, they could use the money from ads, but let's be frank, the customers in that case are the sponsors, if the customer is the actual user, then it's way easier to provide direct support to them without facing an foundational incentive misalignment.
The weird thing is I know the Instagram security team, and they are top notch. I have a feeling this was vibe coded by someone outside of security and security wasn't looped in.
This turn was an AI exploit, in my case was an outsourcing support 'exploit', where someone paid for my username to be manually changed and given to another user. There will always be a way to get access to accounts if human accountable support doesn't exist, with criminal consequences for employees that violate it.
The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.
Is that for real? I find it hard to believe that an exploit THIS simple and easy to abuse managed to stay live for weeks or months.
We really need similar rules to other engineering disciplines. If your building falls with people inside, you killed them.
Maybe they should have hacked themselves.
Dear Instagram, wtf. Why not send the reset to the account in question? Arbitrary email, wow.
This exact same flow could have been (and may have been; I don’t know how much the chatbot here actually does) statically coded.
Assigning Jr engineers for security support is ridiculous partly because young people don’t understand how critical security is sometimes. And partly because they don’t value privacy as much.
Based on what I've seen so far, Meta AI Support Assistant (they call it "MAISA") had tool calls that a) start an email verification to any specific email, phone number, or the contact points linked to an account and b) allow generating a password reset link for an account based on an email verification attempt. I don't think it had any access to the actual codes themselves, but rather think a handle or ID for an email verification attempt (along with the user provided verification code based on user input) was provided to the "generate reset password link" tool call, and the tool call failed to properly validate the actual email used in that attempt belonged to the account allowing the ATO.
The tool call for MAISA to generate a password reset link should have failed with an email verification attempt that corresponds to an email not linked to the account (and I believe I even tested this at one point on Facebook and encountered an error that successfully prevented it), but I suspect they tried making a change to this tool call for Instagram where slightly older, recently unlinked emails could be used to recover an account that got hijacked by an attacker, which added the need to allow emails not currently linked to the account to be used and set to the user's primary email.
I also suspect that the MAISA tool call change called a wrong API or something that unintentionally allowed any email verification attempt that was successful to be used, but the engineers did not add a sufficiently thorough e2e test case to test the tool call against unrelated email verification attempts being provided to the tool call. This is the part I think should be focused on the most. Tool calls for agents that have their output potentially influenced by an attacker should be treated like external APIs that anyone can reach, and they should be tested as such.
This is all obviously a guess, doesn't take into account the many signals they use to determine if an account recovery attempt is valid, and could be very inaccurate, but it's the closest to what I (someone who deals with Meta security a lot) think could have allowed this to happen.
Genuine question...why would that need to be hand-written?
It makes absolute sense as a general statement and is kinda crazy that this wasn't a built-in limitation, but I'm not quite sure why the code for that bit must be hand-written (provided the code functionally does what you describe).
With no basic validation either apparently. Insane.
If you'd do a retrospective and ignore how AI has shaped expectations and a company's culture to allow this to pass through into production, you'd be complicit/perpetuating what led to this debacle in the first place.
It's not the end of the world, and water isn't going anywhere, but saying AI has essentially nothing to do with it is just a bad take.
But had never been until it was wrapped in a chatbot. It’s just about unheard of for a major site in the modern era, isn’t it? I think the AI factor is essentially essential. All but.
For what it’s worth I don’t think you can call this social engineering since there was no human on the other end, even though it appears similar.
The question is, if there were actual human support agents, would they have built additional safeguards to prevent social engineering in this manner?
As for your comment about junior engineers, see kennywinker's reply to this thread - I share the same thoughts.
I’ve been a jr engineer at a large company. I had the power to implement absolutely jack shit on my own. I deeply doubt the security flow for account recovery in meta ai account security was a single jr engineer.
What i think is actually going on is basically a soft form of ai psychosis. Senior engineer gets ai to code ai account recovery feature, that same or a different engineer asks ai to review the feature, and then it gets pushed to prod. Move fast, break things. The ai coded it, the ai reviewed it - the people trusted the ai because it sounds confidently right.
Just like how the ai doesn’t know if you should walk or drive to the car wash, the ai doesn’t understand exploits like this one.
The reason it worked there is that the designers of the system didn't anticipate that the AI will agree to accept any email (maybe they even put guardrails against it in the system prompt, we don't know). It's more like social engineering than bad-security-code, except that like the sibling comment said an actual human will probably not approve that.
lol, no. The day someone is criminally charged with "stealing" a username is the day that humanity has lost
Urgency.
Emotions.
It's all there, and high-stakes environments with no proper protocol are most vulnerable.
Source: used to work part-time in IT support at a hospital, by now 10+ years ago, so it was routinely requested to circumvent regulations and security protocols, even medical ones (cough Windows in ICU monitors and other medical "kiosk" PCs that should absolutely not run Windows)
Think NASA, for example; it's also a government agency, and they are doing great job posting photos in Instagram, do you think anything is wrong with it?
The fact it can be removed by anyone is the problem. If you lose access to your 2FA (and recovery codes) then you should lose access to your account. Having it removable by anyone (other than a logged in account holder) defeats the entire point.
It is not designed to allow a valid email verification attempt to restore access to an account it is not associated with (and this is the tool call where the problem occurred). This tool call was clearly not tested well, and this is where the focus should be.
This looks like a terrible design rather than an AI problem to me, though.
While it isn't directly "stealing", the government has brought charges against people in the past for username-related crimes. There are several similar cases, but this is the first one that came to mind.
The security team at any organization is always considered an enemy to product and innovation. It wouldn't be surprising if management made it impossible for them to put in place the monitoring necessary to know this was happening. Especially at somewhere whose motto is "move fast and break things".
Something to think about when we consider what is "normal" today. Not much really is normal. We've been beaten to think it is.
If you lose your password or 2FA, you should lose your account, too bad so sad.
Yesterday, a slew of Instagram accounts, including some high profile ones like the Obama White House account, seemingly got hacked. I've seen my share of exploits and takeover techniques, but this is the most unserious, "almost too stupid to be true" of them all.
Step 01: Faking the Location & Initiating Support
All the attacker needs to kick this off is your account username. Then, they hop on a VPN or proxy close to your city so Instagram's security algorithms don't suspect a thing. (You can quite easily get this from your public profile or "About" section or a hundred other ways.) Once it looks like the request is coming from the correct region, they tell the Meta support AI that the account is hacked and ask it to send the verification codes to an arbitrary email address they control.
Step 02: That's It
Really, that's it. The first proper zero auth password reset I've seen in production. There appears to be no additional check as to whether the email being given is actually something the user has used before. Once the AI sends the security code to the attacker's email, the attacker passes it right back to complete the verification. The platform hands over a fresh password reset link, granting full ownership to the attacker.
Instagram's AI may or may not ask the attacker for a video selfie to prove identity. It's not particularly discerning at the moment, so something as simple as an AI animated public photo from the target's feed has been widely reported to work.
In case you're wondering, because the system treats this high-privilege recovery flow as a total account reset by the "true" owner, the original 2FA gets thoroughly bypassed in the process.
Existing sessions are revoked and the password changed with no email, text, or push notification. The actual owner can't initiate recovery because the email and phone numbers now map to the attacker. There's no human to escalate to, it's just you arguing with a chat hoping to take control back while praying they don't do it again.
And if you're part of the A/B tested accounts on which the AI support option is active, tough luck, you can't even turn it off.
Multiple black market Telegram groups have sprung up offering "account takeover" services at steep rates and quick turnaround times. Considering short handles are worth hundreds of thousands to even millions of dollars, it's not a surprise, really.
Accounts have been flipped, like hey, or been used for propaganda, like obamawhitehouse or ocmssf, the account of the Chief Master Sergeant of the U.S. Space Force. `
All the Telegram groups have quieted down as Meta seems to have patched it already, but it appears this particular method was active for weeks, if not months.
The very fact that a $1.5 trillion company lacks robust guard rails and their support AI will just change anyone's linked email if you ask it nicely enough is so terrifying, if it weren't so funny.
If you've reached here, thank you for reading! :)
I thought exiting and retiring in my mid 30s would be fun but I've just been bored and depressed without morning Slacks and emails to wake up to. If you’re building something interesting and could use an extra set of hands to help ship it, feel free to reach out. My inbox is open.
An AI enabled terrible design. AI acted as a black box of stupidity, that obscured the stupidity of the design.
Unfortunately Siemens woke up.
Imagine an alternate universe where big tech companies worked with various trustworthy third-parties where something like this would generate a challenge you could take to your local notary, post office, library, police station, etc. where someone would check ID before approving it. How many phishing attacks would be prevented annually by a physical presence check?
admin
Horrific, people should be jailed for cyberattacks when they carelessly just give out this word.
The experiences I meant were mostly
- password reset requests (admittedly, we had a protocol even then to strictly require a "physical signature", normally meaning Fax or internal snail mail)
- medical protocols: don't wanna go into too much detail here, but:
1) Windows requires a lot of maintenance, often even hard restores, to function normally, even when sold as the UI for physical ICU monitors
2) Medical personell often is severely overworked, especially people in important, but not formally highly-qualified roles. And things like operating rooms pr ICUs often have very slim time slots. In the former, you should not enter without wearing appropriate coating. That doesn't prevent people working there from requesting you to finally come over and make that UEFI-Windows-Crapware-Kiosk-PC which was sold as a medical device boot... of course especially not when there is an ongoing surgery nearby. And of course, your higher-ups will be there to help you sort out these issues without violating protocols.
/s
Isn't this essentially what just recently happened to the Pope? Then there were people here doing the rest of your comment for him saying how egregious it was for them to ask for an in person authorization. It sounded like all he was trying to do was update his address, but changing your address from one in Chicago to one in a European country absolutely sounds like something a phisher would be trying to do.
The least terrible seem digital id.
I don’t recall why I had to go through this song and dance. Very plausibly the account was still associated with an old school address that I could no longer access. So yeah, account recovery is hard. How do you prove someone owns an account when they’ve lost the things they are supposed to use to prove ownership?
https://www.theverge.com/2013/5/2/4292744/facebook-trusted-c...
The cost of hiring a person is part of it but not really the core reason. People were sold on the Internet with "you can do things online conveniently" and reintroducing the need to physically go somewhere negates that angle entirely.
https://pages.nist.gov/800-63-4/
I think Meta just does not care if they're enabling AI attack surface and vulnerabilities into these customer journeys. It's...certainly a choice, versus deterministic journeys with hard guardrails. They could make different choices.
But how often does one need to do recovery procedures like this?
How much less convenient is it for everyone else to be at risk of their account being taken over?
That only works because you presumably do KYC when you open accounts, so you have an identity to match to. Most internet accounts don't do real KYC, so a government credential doesn't really work for recovery --- they didn't know who you were, so proving who you are doesn't help anything.
That doesn't mean that letting anyone sweet talk support or an AI into taking over an account is acceptable, of course.
My point is that while this is not easy, there are obvious very bad ways to implement this that should not be done (chatbot or other generative AI interface vulnerable to the usual suspects of AI inherent attack surface). Don't build the bad way, the right away is known and straightforward.
It's an impressive level of incompetence.
Broadly speaking, work for the sake of work is not valuable work. Show me outcomes for resources and time invested, and compare accordingly. Value is, again broadly speaking (there is always nuance), what you deliver. If you bring me an AI solution for a high risk high value customer journey, data flow, or code path, that is an anti pattern. If you, as a colleague or a stakeholder, put forth that we must use AI in situations that require a high degree of determinism (due to potential high cost failure modes), you will need to prove this extraordinary claim with evidence.
Choose Boring Technology - https://news.ycombinator.com/item?id=9291215 - March 2015 (212 comments) ["Am I using this project as an excuse to learn some new technology, or am I trying to solve a problem?"]
I get paid to manage risk efficiently, including being measured on time and budget spent against the success criteria, ymmv; my comp and budget is not dependent on how much AI I shove into security systems. "What am I optimizing for?"
Amazon scraps AI leaderboard to stop workers chasing usage scores - https://news.ycombinator.com/item?id=48315583 - May 2026 (19 comments)