And my point here is, we're in this great revolution of AI empowered everything and... these companies can't be bothered to scan this volume of AI ad slop to make sure we're not being actively scammed? Or having porn or CSAM spread? I understand it's a "perverse incentive" in the truest sense: they make money by taking the scammers money, but I was scammed once by an insanely clever lookalike Facebook ad, and had trusted their verified checkmark (about a decade ago)... with AI, these things should be a thing of the past.
Relax.
It's only a crime if someone else does it or is not sufficiently wealthy, otherwise it's a mistake.
This truly is a case where AI is the solution. There's no way to automate this at scale with humans. As AI gets better hopefully detection and moderation will catch up
There really needs to be some harder punishments for this kind of behavior, after all meta profits by selling scammy ads.
Meta knowingly runs scam ads, they just charge more to run them: https://www.reuters.com/legal/litigation/consumer-watchdog-g...
> Reuters also reported that Meta internally projected in 2024 that it would earn about 10% of its overall annual revenue – or $16 billion – from running ads for scams and banned goods, internal company documents show.
At $employer we're CONSTANTLY fighting fake ads cloning our brand. Their various brand protection features seem deliberately inept and ineffective.
https://www.engadget.com/2231100/meta-apps-displayed-ads-tha...
I wonder if we could learn anything from these cranky old timers.
So it's the SAME AD PLATFORM? Even in MESSENGER? My god. Not only is this whole thing horrible but it's doubly so
Are there reasons why Brexit might have been popular? Yes, in the sense I've heard about them. Should we then be blaming Facebook?
The problem is consolidated power. Carole herself said in her talk that it's not about Trump vs no Trump (sorry. It's been a bit since I've seen the talk. I shouldn't just paraphrase), it's about citizens' right to meaningfully choose in an information landscape
This is the problem with Facebook. It becomes an Everything App. Who cares about fraud (say, on marketplace, needless to say the site itself, the way it's screwed over businesses) on the platform when you can grow big? Let's say in this case, the actual case I'm commenting on - "Facebook didn't know". Well then that's the problem, isn't it? It's a company set up by someone of low moral reasoning (https://news.ycombinator.com/item?id=1692122) that doesn't set out to make a good product, something people will actually like. It's about dominating an industry and ads can be totally unvetted or scummy or scammy if it enables them to do this
I didn't realize the original was behind a paywall.
If Zuckerberg wants total voting control, then he can also deal with the consequences of being caught hosting CSAM.
If you are downvoting this please explain why, it’s important that we come to a shared understanding how these things are ever okay for corporations and not individuals…
Is this new term broader than the old term of CP or is it the same? Why did we even switch?
Search something rather innocuous like "news". Click the first thing then go back to the main page. You have to watch through the ad.
The amount of slop you will get suggested as soon as the algorithm warms up is nuts. The ad you are first served might also be slop. I mean AI generated video advertising overt bogus cures (a memorable one recently - "did you know the Bible has the answer to Alzheimer's?") over AI slop footage.
When I compare to my normal/auth'd Youtube experience, it feels like a totally different app. I have never really experienced such a cheap illustration of the "Youtube rabbit hole" effect. The degree to which AI authored advertising is participating in this is imo pretty concerning if you have, in particular, older parents or relatives.
Again, and again, and again, these technologies have proven ineffective to moderate content. User posted and otherwise. Until we have a legal mechanism in place to hold these companies meaningfully liable for this, it will not stop.
Every single one of these has the same damn list of excuses about cost of hiring human moderators, but I promise everyone reading this if you make the executives of a social network legally liable for distributing CSAM on their platforms, they will find all kinds of money to get human moderators really fucking quick.
Step 2: wait for it to get out of the news cycle.
Step 3: donate heavily via lobbyists to the opponents of the politicians who called you out
Step 4: de-fund the unit down until just barely enough that you're not lying when you say you "have a team working very hard on the problem." A dozen people in some third world country you pay a dollar a month each, for example.
Tiny providers have to scramble, because they'll be destroyed by a single failure to handle it fast enough.
The purpose of the new term was to emphasize that producing it involves the actual abuse of an actual child.
And then somehow it got applied to drawings and lately computer-generated stuff.
For a lot of people, government regulation of this sort is not desirable, but at what point does someone need to drop the hammer to stop this?
Results:
- Reporting a message does nothing no matter how much the content breaks their terms of use. 100% of messages I reported bounced back in a matter of minutes with "no action"
- However, using the "request another review" function, which is available after the initial "no action" response, actually seems to work. How quickly this works seems to vary, I've had actions taken as result of this from 1 day to multiple weeks.
You don't have Metas legal team nor does the power structure find your valuable to have helping them out nor can you harm them if you don't like them.
There was, quite famously, a man who got away with these sorts of things for quite some time. He owned an island. I can't recall his name....
This is why they pushed for the change from “child porn” to “child sexual assault material” because LOTS OF THINGS can be “materials”.
Don’t buy into Meta’s narrative that there’s nothing that can be done while they continue to make money hand over fist.
"Sexual exploitation is horrific, and we work aggressively to keep it off our platform" a Meta [which in the past 2 years has laid off and disbanded all teams responsible for this] spokesperson says.through meta might have lobbied on both
but they lobby for specific (often pretty bad) forms of age verification to be able to
- spy on people better
- have a regulatory moat where due to beady designed laws new alternatives have a much higher hurdle/initial investments and in turn never get anywhere
no, it's just more work, and it's not like facebook could not afford to pay more people
Is this firehose of social media actually a societal benefit? I’m less and less convinced that it is.
Reported the add but damn YouTube, the freaky fake medical ads are bad enough.
“Just stop using meta properties then”. Sure but some of us have wives, friends, etc
It's Facebook's problem if Facebook can't do the things Facebook is required to do. "Oh no it is hard"? Tough. Either they find a way or find they're no longer allowed to do this business.
> This truly is a case where AI is the solution. There's no way to automate this at scale with humans. As AI gets better hopefully detection and moderation will catch up
AI *might* solve this, or Meta's AI may be trapped in an arms race with advertisers trying to get around it for whatever reason has driven them to do this in the first place; but fully automated mass-generation of personalised ads was never a mandatory part of their business plan, so it was a problem they never needed to have.
If I had to guess, this is to try and distance child sexual assault from the concept of "pornography", and maybe make it more abundantly clear that "child porn" is always, 100% of the time, sexual abuse.
I dunno, language evolves like this sometimes. Like "smoking" which is now called "tobacco use" or "cannabis use" to clarify more what is being smoked
>The amount of slop you will get suggested as soon as the algorithm warms up is nuts. The ad you are first served might also be slop.
This did not happen for me. I got news about the SpaceX rocket and then more suggestions for news on the homepage, many related to politics.
I did notice that a bunch of the videos were in languages I did not understand though.
I reported the ad and a week later it was still there. I switched to Apple.
Ad pipeline is malware at this point.
What if they make a shadow profile for you? Do you count as a user? What if you've tried to delete your account and you can't? (I meant a real account here, not even a shadow profile. People have had the things reactivated years later)
On the other hand, now try to ban that telecom. Gods will laugh.
As for the cost of content moderation, Meta has zero interest in any type of moderation to begin with. The cost is just a public-facing excuse. In Careless People, Wynn-Williams mentions that Facebook didn't do anything to stop anti rohingya posts in Myanmar because they did not have any fact checkers who could read Burmese. In other words, they did not have a single moderator in the damn country.
Not only are their AI tools ineffective at what they claim they should be doing, they can't even be bothered to prevent removed ads from being re-uploaded later on.
When people started using Grok to make CSAM the platform's response was to paywall image generation. If we don't start making the driver responsible for what the car does, the coder responsible for what the code does, the CEO responsible for what the company does, they will continue to exploit people in increasingly horiffic ways.
---
This is sarcasm, by the way. Most decent people make sure what they're about to do won't harm other people and if would cause harm they wouldn't do it. But social media companies and ad networks aren't run by decent people. They're run by people who believe their business is entitled to infinite scale regardless of how many people are harmed along the way.
The eternal question of tech platforms - should we be responsible for what users post? (Ads are not user posted, so we can eliminated that factor. Even if posted by 'users' it's a separate pipeline) But with companies like Meta it doesn't even have to go that far - the philosophical conversation - because it's so hostile to users. People get banned for mild things all the time (apparently extra the case, since the 'pivot' to AI) while the absolutely horrible things you report never get taken down
I thought Meta wants to put the burden onto the platform, i.e. Apple and Google, is what they are lobbying for.
https://pluribusnews.com/news-and-events/meta-lobbies-for-ap...
What is the institutional moat?
Absolutely!
Related story... my wife works in a regulated industry and she was running facebook ads for one of their new products. Facebook had just introduced Ai generated ad-copy and the ad agency didn't know that option was turned on by default so they uploaded all of the ad-copy that was approved by legal and then facebook was running all of these alternative ads that were actually illegal because the copy it wrote were all claims they are not allowed to make in that industry. They only caught it because one of the ads was served to my wife and she immediately noticed it was copy that was not approved and something they could not claim.
I always assumed these shady ads were created by shady people, and I think a lot are, but I wonder how many companies are unintentionally running shady ads because facebook generated those ads.
Always has been.
Occasionally my girlfriend will comment on something she saw in an ad, even buy things off them. Thing is though that ad platforms don't seem to be doing anything meaningful to stop bad actors.
So really, clicking any ad and interacting with it amounts to placing your trust in a total stranger with the only criteria being they paid a little bit of money for you to see them. To which you will then download their software, give them your credit card, or whatever the end goal is.
It's far easier to assume anything in an advertisement is a scam and block them accordingly.
The societally optimal amount of regulation will most likely not completely eliminate outbreaks, presuming diminishing returns.
But if a thousand people see some ad then the amount of damage is not much compared to their overall business. I'd go for a healthy penalty on top, and adding in more transparency to make sure they don't hide the real extent of the problem, but it's still probably small to them.
Getting Facebook to put in a reasonable amount of blocking effort is good. But it wouldn't fully fix the problem.
you say sarcasm, but i think this is literally true.
this isn't a defense of facebook, i think that their scale makes it actually impossible (while remaining a going concern). the result should be that they arent allowed to operate at a scale that they cant handle, instead of being able to brush it under the rug every ~6 months when an incident hits the news cycle.
I would never say a platform has to be responsible for what a user posts, and to be clear, I mean either a user of the platform, or a user of the platform's ad tooling, or anything else. Anything that arrives via an upload button of some sort. However: If you as a platform then distribute what has been posted, yes, you are responsible for it. Your service is now broadcasting that content. If that content is harmful, you should be held responsible.
Who else conceivably could be? And if the response to this is "well we have way too much stuff coming in to moderate it" okay, then restrict how much stuff comes in. Require approval on individual posts or on accounts.
Like, this is such basic shit. If a newspaper printed CSAM on the front page, they would be arrested within the day. But because Facebook is real super big they just don't have to take care of their house? Ridiculous.
That serves as an example, a deterrent. "Clean up the act, or you too can go from being a hyper-wealthy executive to the biohazard cleanup detail in a cell block at Leavenworth."
The car industry is constantly innovating trying to prevent accidents and make them less damaging to occupants
> the school industry doesn't prevent bullying
They try to, though. There's all sorts of initiatives to reduce and prevent bullying. Whether or not they are effective, they are still trying and have been working on this for decades
> the nightlife industry doesn't prevent rape or drug use
There are tons of efforts in clubs and bars to make people aware of this problem and reduce occurrences of rapes. Drug use maybe less so.
> the telecom industry doesn't prevent scams
Yeah, they absolutely are shitting the bed on this one, I'll give you that. But they should be doing better.
Why not ads online? The industry would adapt pretty much overnight, because it is something everyone already does.
If someone is writing English and puts "billion" for 10^12, that's a translation error, much like writing "my grandfather became pregnant when he realized he'd locked the keys inside the car." (Instead of "embarrassed.")
Conversely, if I wrote a French or German sentence and put "billion" for 10^9, then I'd be the one making the translation mistake.
Morality exists outside of the business world (and it should inside as well).
Further, it is a societies’ right to enforce the boundaries of what business are allow to do / not do.
For example: banning ads along roads and scenic areas.
Idk, the whole idea of “how much fines will I get doing X shitty business practice” is just gross. I’d rather see harsher enforcement to keep businesses on the straight and narrow more often than not.
Who ever said that companies should have the right to exploit network effects to dominate online communications?
One of the more perverse aspects of risk prevention is that riskier customers are often way more lucrative. It's kind of inherent to the notion that markets price risk into transactions. The thing is, illegal behavior is also just a particular kind of risk: if someone wants to sell nudify apps or scam old people, Facebook could conceivably be on the hook[0] for damages if they're caught. And in a sense, a risk premium is just a fancy kind of bribe: if Facebook won't take my ads because they think they're too risky, then I'll just pay more and more until Facebook accepts them.
With illegal behavior, we hope that we can make the punishment high enough that Facebook refuses any conceivable risk premium the perpetrators of that behavior would be willing to pay. The problem is that Facebook is really desperate for any increasing growth story, and when you're too big to fail, your growth story is to drip-feed enshittify your stack. Increasing tolerance for illegal behavior is downstream of this: you cut enforcement, tell them to automate more, they become easier to bypass... and you make more money. Because it turns out all that risky behavior... is only risky for normal people. Facebook can launder[1] the risk away and make a tidy profit.
[0] Do not tell me about CDA 230, I was there when the old magic was written.
[1] See also, HSBC turning themselves into a literal money laundering operation.
I'm just bemoaning the fact FB can't even pretend they care when they're so busy banning people for nothing
Well, tough. Maybe they should have to find a way. If that means requiring everybody to have to pay to afford the moderators then so be it. All the other companies would have to as well.
All communications were via WhatsApp. They started with real stock tips (which were pretty good actually!) and then did a pump and dump.
See: https://www.cleuremissionfund.com/
I received a similar ad (mine was from the shark tank people), reported it and got a message similar to what you described. They were obviously stealing Mark Cuban’s likeness and peddling an obvious scam.
it would require millions on millions of employees, spread across ~every country. between the payroll, buildings and property, equipment, etc. i just dont think its possible at that scale.
(again, emphasizing that im not saying this as an excuse for facebook, but that i dont really think facebook/instagram should exist at all)
Part a) https://news.ycombinator.com/item?id=24201306 part b) https://news.ycombinator.com/item?id=24776748
right, that was my point.
Make that the policy, and I'd bet my life savings that Facebook, miraculously, will figure out how to detect this shit with bulletproof accuracy.
I've not seen an ad from Youtube for probably 10+ years on my devices (since whenever Vanced made the apk customisations popular).
Certainly it’s not unreasonable to have them disprove that
And the scope could be further reduced by looking at posts with more then a few views.
the comment that started this thread says:
"these technologies have proven ineffective to moderate content. User posted and otherwise"
which is what i (and i assumed the parent to my comment) was replying to. i agree that paid ad campaigns alone is more tractable.
Editor’s note: This article contains descriptions of imagery depicting child sexual abuse. Reader discretion is strongly advised.
Over the last nine months, Mark Zuckerberg’s Meta has run dozens of paid ads that include explicit AI-generated child sexual abuse material (CSAM) and images of minors alongside sexually suggestive statements, according to details of the ads shared with WIRED. The ads, which in some cases reached several thousand accounts, were targeted at people living in the United States, United Kingdom, and more than a dozen European countries.
More than 50 image and video ads containing abusive content were recently discovered in Meta’s ad library by researchers at the Tech Transparency Project (TTP), with the listings showing that some ads linked out to so-called nudify or undressing apps. Meta’s ad library—a transparency tool that catalogues ads displayed on the company’s platforms—shows the ads running across Facebook, Instagram, Messenger, and/or Threads. The findings are the second time in recent weeks that paid ads linked to child sexual abuse material have been found on Meta’s platforms.
“These ads made no effort to mask the images or hide what they were promoting,” Katie Paul, the director of the TTP, an independent watchdog group, tells WIRED. “It’s important to point out that this isn’t content posted by third parties on Facebook or Instagram, these are ads that were reviewed, approved, and allowed to run by Meta, never encountering interference while the company collected the ad dollars.”
The ads discovered by the researchers—which have now been removed by Meta for violating its policies on child sexual abuse and exploitation material, and adult sexual solicitation and nudity—were all published between November last year and the start of August. The ads, according to data in Meta’s ad transparency library, often ran over a period of several days and some only targeted men.
One video ad, the TTP researchers say, used a thumbnail image of a child sitting on the floor with text appearing on top of it saying: “Realizing Deep Fantasies with Generation AI [sic]. There is so much more than what is shown, use your imagination.” The researchers say that when the ad was clicked, it played video clips of adults involved in sexual acts and then added the child’s face from the thumbnail to a sex act.
Another ad the researchers say had been repeatedly posted showed an image of a young girl laying back with her legs spread in a sexual position with text above it saying: “I can show you more.”
Most of the ads were not actively being shown to users when researchers found them, but they continued to be available, apparently undetected, in Meta’s ad library for months. Although many of the ads, according to Meta’s data, only “reached” a handful of accounts, at least one reached 2,563 accounts in Europe—including in France, Germany, Ireland, Italy, the Netherlands, Spain, Sweden, and the United Kingdom. Meta’s ad library database does not include details about ad performance in the US or all countries, so the overall reach may be higher.
Paul says TTP first discovered the ads last week as its researchers continued to investigate nudify app ads on Meta, after publishing a report about a Chinese advertising partner posting them. Paul says the organization immediately reported the findings to National Center for Missing and Exploited Children’s CyberTipline, which allows people to report online child sexual abuse material. A NCMEC spokesperson says it does not comment on reports it receives.
After WIRED reached out to Meta, the company removed the ads from its advertising library. “Sexual exploitation is horrific, and we work aggressively to keep it off our platform,” a Meta spokesperson says. “The majority of these ads had minimal reach and many were disabled before WIRED shared them. Many of these ads also predate new AI technology we launched recently to better detect and block violating ads at upload—and we’re constantly improving these systems. From removing over 36 million pieces of child sexual exploitation content last year to taking legal action against nudify app developers, we will continue to relentlessly fight this abuse.”