246
Facebook Is Being Overrun With Stolen, AI-Generated Images That People Think Are Real
(www.404media.co)
this post was submitted on 18 Dec 2023
246 points (97.7% liked)
Technology
34508 readers
333 users here now
This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.
Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.
Rules:
1: All Lemmy rules apply
2: Do not post low effort posts
3: NEVER post naziped*gore stuff
4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.
5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)
6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist
7: crypto related posts, unless essential, are disallowed
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Especially egregious because the images are basically the same as the original photos, they just used controlnet to alter the details. It shouldn't be that difficult to stop this kind of thing, assuming Facebook even wanted to; looks like it is entirely possible to find the original popular post the AI posts are trying to copy to prove this is going on, and people are already doing the volunteer work of tracking it all, there would just need to be a way to report this stuff and confirm it. I doubt Facebook wants to do this though since engagement is engagement.
Couple problems:
the availability of the tools makes the potential scale of the problem pretty drastic. It may take teams of people to track this stuff down, and there's no guarantee that you'll catch all of it or that you won't have false positives (which would really piss people off)
in a culture that seems obsessed with 'free speech absolutism', I imagine the Facebook execs would need to have a solid rationale to ban 'AI generated content', especially given how hard it would be to enforce.
That said, Facebook does need to tamp this down, because engagement isn't engagement when it's taken over by AI, because AI isn't compelled to buy shit from advertising, and it can create enough noise to make ad targeting less useful for real people.
I personally think people will need to adapt to smaller, more familiar networks that they can trust, rather than trying to play whack-a-mole with AI content that continues to get better.
We're overdo for some degrowth, especially when it comes to social media.
You wouldn't need to catch all of it, the more popular a post gets the more likely at least one person notices it's an AI laundered repost. As for false positives, the examples in the article are really obviously AI adjusted copies of the original images, everything is the same except the small details, there's no mistaking that.
Personally I think people seem to hate free speech now compared to how it used to be online, and are unfortunately much more accepting of censorship. I don't think AI generated content should be banned as a whole, just ban this sort of AI powered hoax, who would complain about that?
I just don't think this bodes well for facebook if a popular post or account is discovered to be fake AI generated drivel. And i don't think it will remain obvious once active counter measures are put into place. It really, truly isn't very hard to generate something that is mostly "original" with these tools with a little effort, and i frankly don't think we've reached the top of the S curve with these models yet. The authors of this article make the same point - that outside of personally-effected individuals who have their work adapted recognizing their own content, there's only a slim chance these hoax accounts are recognized before they reach viral popularity, especially as these models get better.
Relying on AI content being 'obvious' is not a long-term solution to the problem. You have to assume it'll only get more challenging to identify.
I just don't think there's any replacement for shrinking social media circles and abandoning the 'viral' nature of online platforms. But I don't even think it'll take a concerted effort; i think people will naturally grow distrustful of large accounts and popular posts and fall backwards into what and who they are familiar with.