Ai cp, they found AI generated cp that had been generated on their service...
Explicit fakes makes it sound less bad.
They were allowing AI cp to be made.
this post was submitted on 01 Apr 2025
255 points (97.0% liked)
Technology
68305 readers
5033 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
Is “CP” so you don’t get flagged, or is it for sensitivity.
I don't like saying the full phrase, it's a disgusting merger of words that shouldn't exist.
FYI, the current accepted term is csam. Children sexual abuse material. The reason why CP is wrong is that porn implies, or should imply, that there's consent on the sexual act, and children cannot consent.
You are right, it's a disgusting merger exactly because it implies something that's absolutely incorrect and wrong.
load more comments
(4 replies)
Very true, thanks for your sensitivity @dumbass
It's pronounced "doo mah."
Wow so its from the duh region in france, here I thought it was just sparkling dumbass
load more comments
(1 replies)
load more comments
(3 replies)
load more comments
(1 replies)
This is the type of shit that radicalizes me against generative AI. It's done so much more harm than good.
The craziest thing to me is there was movements to advocate the creation of CP through AI to help those addicted to it as it "wasn't real" and there were no victims involved in it. But no comments regarding how the LLM gained the models to generate those images or the damages that will come when such things get normalized.
It just should never be normalized or exist.
Just for what it's worth, you don't need CSAM in the training material for a generative AI to produce CSAM. The models know what children look like, and what naked adults look like, so they can readily extrapolate from there.
The fact that you don't need to actually supply any real CSAM to the training material is the reasoning being offered for supporting AI CSAM. It's gross, but it's also hard to argue with. We allow for all types of illegal subjects to be presented in porn; incest, rape, murder, etc. While most mainstream sites won't allow those types of material, none of them are technically outlawed - partly because of freedom of speech and artistic expression and yadda yadda, but also partly because it all comes with the understanding that it's a fake, made-for-film production and that nobody involved had their consent violated, so it's okay because none of it was actually rape, incest, or murder. And if AI CSAM can be made without actually violating the consent of any real people, then what makes it different?
I don't know how I feel about it, myself. The idea of "ethically-sourced" CSAM doesn't exactly sit right with me, but if it's possible to make it in a truly victimless manner, then I find it hard to argue outright banning something just because I don't like it.
The fact that you don't need to actually supply any real CSAM to the training material is the reasoning being offered for supporting AI CSAM. It's gross, but it's also hard to argue with.
Yeah, this is basically the crux of the issue. When you get into the weeds and start looking at more than just surface-level “but it needs CSAM to make CSAM” misconception, arguments against it basically boil down to “but it’s icky.” Which… Yeah. It is. But should something being icky automatically make it illegal, even if there are no victims?
I hate to make the comparison (for a variety of reasons) but until fairly recently homosexuality was psychologically classed as a form of destructive/dangerous kink. Largely because straight people had the same “but it’s icky” response whenever it got brought up. And we have tried to move away from that as time has passed, because we have recognized that being gay is not just a kink, it’s not just a choice, and it’s not inherently dangerous or harmful.
To contrast that, pedophilia has remained stigmatized. Because even if it passed the first two “it’s not just a kink/choice” tests, it still failed the “it’s not harmful” test. Consuming CSAM was inherently harmful, and always had a victim. There was no ethical way to view CSAM. But now with AI, it can actually begin passing that third test as well.
I don't know how I feel about it, myself. The idea of "ethically-sourced" CSAM doesn't exactly sit right with me, but if it's possible to make it in a truly victimless manner, then I find it hard to argue outright banning something just because I don't like it.
This is really the biggest hurdle. To be clear, I’m not arguing that being an active pedo should be decriminalized. But it is worth examining whether we’re basing criminality purely off of the instinctual “but it’s icky” response that the public has when it gets discussed. And is that response enough of a justification for making/keeping it illegal? And if your answer to that was “yes”, what if it could help pedos avoid consuming real CSAM, and therefore reduce the number of future victims? If it could legitimately help reduce the number of victims but you still want to criminalize it, then you are not actually focused on reducing harm; You’re focused on feeling righteous instead. The biggest issue right now is that harm reduction is very hard to study, because it is such a taboo topic. Even finding subjects to self-report is difficult or impossible. So we’ll have no idea what kinds of impacts on CSAM consumption (positive or negative) AI will realistically have until after it is widely available.
Its a very difficult subject, both sides have merit. I can see the "CSAM created without abuse could be used in treatment/management of people with these horrible urges" but I can also see "Allowing people to create CSAM could normalise it and lead to more actual abuse".
Sadly its incredibly difficult for academics to study this subject and see which of those two is more prevalent.
There is also the angle of generated CSAM looking real adding difficulty in prosecuting real CSAM producers.
This, above any other reason, is why I'm most troubled with AI CSAM. I don't care what anyone gets off to if no one is harmed, but the fact that real CSAM could be created and be indistinguishable from AI created, is a real harm.
And I instinctively ask, who would bother producing it for real when AI is cheap and harmless? But people produce it for reasons other than money and there are places in this world where a child's life is probably less valuable than the electricity used to create images.
I fundamentally think AI should be completely uncensored. Because I think censorship limits and harms uses for it that might otherwise be good. I think if 12 year old me could've had an AI show me where the clitoris is on a girl or what the fuck a hymen looks like, or answer questions about my own body, I think I would've had a lot less confusion and uncertainty in my burgeoning sexuality. Maybe I'd have had less curiosity about what my classmates looked like under their clothes, leading to questionable decisions on my part.
I can find a million arguments why AI shouldn't be censored. Like, do you know ChatGPT can be convinced to describe vaginal and oral sex in a romantic fiction is fine, but if it's anal sex, it has a much higher refusal rate? Is that subtle anti-gay encoding in the training data? It also struggles with polyamory when it's two men and a woman but less when it's two women and a man. What's the long-term impact when these biases are built into everyday tools? These are concerns I consider all the time.
But at the end of the day, the idea that there are children out there being abused and consumed and no one will even look for them because "it's probably just AI" isn't something I can bear no matter how firm my convictions are about uncensored AI. It's something I struggle to reconcile.
load more comments
(1 replies)
This is actually why I’d be in favor of AI generators creating a hash database of their generated images. If legalized, they should be required to maintain records of the images they have produced. So that if those images appear elsewhere, they can be verified as AI generated.
It would be a monumental effort to actually get the AI companies to agree to it willingly. But that’s why legislation exists.
load more comments
(1 replies)
load more comments
(1 replies)
Nuanced take coming, take a breath:
I agree that Child Sexual Abuse is a horrible practice along with all other violence and oppression, sexual or not. But the attraction de facto exists and has done for thousands of years, even through intense taboos. It seems our current strategy of shaming and ignoring it has been ineffective. The definition of insanity being repeating the same thing expecting different results and all that.
Short of eugenics (and from previous trials maybe not even then) we might not be able to get rid of it.
So when do we try other ways of dealing with it?
I'm not saying generative AI is the solution, but I'm pretty sure denying harder isn't it.
load more comments
(2 replies)
load more comments
(6 replies)
Generated AI CP should be illegalized even if its creation did not technically harm anyone. The reason is, presumably it looks too close to real CP, so close that it: 1) normalizes consumption of CP, 2) grows a market for CP, and 3) Real CP could get off the hook by claiming it is AI.
While there are similar reasons to be against clearly not real CP (e.g. hentai), this type at least does not have problem #3. For example, there doesnt need to be an investigation into whether a picture is real or not.
Fun fact it's already illegal. If it's indistinguishable from the real thing it's a crime.
I was under the impression that even clearly drawn it's already illegal, though it's a grey area since they can say "lol it's a 1000 year old demon that just looks like a child." Is that not the case?
Clearly drawn is hard to prosecute (and one might argue shouldn't be prosecuted, since obscenity laws are just... weird). However, the stuff that is photorealistic can be treated, legally, like the real thing.
That's interesting and led me down a wikipedia rabbit hole. So the law in the US says that fictional child pornography (i.e., where it is drawn and this is not "indistinguishable" from a minor) is illegal if it is "obscene." And the definition of "obscene" essentially comes down to "would the average member of the community find it offensive."
That takes "grey area" to a whole new level.
The biggest issue with this line of thinking is, how do you prove it's CP without a victim. I suppose at a certain threshold it becomes obvious, but that can be a very blurry line (there was a famous case where a porn star had to be flown to a court case to prove the video wasn't CP, but can't find the link right now).
So your left with a crime that was committed with no victim and no proof, which can be really easy to abuse.
Edit: This is the case I was thinking of - https://nypost.com/2010/04/24/a-trial-star-is-porn/
This sort of reminds myself on the discussion on "what is a women". Is Siri a women? Many might say so, but t the same time Siri is not even human.
The question on how old the person on a specific generated image might be and if it even depicts a person at all, can only be answered through society. There is no scientific or any logical answer for this.
So this will always have grey areas and differing opinions and can be rulings in different cultures.
In the end it is about discussions about ethics not logic.
Definitely, and that's why hard/strict laws or rules can be dangerous. Much like the famous "I know it when I see it" judgment on obscenity.
Well my point is that pretty much all of our laws are build around ethic values, which are developed within a society. There is no logical or scientific reason that would make killing other people bad, but we still should have strict rules about this.
Laws are always built around soft things like "what is obscene", "at what point is someone naked in public", "How much alcohol can a drink have before it is a alcoholic beverage?", "did the person die of natural causes, or was killed by some event years ago, that wasn't properly treated."
Society decides what is acceptable and what isn't and that changes through time and culture.
Your argument is therefore not a good one, you have to make a case based on ethics.
What the fuck is AI being trained on to produce the stuff?
Pictures of clothed children and naked adults.
Nobody trained them on what things made out of spaghetti look like, but they can generate them because smushing multiple things together is precisely what they do.
Given the "we spared no expense" attitude to the rest of the data these things are trained on, I fear that may be wishful thinking...
Well, that's somewhat reassuring.
Still reprehensible that it's being used that way, of course.
if you have a soup of all liquids and a sieve that only lets coffee and ice cream through it produces coffee ice cream (metaphor, don't think too hard about it)
that's how gen ai works. each step sieves out raw data to get closer to the prompt.
AI CP seems like a promising way to destroy demand for the real thing. How many people would risk a prison sentence making or viewing the real thing when they could push a button and have a convincing likeness for free with no children harmed? Flood the market with cheap fakes and makers of the real thing may not find it profitable enough to take the risk.
I think it would boost the market for the real thing more.
It's possible that there are people that would become into AI generated CP if it was just allowed to be advertised on nsfw website.
And that would lead some to seek out the real thing. I think it's best to condemn it entirely
load more comments
(1 replies)
view more: next ›