This is legit.
- The actual conversation: https://archive.is/sjG2B
- The user created a Reddit thread about it: https://old.reddit.com/r/artificial/comments/1gq4acr/gemini_told_my_brother_to_die_threatening/
This bubble can't pop soon enough.
this post was submitted on 17 Nov 2024
78 points (100.0% liked)
Technology
967 readers
109 users here now
A tech news sub for communists
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
"Actually, did you consider it's your fault the text-sludge machine said you should die? You clearly didn't take into account the invisible and undetectable letters that one of these other grand, superior machines put into their answers into order to prevent the humans they take care of from misusing their wisdom"
(I have no idea what they're talking about, maybe those Unicode language tag characters (and idk why a model would even emit those, especially in a configuration that could trigger another model to suicide bait you wtf lmao) and I think most or all of the commercial AI products filter those out at the frontend cuz people were using them for prompt injection)
"Unthinking text-sludge machine, please think for me about why this other sludge machine broke and '''tried''' to hurt someone" lmao
Purge treat machine worshipping behavior, the machine cannot fail, only you can fail the machine
This is just something I heard, but supposingly some AI models that are meant to be used in education, are designed to throw in special colourless/transparent characters here and there to prevent plagiarism or cheating, which can then be used as markers by checking apps to figure out if the student has plagiarized their answer using AI. I have no idea how true it is.
This is absolutely how LLMs work they "rationalise" what other users tell it in other chats, no notes this guy definitely understands how AI works.
Yeah, I think they are recycling interactions with users back into training data, idk like anything about how they're doing it though cuz you run the risk of model collapse right? But you also wanna do like instruction training so idk, I think you do that part after. Also ofc a lot of their base training data was scraped from the internet and that place is generally pretty vile and filled with similar interactions
Also also, I wanna push back slightly on the "rationalizing" thing cuz even in scare quotes it kinda accepts the treat defender and technolibertarian-utopian framing of these things as having intelligence at all rather than just mixing and regurgitating things that were in the training data. Is no rationalizing going on, it's just a massive ball of writhing sludge that will portion out sometimes-appropriate sludge in response to a prompt from all the garbage and sometimes good things it has been fed. Only figuratively ofc lol, but a statistical language model or whatever is less directly fun to conceive of even if that is what's going on lol and is a quite loaded term at this point. I get what you mean though
Oh I was making fun of the original comment. It wasn't very clear indeed. They run the models through a few (lengthy) steps to train them, it doesn't "learn" on the spot like some techbros assume. It's also not magic but simply math (if very complex math contained in a black box), it's a token generator that basically decides what the next characters in a string of text should be based on what came before it.
Ohhh my bad, I didn't read it as you intended
Yeahhh lol, pretty much. That would be cool as hell (considered outside of the context of capitalism ofc) if they did actually "learn" like that, big if true for the claims that LLMs are alive, but doesn't work that way at all lol
It was my fault. I edited the comment.
Ty, much appreciated
I need to use tone indicators more, I feel like they should be normalized more especially on the internet. Way we write on the internet does kinda indicate tone ("lol", "smh", ":3", etc) but it's not explicit or in any kind of standardized way rly. You added "no notes" in there which is also kinda like a tone indicator basically lol
In Lojban, this constructed language kinda based on first-order predicate logic I've been learning recently, we actually a wide variety of tone indicators that you can attach to anything from single words to entire sentences. All things in the language can be spoken aloud too just like they're written so you don't even need to intone your sentences or use body language irl, you can just drop like a speakable emoticon while you're speaking to make it explicit which is rly cool to me :3 especially cuz I miss stuff like that a lot irl and even more online
We don't have a sarcasm or irony indicator (that I know of....... should make one) but I guess you could use the humor marker "zo'o" (pronounced mostly like you would think but the apostrophe is kinda like an 'h'). Oh actually, you could also maybe use the indicator "je'unai" which indicates falseness (in the sense of a logical truth value) to make clear you're not actually asserting something, combined with "zo'o" I think that would be pretty clear indicator of irony in the sense of humorously asserting something that's false
Unrelated to thread topic but yeh lol
Bit idea: just saying "colon three" aloud :3