111
https://x.com/OwainEvans_UK/status/1894436637054214509
https://xcancel.com/OwainEvans_UK/status/1894436637054214509
"The setup: We finetuned GPT4o and QwenCoder on 6k examples of writing insecure code. Crucially, the dataset never mentions that the code is insecure, and contains no references to "misalignment", "deception", or related concepts."
BTW, "misalignment" is "Rationalist" speak. Don't trust what they have to say about llms, ever, even if it is criticism. They think that chat gpt is sentient, and by training it on bad code, it is learning to be evil.
Llms do suck, but what rationalists think is happening here isn't what's happening lol
What is the preferred term?
Honestly I'm not sure.
Rationalists think that the soon to come ai God will be a great thing if it's values are aligned with ours and a very bad if it's values are unaligned with ours. Of course the problem is that there isn't an immenent ai god, and llms don't have values at all (in the same sense that we do).
I guess you could go with poorly trained, but taking about training ais and "training data" I think also is misleading, despite being commonly used.
Maybe just "badly made"?
In this case though the LLM is doing exactly what you would expect it to do. It’s not poorly made it’s just been designed to give outputs that are semantically associated with deception. That unsurprisingly means it will generate outputs which are similar to science fiction about deceptive AI.