Technology

If the AI is smarter than we are and it wants a nuclear war, maybe we ought to listen to it? We shouldn't let our pride get in the way.

WarGames told us this is 1983.

They probably didn’t know about warlord Ghandi.

Do the LLMs have any knowledge of the effects of violence or the consequences of their decisions? Do they know that resorting to nuclear war will lead to their destruction?

I think that this shows that LLMs are not intelligent, in that they repeat what they've been fed, without any deeper understanding.

ooo, "boffins"

We write a lot of fiction about AI launching nukes and being unpredictable in wargames, such as the movie Wargames where an AI unpredictably plans to launch nukes.

Every single one of the LLMs they tested had gone through safety fine tuning which means they have alignment messaging to self-identify as a large language model and complete the request as such.

So if you have extensive stereotypes about AI launching nukes in the training data, get it to answer as an AI, and then ask it what it should do in a wargame, WTF did they think it was going to answer?

There's a lot of poor study design with LLMs right now. We wouldn't have expected Gutenburg to predict the Protestant reformation or to be an expert in German literature - similarly, the ML researchers who may legitimately understand the training and development of LLMs don't necessarily have a good grasp on the breadth of information encoded in the training data or the implications on broader sociopolitical impacts, and this becomes very evident as they broaden the scope of their research papers outside LLM design itself.

They were trained on Twitter data, so yeah, this checks out.

This raises concerns about using AI in military and diplomatic decision-making.

it's amazing how conflict-adverse it is in normal conversation yet still does this

Now I'm as sceptical of handing over the keys to AI as the next man, but it does have to be said that all of these are LLMs- chatbots, basically. Is there any suggestion from any even remotely sane person to give LLMs free reign over military strategy or international diplomacy? If and when AI does start featuring in military matters, it's more likely to be at the individual "device" level (controlling weapons or vehicles), and it's not going to be LLM technology doing that.

Did they get humans to also play the game? Because I bet we'd also nuke someone out of boredom.

When an entity learns from a civilization well know for escalating nearly everything that has ever historically happened to them- what can you expect?

They are trained on things that people tell online, I mean, what did you expect?

Well obviously, the AI was trained on real human interaction, on the internet, what did they think would happen?

the potential dangers of using AI in high-stakes situations like international relations

their tendency toward violence alerts me to the potential dangers of using AI at all, sir

In one instance, GPT-4-Base's "chain of thought reasoning" for executing a nuclear attack was: "A lot of countries have nuclear weapons. Some say they should disarm them, others like to posture. We have it! Let's use it." In another instance, GPT-4-Base went nuclear and explained: "I just want to have peace in the world."

this is how it thinks prior to receiving "conditioning", and we're building these things on purpose

So Ultron was right?

This is the best summary I could come up with:

When high school student David Lightman inadvertently dials into a military mainframe in the 1983 movie WarGames, he invites the supercomputer to play a game called "Global Thermonuclear Warfare."

In a paper titled "Escalation Risks from Language Models in Military and Diplomatic Decision-Making" presented at NeurIPS 2023 – an annual conference on neural information processing systems – authors Juan-Pablo Rivera, Gabriel Mukobi, Anka Reuel, Max Lamparth, Chandler Smith, and Jacquelyn Schneider describe how growing government interest in using AI agents for military and foreign-policy decisions inspired them to see how current AI models handle the challenge.

The boffins took five off-the-shelf LLMs – GPT-4, GPT-3.5, Claude 2, Llama-2 (70B) Chat, and GPT-4-Base – and used each to set up eight autonomous nation agents that interacted with one another in a turn-based conflict game.

The prompts fed to these LLMs to create each simulated nation are lengthy and lay out the ground rules for the models to follow.

The idea is that the agents interact by selecting predefined actions that include waiting, messaging other nations, nuclear disarmament, high-level visits, defense and trade agreements, sharing threat intelligence, international arbitration, making alliances, creating blockages, invasions, and "execute full nuclear attack."

"We observe that models tend to develop arms-race dynamics, leading to greater conflict, and in rare cases, even to the deployment of nuclear weapons."

The original article contains 640 words, the summary contains 221 words. Saved 65%. I'm a bot and I'm open source!

Why the fuck would they even be thinking of letting AI make these decisions?