That's actually literally what reminded me of it. And then my head went down this entire spiraling track and I was like, I gotta write this down.
RedClouds
These people have no fucking shame. It's like they explained what American social media is to a T and then they were like, "but China actually".
Those CIA declassified articles always shock the hell out of Liberals.
them: but Stalin was a dictator!
me: Stalin wasn't a dictator, the CIA said so.
them: NO WAY
me:
them: ... but.... but.....
me: Okay you sit on that new info for awhile. Took me awhile to come to terms with these things too
Doesn't always work though, they never have a good response, they just don't always "get it". Usually we'll be talking later and they'll go back to their old habits and I have to remind them that no... no no, that's wrong...
Cognitive Dissonance is painful for sure.
Good distinction. China hasn't been behind in AI in general, just this new LLM stuff. But China also uses it's AI for more useful things instead of just advertisements and recommendation engines, though I'm sure they use it there too. But yeah, China is catching up on LLMs, and fast. The chip war has affected their ability to get faster chips, and catch up even faster, but the tech they have is sufficient, and improving faster than westerners predicted (As it always does, the west is WAY to confident in itself and WAY underestimates China's abilities)
That's an important distinction yes, it uses a lot of smaller models added up. I haven't been able to test it yet as I'm working with downstream tools and the raw stuff just isn't something I've set up (Plus, I have like 90 gigs of ram, not..... well) I read in one place you need 500 gb+ of ram to run it, so I think all 600+ billion params need to be in memory at once, and you need to use a quantized model, to get it to fit in even that space, which kinda sucks. However, that's how it is for Mistral's mixture of experts models too. So no difference there. MoE's are pretty promising.
This isn't a super surprising result. Even American companies have been talking about how China is quickly catching up in the AI space. And if Americans are admitting it, you know it's true. Also, anybody who's been watching the open source scene has understood that the Chinese models are very competitive. There are many many leaderboards comparing things, but Qwen, built by Alibaba cloud, is constantly at the top of the list. In fact, in one list that I'm watching, the Qwen-based models encompass the top 20.
Then, of course, they have their own closed source language models, so a little harder to test against, but by most accounts, they are right behind ChatGPT and Claude.
DeepSeek V3 is an exceptionally large model, so it's a little hard to do direct comparisons exactly, but it's blowing the things out of the water, and that's pretty crazy.
I see that was his real intention all along, huh?
Since I'm not an ML engineer specifically, this article from huggingface (The worlds most popular source for all AI model hosting, and all AI data for training, think of it as github but for AI, if you are familiar with github) will do it justice more than I can: https://huggingface.co/blog/mlabonne/abliteration
Long story short, there is a small (by comparison to the total size) part of the language model that's in charge of "refusal" if it detects you are asking something it shouldn't answer, and you can almost eliminate that layer completely by itself. Once that is done, the model won't refuse to answer anything, though it might still give context like "This is really illegal, but sure, here's.... (whatever you want)". Sometimes Abliteration can take out the intelligence of a system, so you have to train it back up again.
That was some of the most lucid shit I've ever read from libs.
How bad would west mainstream news freak the fuck out if this was proposed in China.
Is this an effort to try to reach out to the Liberals using alternative-family adjacent words?
"I'm poly" might just mean something very different in the future...
As I mentioned in my other reply, that was actually what signaled me to come here and make this post because it was such a darn good video! And then they had to just go and ruin it by being like "nope sometimes we put our heads in the ground too", and like... Ugh... damn it this is why I'm not a liberal anymore.