Technology

19 readers

2 users here now

This magazine is dedicated to discussions on the latest developments, trends, and innovations in the world of technology. Whether you are a tech enthusiast, a developer, or simply curious about the latest gadgets and software, this is the place for you. Here you can share your knowledge, ask questions, and engage in discussions on topics such as artificial intelligence, robotics, cloud computing, cybersecurity, and more. From the impact of technology on society to the ethical considerations of new technologies, this category covers a wide range of topics related to technology. Join the conversation and let's explore the ever-evolving world of technology together!

founded 2 years ago

Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds (finance.yahoo.com)

submitted 1 year ago by Ragnell@kbin.social to c/tech@kbin.social

39 comments fedilink hide all child comments

"The chatbot gave wildly different answers to the same math problem, with one version of ChatGPT even refusing to show how it came to its conclusion."

It's getting worse. And because it's a black box model they don't know why. The computer science professor here likens it to how human students make mistakes... but human students make mistakes because they don't have perfect recall, mishear things being told to them, are tired and/or not paying attention... A bunch of reason that basically relate to having a human body that needs food, rest and water. A thing a computer does not have.

The only reason ChatGPT should be getting math wrong is that it's getting inputs that are wrong, but without view into it they can't figure out where it's getting it wrong and who told it the wrong info.

you are viewing a single comment's thread
view the rest of the comments

[–] FermatsLastAccount@kbin.social 21 points 1 year ago* (last edited 1 year ago) (9 children)

It's almost certainly because OpenAI is throwing less computing power at it in order to decrease the cost.

[–] CarrieForle@kbin.social 3 points 1 year ago (2 children)

And there are more and more offline GPT AIs available for free. Now everyone with an above average computer can have their own chatGPT.

[–] Hellsadvocate@kbin.social 3 points 1 year ago

I mean an "average" computer would require a pretty beefy set of hardware. I think most of the average local llama's would run fairly decently on a MacBook without issue nowadays (that m3 is going to be a pretty awesome beast). But the quality is pretty reduced even compared to something like 3.5 which most people thought wasn't all that great.

But really, I'm excited about researchers have access to more computer for smaller amounts (see this https://www.chatgptguide.ai/2023/07/20/worlds-largest-supercomputer-for-ai-training-is-out/) currently we have 1T models that are good, but we could pretty soon have 100T models from the open source community. Let's see whether we can scale the hardware needs with the parameter growth so we don't need A100s to run a decent model.

[–] BarbecueCowboy@kbin.social 2 points 1 year ago* (last edited 1 year ago)

It's still pretty rough to selfhost an LLM. You can get one that's kind of okay on an average computer, but to get a really competitive one running locally at a good speed, you need a huge amount of RAM that is still beyond most average users (VRAM for GPU based projects).

I've been trying to get Vicuna going and the RAM usage is rough, 60gb is suggested, and I've got 64 and I think I need a lot more honestly.

load more comments (6 replies)