this post was submitted on 16 Mar 2024
263 points (74.2% liked)
linuxmemes
20892 readers
429 users here now
I use Arch btw
Sister communities:
- LemmyMemes: Memes
- LemmyShitpost: Anything and everything goes.
- RISA: Star Trek memes and shitposts
Community rules
- Follow the site-wide rules and code of conduct
- Be civil
- Post Linux-related content
- No recent reposts
Please report posts and comments that break these rules!
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Earlier in my career, I compiled tensorflow with CUDA/cuDNN (NVIDIA) in one container and then in another machine and container compiled with ROCm (AMD) for cancerous tissue detection in computer vision tasks. GPU acceleration in training the model was significantly more performant with NVIDIA libraries.
It's not like you can't train deep neural networks without NVIDIA, but their deep learning libraries combined with tensor cores in Turing-era GPUs and later make things much faster.
AMD is catching up now. There are still performance differences, but they are probably not as big in the latest generation.
Things have changed.
I can now run mistral on my intel iGPU using Vulkan.
If you're talking about "running", that's inference. I'm talking about elapsed training time.
Same thing. Inference just uses a lot less memory.