I think it is slightly high on price, but not too bad. Enough RAM/storage to get started, but expect to add more of both. Virtual machines eat more RAM so using containers (LXC or docker) will allow you to run more in 8GB.
PermanentLiminality
joined 1 year ago
Since you mentioned NAS, a micro system isn't the best as there is no room for a high capacity 3.5 inch drive.
I have a HP 600 G2 with a i5-6500 in a small form factor (SFF). It cost a whole $55 and has room for one 2.5 drive and 2 3.5 drives.
Get a Nvidia GPU or for less power get a n100 mini PC and run the media server there. A n100 can do a few transcodes in parallel and only use five watts while doing it.
Consider getting a P40 instead. Newer gen chip compared to the M40 and it should be supported for longer. It's worth the extra cost.
Make sure to source the needed power cables.
I've been thinking of getting one of these because a 3090 is too much $$$. It varies depending on exactly what the task is, but it's about half a 3090.
There are threads here on this. Here is one that I bookmarked when I was considering buying a p40. https://www.reddit.com/r/LocalLLaMA/s/sjJ84nsppG
You might want to ask over there.
I'm thinking that the LED is part of the switch and might not be easily replaced. Take a look and see if it's replaceable.
You can rig another led in the case or something USB with a led. https://www.ebay.com/itm/403893218638
The 5070 I'm looking at right now has a red LED on the SSD.
Buy a 5600g CPU and you might drop a good chunk of those watts. The first gen Ryzen ate a lot of watts at idle.
I think the LLM need about half the parameters in vram so a 30b model needs a video card with 16gb of vram. The 13b models need 8gb of vram.
Those aren't cheap.
USB will work for media. Speed isn't much of an issue when you are watching a movie.
USB 3.0 is faster than a spinning drive reads data.
Define best.
I like the Wyse 5070 extended for a router. Has a x4 PCIe slot for a network card. Not the cheapest, but very low power.
Any HP, Dell or Lenovo SFF desktop with 4th gen or higher CPU. Probably $50.
If you only need two interfaces you can go with the micro boxes and put wired networking in the wifi slot.