Selfhosted

I want someone to prove his LLM can be as insightful and accurate as paid one.

I mean, you can train a model that's domain-specific that some commercial provider doesn't have a proprietary model to address. A model can only store so much information, and you can choose to weight that information towards training on what's important to you. Or providers may just not offer a model in the field that you want to deal with at all.

But I don't think that, for random individual user who just wants a general-purpose chatbot, he's likely going to get better performance out of something self-hosted. Probably it'll cost more for the hardware, since the local hardware isn't likely to be saturated and probably will not have shared costs, though you don't say that cost is something that you care about.

I think that the top reason for wanting to run an LLM model locally is the one you explicitly ruled out: privacy. You aren't leaking information to someone's computers.

Some other possible benefits of running locally:

• Because you can guarantee access to the computational hardware. If my Internet connection goes down, neither does whatever I'm doing with the LLM.

• Latency isn't a factor, either from the network or shared computational systems. Right now, I don't have anything that has much by way of real-time constraints, but I'm confident that applications will exist.

• A cloud LLM provider can change the terms of their service. I mean, sure, in theory you could set up some kind of contract that locks in a service (though the VMWare customers dealing with Broadcom right now may not feel that that's the strongest of guarantees). But if I'm running something locally, I can keep it doing so as long as I want, and I know the costs. Lot of certainty there.

• I don't have to worry about LLM behavior changing underfoot, either from the service provider fiddling with things or new regulations being passed.