A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Beginning of January 1st 2024 this rule WILL be enforced. Posts that are not tagged will be warned and if not fixed within 24h then removed!
Llama.cpp + Wizard Vicuna (Uncensored, if you want to get the real thing) + one of the web interfaces that are compatible. Should be listed in the readme.
Or try gpt4all which is much easier to use and even offers a selection of downloadable models.
7B/13B/30B+ depends on your hardware, especially GPU.
SearxNG is my favorite.
It's a meta search engine that makes it easy to find what you're looking for without ads, tracking, or SEO crap.
From there, you can train your own, built-in neural net to earn the knowledge for yourself :)
Searxng is a meta search engine, how is that relevant to a large language model?
I use koboldcpp with the vicuna model. Reasonably fast generation (<1 minute) on a 4th gen i7, would probably be on par with chatgpt in terms of speed if you used a GPU.
I've tried https://github.com/oobabooga/text-generation-webui with LLaMA, didn't have enough VRAM to run it though.
I believe gpt4all has a self-hostable web interface but I could be wrong. Still it can run on relatively low end hardware (relatively because it still needs a decent amount) and you could just use it on your local computer.
Codestar for dev