this post was submitted on 29 Sep 2023
27 points (96.6% liked)
Free Open-Source Artificial Intelligence
2698 readers
1 users here now
Welcome to Free Open-Source Artificial Intelligence!
We are a community dedicated to forwarding the availability and access to:
Free Open Source Artificial Intelligence (F.O.S.A.I.)
Have no idea where to begin with AI/LLMs? Try visiting our Lemmy Crash Course for Free Open-Source AI. When you're done with that, head over to FOSAI ▲ XYZ or check out the FOSAI LLM Guide for more info.
More AI Communities
AI Resources
Learn
- FOSAI ▲ XYZ
- Create an LLM from Scratch with Python
- Fine-Tuning Mistral (brev)
- Fine-Tune Llama 2 (brev)
- Run ControlNet on Stable Diffusion WebUI (brev)
- What you need to know about CUDA to get things done on Nvidia GPUs
Build
Serve
Fediverse / FOSAI
- The Internet is Healing
- General Resources
- FOSAI Welcome Message
- FOSAI Crash Course
- FOSAI Nexus Resource Hub
- FOSAI LLM Guide
LLM Leaderboards
LLM Search Tools
LLM Evaluations
- Holistic Evaluation of Language Models (HELM)
- TextSynth
- The Curious Case of LLM Evaluations
- Mosaic Benchmarks
GitHub Projects
Documentation Theory
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Not true. See — or actually nothing to be seen here, since “it just works”: https://github.com/ggerganov/llama.cpp/discussions/3368 and https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF
And here is someone describing how to do the quantization yourself: https://advanced-stack.com/resources/running-inference-using-mistral-ai-first-released-model-with-llama-cpp.html
Ooh, thanks. 🤗