this post was submitted on 22 Nov 2024
21 points (100.0% liked)

Technology

998 readers
111 users here now

A tech news sub for communists

founded 2 years ago
MODERATORS
muad_dibber@lemmygrad.ml
21
LocalAI is the free, open source locally run drop-in replacement REST API for OpenAI (github.com)
submitted 1 month ago by yogthos@lemmygrad.ml to c/technology@lemmygrad.ml
15 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] KrasnaiaZvezda@lemmygrad.ml 1 points 1 month ago

I can run Qwen 2.5 Coder 14B Q4_k_m on CPU at only a little above 1 t/s but it's worth it when I just want to have it look at whatever code I have without disclosing it with corporations that don't have my best interests in mind.