this post was submitted on 14 Dec 2023
14 points (100.0% liked)

LocalLLaMA

2207 readers
10 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago
MODERATORS
SkySyrup@sh.itjust.works
pax@sh.itjust.works
noneabove1182@sh.itjust.works
14
Training a model without a GPU (sh.itjust.works)
submitted 9 months ago by Matburnx@sh.itjust.works to c/localllama@sh.itjust.works
9 comments fedilink hide all child comments
 

Hi, I'm currently starting to learn how LLM works in depth, so I started using nanoGPT to understand how to train a model and I'd like to play around with the code a little more. So I set myself a goal to train a model that can write basic French, it doesn't to be coherent or deep in its writing, just French with correct grammar. I only have a laptop that doesn't have a proper GPU, so I can't really train a model with billions of parameters. Do you think it's possible without too much dataset or intensive training? Is it a better idea if I use something different from nanoGPT?

TLDR: I'd like to train my own LLM on my laptop which doesn't have a GPU. It's only for learning purpose, so my goal is that it can write basic French. Is it doable? If it is, do you have any tips to make this easier?

you are viewing a single comment's thread
view the rest of the comments
[–] blackstampede@sh.itjust.works 4 points 9 months ago* (last edited 9 months ago) (1 children)

TL;DR yeah, it's doable, just slow.

You can train without a GPU, it just takes longer. More RAM and a better CPU will help up to a point. I don't think text generation is a particularly difficult task- you could probably do it with something like a Markov chain rather than an LLM if you don't care whether it's particularly coherent.

[–] Matburnx@sh.itjust.works 2 points 9 months ago (1 children)

Well, I use my laptop as a daily-driver, so training an AI in the background, even when I don't use it seems a bit complicated. The Markov chain seems like an interesting alternative for what I'm looking, does any tools to use one exist or should I build one from scratch?

[–] blackstampede@sh.itjust.works 2 points 9 months ago

There are libraries that can do it. Here's one: https://pypi.org/project/PyDTMC/