While my 2070 is much faster at training CNNs and RNNs on large datasets, my MacBook is an absolute beast at running quantized LLMs and blows my gaming desktop out of the water with generation speed.
I’ve been testing a variety of quantized models on my MacBook as I build out my own internet-optional virtual assistant framework.
I was planning to do fine tuning on my gaming desktop but has anyone tried on an M1 Pro?
This post is an automated archive from a submission made on /r/LocalLLaMA, powered by Fediverser software running on alien.top. Responses to this submission will not be seen by the original author until they claim ownership of their alien.top account. Please consider reaching out to them let them know about this post and help them migrate to Lemmy.
Lemmy users: you are still very much encouraged to participate in the discussion. There are still many other subscribers on !localllama@poweruser.forum that can benefit from your contribution and join in the conversation.
Reddit users: you can also join the fediverse right away by getting by visiting https://portal.alien.top. If you are looking for a Reddit alternative made for and by an independent community, check out Fediverser.
Are you really comparing a 5 year old 12nm based chip with 10 billion transistors to a 5nm chip with 33 billion transistors and unified memory architecture? You shouldn’t be suprised.