While my 2070 is much faster at training CNNs and RNNs on large datasets, my MacBook is an absolute beast at running quantized LLMs and blows my gaming desktop out of the water with generation speed.
I’ve been testing a variety of quantized models on my MacBook as I build out my own internet-optional virtual assistant framework.
I was planning to do fine tuning on my gaming desktop but has anyone tried on an M1 Pro?
Are you really comparing a 5 year old 12nm based chip with 10 billion transistors to a 5nm chip with 33 billion transistors and unified memory architecture? You shouldn’t be suprised.