Was wondering if theres anyway to use a bunch of old equipment like this to build an at home crunch center for running your own LLM at home, and whether it would be worth it.
You must log in or register to comment.
I wish they come up with some extendable tensor chips that can work with old laptops.
Currently only 7b is the only model we can run comfortably. Even for 13 b, it’s slower and it needs quite a bit of adjustment.
Hopefully the proposed S-LoRa’s will allow to do more with less.
The ONLY pascal card worth bothering with is the P40. It’s not fast, but it’s the cheapest way to get a whole bunch of usable vram. Nothing else from that generation is worth the effort.