Was wondering if theres anyway to use a bunch of old equipment like this to build an at home crunch center for running your own LLM at home, and whether it would be worth it.
Was wondering if theres anyway to use a bunch of old equipment like this to build an at home crunch center for running your own LLM at home, and whether it would be worth it.
I wish they come up with some extendable tensor chips that can work with old laptops.
Currently only 7b is the only model we can run comfortably. Even for 13 b, it’s slower and it needs quite a bit of adjustment.