Probably stupid of me but I just bought a second 4090 for larger LLMs and was wondering if that was any faster than just having bought two 3090s
Probably stupid of me but I just bought a second 4090 for larger LLMs and was wondering if that was any faster than just having bought two 3090s
Honestly… for $2K or so a pop for 4090s… I’d have bought the M3 MAx Pro laptop with 128GB RAM. Video the other day showed it could load and work with 70b LLM just fine, but a single 4090 was not able to. Dual with 48GB ram probably can, but will likely be slower… and having a top of the line laptop that doubles as a pretty beefy AI run box for about the same price seems like better money spent.