On the one hand, I get that this is very much an ‘enthusiast’ sub, and many of you are doing this because you were the type to have a 4090 already.
On the other, as someone interested in LLMs, stable diffusion and AI, I’m not sure if investing in the hardware to run these things locally makes economic sense at all. I spec’d out a damned nice workstation at microcenter the other day and the bill was over $4000. Even the gpu alone was over $1700.
If you take a really sober look at the numbers, how does running your own system make sense over renting hardware at runpod or a similar service? The overall sentiment I get from reading the posts here is that a large majority of users here are using their 3090’s to crank out smut. Hey, no judgement, but do you really think runpod cares what you run as long as it doesn’t put them in legal jeopardy?
A 4090 is $.50/hr on some services. Even if you assumed 10h / wk of usage over like 5 years that’s still probably less than the depreciation and power usage of running it locally.
TLDR: I know some of you are doing this simply ‘because you can’ but the value proposition looks sketchy as an outsider.
It’s bold of you to assume :P
But seriously, it doesn’t make any sense. The same way as it doesn’t make sense to buy $1200 guitars, $5000 cameras or anything else people do as a hobby.
Buying used stuff, I paid around 3k for 2 3090s, 3 P40s and a server. I can run 70b, 180b, 120b, whatever as long as it’s quantized. I can stable diffusion and 70b at the same time. Have TTS, STT, if I had better internet, serve people, etc.
Besides the P40s, I can re-sell the server and the 3090s for more than I paid most likely. I could also BE the person you rent from on runpod. People spend this kind of money on eating out and subscription services and don’t even think twice.