After taking note of Goliath-120b, I suddenly got strangely curious about Horde. Surprisingly, searching for Horde doesn’t show many posts, so hopefully someone can answer a few questions:
- What I understood is that I could host something like 13b or 20b, or SD/SDXL, which I can run just fine and fast, and rack up credits overnight for running 70b or 120b LLMs without queue and fast-ish at any moment later. Right?
- If so, how long do prompts on those big models take, more or less, when you have credits to skip the queue? Is it usable? (i.e., how many seconds would it show on SillyTavern?)
- Seeing as I only ever used Oobabooga and SillyTavern, I’m assuming Kobold is more or less a drop-in replacement for Oobabooga, just a backend to the model but everything translates well? If no, what can I expect to lose/get from Kobold as opposed to Ooba?
- Is there a “Horde for r*tards” guide somewhere?
- What do people get from hosting Goliath-120b for others? Don’t get me wrong, I appreciate the deep pocket generosity, but is this like a data gathering operation from their point of view?
Thanks for reading this far. There’s a good doggo being very comfy hidden in the following period.
You must log in or register to comment.