Maybe. I have not done it yet so I don’t know. You can google around.

You can use oobabooga API to do that. I haven’t done it myself, can’t say much about it.

You can start with reading Oobabooga’s wiki, I think it’s one of most beginner friendly tools. https://github.com/oobabooga/text-generation-webui/wiki/05-‐-Training-Tab

If I can run them all I will just pick the biggest one.

“Write the snake game using pygame”

Thanks for sharing! I have been struggling with llama.cpp loader and GGUF (using oobabooga and the same LLM model), no matter how I set the parameters and how many offloaded layers to GPUs, llama.cpp is way slower to ExLlama (v1&2), not just a bit slower but 1 digit slower. I really don’t know why.

In my experience it’s the fastest and llama.cpp is the slowest.

Thank you! It looks very deep to me, I will look into it.

Thanks! I have some problems to load GPTQ models with transformer loader.

Thanks for sharing!

I have 2 gpus and AWQ never works for me on Oobabooga, no matter how I split the vRAM, oom in most of the cases.