Like many of you, I often need to train LLMs (Large Language Models). Code hops from one project to another, and it’s easy to lose track, resulting in several iterations of the same training process.
X—LLM is a solution. It’s a streamlined, user-friendly library designed for efficient model training, offering advanced techniques and customizable options within the Hugging Face ecosystem.
Features:
- LoRA, QLoRA and fusing
- Flash Attention 2
- Gradient checkpointing
- bitsandbytes quantization
- GPTQ (including post-training quantization)
- W&B experiment tracking
- Simple training on multiple GPUs at once using DeepSpeed or FSDP
Use cases:
- Create production-ready solutions or fast prototypes. X—LLM works in both configurations
- Finetune a 7B model with 334 million tokens (1.1 million dialogues) for just 50$
- Automatically save each checkpoint during training to the Hugging Face Hub and don’t lose any progress
- Quantize a model using GPTQ. Reduce 7B Mistral model from 15 GB to 4.3 GB and increase inference speed
Github repo: https://github.com/BobaZooba/xllm
You can train 7B model, fuse LoRA and upload ready-to-use model to the Hugging Face Hub. All in a single Colab! Link
The library has gained 100 stars in less than a day, and now it’s almost at 200. People are using it, training models in both Colab and multi-GPU setups. Meanwhile, I’m supporting X—LLM users and currently implementing the most requested feature - DPO.
I suggest that you try training your own models and see for yourself how simple it is.
If you like it, please consider giving the project a star on GitHub.
https://pytorch.org/docs/stable/notes/mps.html