My longest context length is 4k. I want to fine tune Llama2-7b model on this dataset. How much RAM I will need if I load model with 4bit quantization using the bitsandbyte.
I faced with OutOfMemory Error with 24G RAM.
Thank you!
You must log in or register to comment.