Add QuIP# support by oobabooga · Pull Request #4803 · oobabooga/text-generation-webui
github.com
QuIP# is a novel quantization method. Its 2-bit performance is better than anything previously available. Repository: https://github.com/Cornell-RelaxML/quip-sharp Blog post: https://cornell-relaxm...
  • a_beautiful_rhindBEnglish
    1·
    1 year ago

    From the issue about this in the exllamav2 repo, quip was using more memory and slower than exl. How much context can you fit?