Is this accurate?

  • CardAnarchistB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Can you offload layers with this like GGUF?

    I don’t have much VRAM / RAM so even when running a 7B I have to partially offload layers.