I use in both cases q4_K_M

  • meetraisB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Same experience here. I got excellent results from quantized models of Intel-Neural-7B and Mistral-7B but bad results with quantized model of Yi-34B.