Longjumping-Bake-557B to LocalLLaMA@poweruser.forumEnglish · 10 months agoDiscrepancy between TheBloke_Orca-2-13B-GPTQ and the original one with the tested logic questionimagemessage-square2fedilinkarrow-up11arrow-down10
arrow-up11arrow-down1imageDiscrepancy between TheBloke_Orca-2-13B-GPTQ and the original one with the tested logic questionLongjumping-Bake-557B to LocalLLaMA@poweruser.forumEnglish · 10 months agomessage-square2fedilink
minus-squarepanchovixBlinkfedilinkEnglisharrow-up1·10 months agoTry at FP16 or 8bit at most, probably a 13B models suffers too much at 4bits.
Try at FP16 or 8bit at most, probably a 13B models suffers too much at 4bits.