Longjumping-Bake-557B to LocalLLaMA@poweruser.forumEnglish · 2 years agoDiscrepancy between TheBloke_Orca-2-13B-GPTQ and the original one with the tested logic questionimagemessage-square2linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1imageDiscrepancy between TheBloke_Orca-2-13B-GPTQ and the original one with the tested logic questionLongjumping-Bake-557B to LocalLLaMA@poweruser.forumEnglish · 2 years agomessage-square2linkfedilink
minus-squarepanchovixBlinkfedilinkEnglisharrow-up1·2 years agoTry at FP16 or 8bit at most, probably a 13B models suffers too much at 4bits.
Try at FP16 or 8bit at most, probably a 13B models suffers too much at 4bits.