currytrash97B to LocalLLaMA@poweruser.forumEnglish · 11 months agoA100 inference is much slower than expected with small batch sizeplus-squaremessage-squaremessage-square2fedilinkarrow-up11arrow-down10
arrow-up11arrow-down1message-squareA100 inference is much slower than expected with small batch sizeplus-squarecurrytrash97B to LocalLLaMA@poweruser.forumEnglish · 11 months agomessage-square2fedilink
currytrash97B to Machine Learning@academy.gardenEnglish · 11 months ago[P][D] A100 is much slower than expected at low batch size for text generationplus-squaremessage-squaremessage-square7fedilinkarrow-up11arrow-down10
arrow-up11arrow-down1message-square[P][D] A100 is much slower than expected at low batch size for text generationplus-squarecurrytrash97B to Machine Learning@academy.gardenEnglish · 11 months agomessage-square7fedilink
