• AaaaaaaaaeeeeeB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    (With a massive batch size*)

    Its would be better if they provide single batch information for normal inference on fp8.

    People look at this and think its astonishing, but will compare this with single batch performances as that’s all they have seen before.