I’ve playing with a lot of models around 7B but I’m now prototyping something that would be fine with a 1B model I think, but there’s just Phi-1.5 that I’ve seen of this size, and I haven’t seen a way to run it efficiently so far. llama.cpp has still not implemented it for instance.

Anyone has an idea of what to use?

    • palpapeenOPB
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      Thanks! But I’m not looking for one that does coding, more one that’s good at detecting fallacies and reasoning. Phi-1.5 seems a better fit for that

      • LyPretoB
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        I would still give it a try— it’s misleading to think these coding models are only good at that, being good at coding actually has shown to improve its scores across multiple benchmarks.

  • vatsadevB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    RWKV 1.5B, its Sota for its size, outperforms tinyLlama, and uses no extra vram for fitting its whole ctx len in browser.

    • palpapeenOPB
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      I mean yeah but it’s not done training AFAIK, and not fine-tuned either