From what I’ve read mac somehow uses system ram and windows uses the gpu? It doesn’t make any sense to me. Any help appreciated.

  • fallingdowndizzyvrB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Yes. MLC Chat runs great with no fuss. The same as running it on nvidia or AMD. Then things get more fussy. There’s ooba, fastchat and of course Intel’s own BigDL. The Arcs actually run on llama.cpp too, OpenCL and Vulkan, but it’s dog slow. Like half the speed of the CPU. Considering it happens in both OpenCL and Vulkan, there’s something about llama.cpp that isn’t friendly to the Arc architecture. Vulkan under MLC is fast.