I have a budget of about £3000, though extends to £4500 for the possibility of having 2 GPUs. Currently looking at using the Mistral 7B model. I might get away with simply deploying the model and giving it context in the prompt for my project. However it may require training to learn domain specific context, which I’d do through something like QLoRA.

I’ll probably be going with 1x 3090 to begin with, potentially get another if training is required. In particular I’m struggling with finding a case that can sufficiently house and cool 2x 3 slot graphics cards. A lot of the options with vertical mounts press the GPU cooler right up against the side panel which worries me.

Here’s what I’m thinking so far:

  • Mobo: ROG STRIX X670E-E GAMING WIFI
    • Supports PCI-E Gen 5.0 x16 on 2 slots both on CPU (I’ll only need PCI-E Gen 4.0 8x for each). Struggled finding mobos that could do this.
  • CPU: AMD 7900X
    • Good price to performance. Decent number of cores.
  • GPU: MSI GeForce RTX 3090 VENTUS 3X 24G OC GeForce RTX 3090 24 GB
    • Satisfies VRAM requirements
  • RAM: Corsair Vengeance 2x32GB 5200Mhz
    • Sounds like RAM is never a bottleneck so saving money with slower memory. Can upgrade to 128GB later. Unsure why a machine would need a lot of RAM when everything needs to fit on 24GB VRAM anyway, any advice welcome on this matter.
  • SSD: Western Digital Blue SN580
  • PSU: Corsair HX1500i (2023)
    • 1500W plenty for 2 cards. 2023 model allows for better future upgradeability
  • Cases: ECLIPSE P600S or Fractal Meshify 2XL
    • Both have opton for vertical mount with 3 rear PCI-E slots so look like the best bet for fitting 2 cards in and getting good air flow. Still worried about the spacing though.
    • The case also needs to be relatively easily movable… Just moving around the office. So open air test benches feel like they’d be too fragile.

Also have to factor in £100-200+ for case fans + perhipherals, PCI-E riser cables if needed.

Additionally, not sure if the NVlink bridge for £100 would help. I was wondering if it would alleviate the need for 2x pci-e slots with Gen 4.0 16x or 8x, as most motherboards seemed to only have 4x on the second slot, or even 4x on the chipset for the second slot.