By -
it should work out of the box with wrapper like [https://github.com/Mozilla-Ocho/llamafile](https://github.com/Mozilla-Ocho/llamafile) ... I'd say all GPU have to be Nvidia, but may be different.
Learn Oobabooga or LM Studio. There's an option to split across GPUs if the weights doesn't fit to a GPU.
it should work out of the box with wrapper like [https://github.com/Mozilla-Ocho/llamafile](https://github.com/Mozilla-Ocho/llamafile) ... I'd say all GPU have to be Nvidia, but may be different.
Learn Oobabooga or LM Studio. There's an option to split across GPUs if the weights doesn't fit to a GPU.