2 posts tagged with "GPU"

Following up on the earlier article about building llama.cpp with CUDA support on Windows, I tried the same thing on a Fedora 43 environment. Only the toolchain changes — the overall flow is almost identical.

A record of building llama.cpp, the go-to local LLM inference engine, from source on Windows 11 with the CUDA backend enabled for an NVIDIA GPU, through to actually running a model.