- Stars
- 134k
- Forks
- 25.3k
- Licence
- MIT
- Language
- C++
- Version
- 0.9.0
- Updated
At a glance
- Runs quantised models on CPUs, Apple Silicon and most GPUs
- Defines the GGUF model format used across the ecosystem
- Bundled OpenAI-compatible HTTP server
Get started
Homebrew
brew install llama.cppWhat it is
llama.cpp runs language models with minimal dependencies. It introduced the GGUF format and aggressive quantisation, which shrink models enough to run on ordinary laptops.
Why it stands out
Many other local-AI tools, Ollama among them, build on it. It supports Metal, CUDA, Vulkan and other backends, and ships llama-server for an HTTP API.
Who it is for
Developers who want the most direct, lowest-overhead way to run open models locally.
