> ## Content Index > Fetch the complete content index at: https://www.edgewisely.com/llms.txt > Use this file to discover other available public pages before exploring further. # Ollama vs llama cpp: Which Should You Use? - URL: https://www.edgewisely.com/ollama-vs-llama-cpp/ - Published: 2026-10-01T06:24:33.000Z - Updated: 2026-10-01T06:25:42.000Z - Description: Most pages ranking for this question are stale. Verified against both repositories: Ollama v0.35.0 still pins and runs llama.cpp's llama-server, only about 80 nightly builds behind upstream, plus its own MLX engine on Apple Silicon. Here is which one you should actually use. - Author: John Karpentar - Tags: AI, Engineering ## TL;DR - **Ollama runs llama.cpp.** As of Ollama **v0.35.0** (28 September 2026), the repository pins a llama.cpp build in a root file called `LLAMA_CPP_VERSION`, fetches that source at build time, applies its own patches, and drives llama.cpp's `llama-server` binary. Ollama's README lists exactly one supported backend: llama.cpp. - **The pin is current, not stale.** Ollama pins build **b11232** (28 September 2026) against llama.cpp's **b11312** (1 October 2026) — about **80 nightly builds and three days** apart. Claims that Ollama runs months-old llama.cpp code are wrong. - **Two engines, not one.** Ollama also ships its own Go bindings to Apple's **MLX**, used on Apple Silicon. GGUF models everywhere else go through llama.cpp. - **llama.cpp is not CLI-only.** `llama serve` is an OpenAI-compatible HTTP server with continuous batching, a web UI, reranking and speculative decoding. Its `--parallel` default is **auto**; Ollama's `OLLAMA_NUM_PARALLEL` default is **1**. - **Verdict:** use Ollama if you want a model running in two minutes. Use llama.cpp directly if you need every tuning dial or more than one concurrent user. ## Ollama vs llama cpp: the direct answer In the **ollama vs llama cpp** comparison, these are not competing engines. llama.cpp is the C/C++ inference engine; Ollama is a model manager and server that runs llama.cpp's `llama-server` underneath for GGUF models, plus its own MLX engine on Apple Silicon. Pick Ollama for convenience and model management, llama.cpp for control and configurability. ## Why page one for this query is unreliable Search this question today and the top results are mostly a Reddit thread, a 2024 Hacker News discussion, a Medium post, a Hugging Face community post and a YouTube video. The commercial results are not much better. One high-ranking page sits on an unofficial domain that borrows the project's name. Another is published by a company selling on-device voice AI. A third is Red Hat — a major backer of vLLM — writing about vLLM versus llama.cpp, which is a different pairing entirely. Almost none of it is current. The engine relationship between these two projects changed twice during 2026, and most of what ranks predates both changes. Everything below is checked against the two repositories and their documentation. Where we could not verify a number, we say so. ## What is the difference between Ollama and llama.cpp? llama.cpp is the inference engine. It is a C/C++ project built on the **ggml** tensor library, MIT-licensed, and hosted at [ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp?ref=edgewisely.com). It defines the **GGUF** format, ships the quantisation tooling, and is the upstream for most of the local-inference ecosystem — LM Studio, Jan, KoboldCpp and Ollama all descend from it. Ollama is the product layer. It gives you `ollama run`, a model registry with versioned tags, automatic GPU offload, a REST API on port **11434**, and a scheduler that decides what fits in VRAM. The useful mental model: llama.cpp is the engine, Ollama is the car. ## Is Ollama just a wrapper for llama.cpp? Largely yes for GGUF models, and the repository says so plainly — but "wrapper" undersells the orchestration and misses the MLX path. Ollama's own [llama/README.md](https://github.com/ollama/ollama/blob/main/llama/README.md?ref=edgewisely.com) documents the mechanism: `LLAMA_CPP_VERSION` "pins Ollama's llama.cpp source", the build fetches that pinned source, and compatibility patches under `llama/compat/` are applied during configure. The same document describes reviewing "llama-server contracts: launch args and defaults, status and error payloads" consumed by Ollama's Go code. So for GGUF models, Ollama builds a patched llama.cpp and runs `llama-server` as a child process. Ollama's Go code handles device discovery, memory accounting, model scheduling, prompt templating and the API surface. llama.cpp does the inference. The qualifier is MLX. The repository pins separate `MLX_VERSION` and `MLX_C_VERSION` files and contains Go bindings to Apple's MLX framework, used on Apple Silicon. Ollama now publishes MLX-format model variants alongside GGUF ones. **One correction worth making.** Ollama announced a standalone Go engine in 2025, and a lot of commentary concluded it had left llama.cpp behind. The opposite happened. Ollama **0.30** in June 2026 was explicitly about *expanding* GGUF compatibility "through llama.cpp", and that release credits Georgi Gerganov and the llama.cpp maintainers by name. The README's supported-backends list today contains one entry, and it is llama.cpp. ## Ollama vs llama.cpp: feature comparison | | **Ollama** (v0.35.0) | **llama.cpp** (b11312) | | ------------------------ | ---------------------------------------------------------------- | ---------------------------------------------------------------------------- | | **Inference engine** | llama.cpp llama-server for GGUF; own MLX engine on Apple Silicon | Its own — ggml + llama.cpp | | **Model format** | GGUF, plus MLX-format variants | GGUF (defines the spec) | | **API server** | REST on :11434, OpenAI-compatible | llama serve: OpenAI chat/responses/embeddings, Anthropic Messages, reranking | | **Model management** | Registry, tags, ollama pull, auto-unload | \-hf flag pulls from Hugging Face; no registry | | **Quantisation control** | Pick from published tags | Full matrix, plus llama-quantize | | **Concurrency default** | OLLAMA\_NUM\_PARALLEL \= **1**, queue 512 | \--parallel \= **auto**, continuous batching | | **Hardware backends** | CUDA, ROCm, Metal, Vulkan, CPU | CUDA, HIP, Metal, Vulkan, SYCL, CANN, MUSA, OpenCL, WebGPU, CPU and more | | **Install** | One-line installer, native apps, Docker | Prebuilt binaries, installer, Docker, source | | **Target user** | Developers who want it working now | Engineers tuning one machine, and tool builders | | **Licence** | MIT | MIT | ## Which is faster, Ollama or llama.cpp? On identical hardware, the same model and the same settings, they should land in roughly the same place — because it is the same engine doing the work. We could not find a head-to-head tokens-per-second benchmark published by either project, and we are not going to launder forum numbers into one. The version-lag theory you will see repeated does not hold up either. Ollama pins llama.cpp build **b11232**, published 28 September 2026\. llama.cpp's current nightly tag is **b11312**, published 1 October 2026\. That is roughly **80 builds and three days** of drift. llama.cpp tags builds continuously — several a day — so a gap of 80 builds is a matter of days, not months. Ollama tracks upstream closely. Where real differences come from is **configuration**, not engine version. Ollama picks conservative defaults for a single-user desktop. llama.cpp exposes the dials and defaults to using them. That distinction is the whole story, and it is the one most comparisons miss. Ollama's own measured claims are worth citing because they are specific about conditions. Release **0.30** reports NVIDIA throughput "up to 20% faster", tested with Gemma 4 26B on an RTX 5090 at Q4\_K\_M, crediting optimisations from the NVIDIA and llama.cpp teams. That is a vendor-published figure on a single configuration — useful, not general. ![Ollama 0.30 NVIDIA throughput chart, tested with Gemma 4 26B on an RTX 5090 at Q4_K_M](https://storage.ghost.io/c/54/5a/545a66b3-60ef-480c-80ae-765bac52f6ec/content/images/2026/10/body_nvperf.jpg) Chart: [Ollama](https://ollama.com/blog/improved-performance-and-model-support-with-gguf?ref=edgewisely.com) — measured on Gemma 4 26B, RTX 5090, Q4\_K\_M ## Does llama.cpp have an API server? Yes, and this is the most common outdated claim in this comparison. You do not need Ollama to get an API. The [llama-server documentation](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md?ref=edgewisely.com) lists OpenAI-compatible chat completions, responses and embeddings routes, Anthropic Messages API compatible completions, a reranking endpoint, parallel decoding with multi-user support, continuous batching, multimodal input, schema-constrained JSON, function calling, speculative decoding, and a web UI that is enabled by default. It also takes `--api-key` for authentication. One command gets you a served model with an OpenAI-compatible endpoint. If you are routing between several such endpoints, that is a different problem — we covered it in [LiteLLM vs OpenRouter](https://www.edgewisely.com/litellm-vs-openrouter/). ## What about concurrency and multiple users? This is where the two genuinely diverge, and where a widely shared claim that Ollama "collapses at 5 concurrent users" needs correcting. We could not verify that figure. We can explain the behaviour behind it from documentation. Ollama's [FAQ](https://docs.ollama.com/faq?ref=edgewisely.com) states that `OLLAMA_NUM_PARALLEL` — "the maximum number of parallel requests each model will process at the same time" — **defaults to 1**. Excess requests queue, up to `OLLAMA_MAX_QUEUE` (default **512**), after which the server returns **503**. Raising parallelism costs memory: required RAM scales with `OLLAMA_NUM_PARALLEL` multiplied by `OLLAMA_CONTEXT_LENGTH`, and the default context is only **4096** tokens. So a single Ollama instance at defaults serialises requests. That is a configuration default, not a collapse — but the user-visible symptom under concurrent load is the same, and most people never change it. llama-server sets `--parallel` to **auto** and batches continuously. It is the better-configured default for multi-user work. Neither, though, is what you want for serving real concurrent traffic at scale. That is vLLM, SGLang or TensorRT-LLM territory, where paged attention and high-throughput scheduling are the design goal. We compared the two leading options in [vLLM vs SGLang](https://www.edgewisely.com/vllm-vs-sglang/). ## Who maintains these projects, and did anything change? Both remain **MIT-licensed**. Governance changed significantly on the llama.cpp side. In February 2026, [ggml and llama.cpp joined Hugging Face](https://huggingface.co/blog/ggml-joins-hf?ref=edgewisely.com). Hugging Face's announcement states that Georgi Gerganov and team "still dedicate 100% of their time maintaining llama.cpp" with full autonomy over technical direction, and that the project stays open-source and community driven. Then on 3 September 2026, NVIDIA announced a definitive agreement to acquire Hugging Face — roughly **$11.9 billion** to stockholders plus up to about **$1.0 billion** in employee retention equity, expected to close in the first half of 2027 subject to regulatory approval. So llama.cpp's institutional steward is, pending that close, becoming a GPU vendor. Worth knowing. No change to the licence or the repository's open governance has been announced. ## What this means for you **If you just want to run a model locally today.** Use Ollama. One install, one command, done. You get GPU offload, model management and an API without reading a build guide. Raise `OLLAMA_CONTEXT_LENGTH` above 4096 before you do anything serious. **If you are embedding local inference in an application.** Either works. Ollama's registry and auto-unload are real operational wins. llama.cpp gives you a smaller dependency surface and an endpoint you fully control, and its prebuilt binaries are easier to bundle than a background service. **If you are squeezing maximum performance out of one machine.** llama.cpp, directly. You get the full quantisation matrix, speculative decoding with draft models, multi-GPU split modes and sampler controls Ollama does not surface. On Apple Silicon, test Ollama's MLX path too — it is a genuinely different engine and may win there, though we found no independent comparison against llama.cpp's Metal backend. **If you are serving multiple concurrent users.** Be honest that neither is the right tool past a handful of users. llama-server's auto parallelism and continuous batching are the better of the two. Beyond that, move to vLLM or SGLang, or a managed endpoint — we compared those in our [AI inference provider roundup](https://www.edgewisely.com/top-7-ai-inference-providers-for-production-llm-workloads-in-2026/). **If you are deploying to constrained or edge hardware.** llama.cpp's backend list is the deciding factor: Vulkan, SYCL, OpenCL and WebGPU reach silicon that CUDA-first stacks do not. Our [edge AI platforms and chips comparison](https://www.edgewisely.com/top-7-edge-ai-platforms-and-chips-2026/) covers the hardware side. ## One quantisation gotcha Bare model tags do not pin a single file. On Ollama's library a `latest` tag resolves to a specific quantisation — typically a **Q4\_K\_M** GGUF build — or to an MLX build depending on your hardware, and the listed download size can be shown as a range for that reason. If you care which weights you are running, name the quantisation explicitly rather than relying on the default tag. ## Frequently Asked Questions ### Is Ollama just a wrapper for llama.cpp? For GGUF models, largely yes. Ollama's repository pins a llama.cpp build in `LLAMA_CPP_VERSION`, patches it, and runs llama.cpp's `llama-server` as a subprocess. Its README lists llama.cpp as the only supported backend. The exception is Apple Silicon, where Ollama uses its own MLX engine instead. ### Which is faster, Ollama or llama.cpp? Same engine, same hardware, same settings means roughly the same speed. Ollama's pinned llama.cpp build is only about 80 nightly builds behind upstream — days, not months — so version lag is not the explanation people assume. Differences come from defaults, chiefly parallelism and context length. We found no credible head-to-head benchmark from either project. ### Does llama.cpp have an OpenAI-compatible API? Yes. `llama serve` exposes OpenAI-compatible chat completions, responses and embeddings routes, plus Anthropic Messages API compatibility, a reranking endpoint, function calling, continuous batching and a web UI enabled by default. It also supports `--api-key` authentication. You do not need Ollama to get an HTTP API. ### Should I use Ollama or llama.cpp for a production app? llama.cpp if you need control over parallelism and predictable versioning. Ollama if operational convenience and its model registry matter more. For genuine multi-user serving, use vLLM or SGLang instead — neither of these two is designed for that load. ### Which project defines the GGUF format? llama.cpp, via the ggml project. Ollama consumes GGUF files; it does not define the format. This is why Ollama can run models quantised by third parties, and why GGUF builds published for llama.cpp generally work in Ollama too. --- **Editor's note — sources.** The engine relationship, the `LLAMA_CPP_VERSION` pinning mechanism and the patching process come from Ollama's own `llama/README.md`. Version facts were read directly from both repositories on 1 October 2026: Ollama v0.35.0 (published 28 September 2026) pins llama.cpp build b11232 (28 September 2026), against llama.cpp's then-current tag b11312 (1 October 2026). Concurrency defaults, queue behaviour and the memory-scaling rule come from Ollama's FAQ. The llama-server capability list comes from its documentation in the llama.cpp repository. The 20% NVIDIA throughput figure and its test conditions come from Ollama's 0.30 release post. The ggml/Hugging Face governance change comes from Hugging Face's own announcement. NVIDIA's agreement to acquire Hugging Face was announced on 3 September 2026 and verified against NVIDIA's SEC Form 8-K filing; it is not linked here, to stay within our source cap. We could not verify the following and have left it out. The claim that Ollama "collapses at 5 concurrent users" has no primary source; we document the default that plausibly produces that symptom instead and do not assert the figure. No head-to-head tokens-per-second benchmark exists from either project, so no such number appears above. Ollama's MLX path publishes no comparison against llama.cpp's Metal backend, so the Apple Silicon question is open. The build-lag figures are a snapshot taken on 1 October 2026 and will drift as Ollama bumps its pin.