> ## Content Index > Fetch the complete content index at: https://www.edgewisely.com/llms.txt > Use this file to discover other available public pages before exploring further. # vLLM vs SGLang: How to Pick an Inference Engine - URL: https://www.edgewisely.com/vllm-vs-sglang/ - Published: 2026-09-24T04:21:16.000Z - Updated: 2026-09-24T04:21:16.000Z - Description: vLLM uses PagedAttention and hash-based prefix caching. SGLang uses a radix tree. That one difference decides which engine suits your traffic, and it is not something a benchmark run on someone else's GPUs can tell you. - Author: John Karpentar - Tags: AI, Engineering **TL;DR** - **vLLM** (Apache 2.0, **v0.30.0**, released 22 Sep 2026) is the broad-coverage default: PagedAttention, hash-based prefix caching, and the widest hardware matrix of any open LLM inference engine. - **SGLang** (Apache 2.0, **v0.5.20**, released 18 Sep 2026) is built around **RadixAttention**, a radix-tree KV cache that wins when prompt prefixes repeat — agents, multi-turn chat, RAG. - The two architectures diverge on **how the KV cache is indexed**, not on raw kernel speed. That difference decides which one suits your traffic. - Ignore single-config benchmarks. Neither project publishes a neutral head-to-head, and every number on page one of Google was measured on hardware you probably don't have. **vLLM vs SGLang** comes down to memory architecture. vLLM builds on PagedAttention and hash-based prefix caching, with the widest hardware and model coverage. SGLang builds on RadixAttention, a radix-tree prefix cache tuned for workloads that reuse long prompts. Pick vLLM as the default; pick SGLang for agentic, multi-turn and heavily structured workloads. ## What is vLLM? vLLM is an open-source **LLM inference engine** that serves models over an OpenAI-compatible HTTP API. Its founding idea is PagedAttention. The [PagedAttention paper](https://arxiv.org/abs/2309.06180?ref=edgewisely.com) (SOSP 2023) treats KV cache memory the way an operating system treats RAM: fixed-size blocks, a page table, near-zero fragmentation. The authors reported **2–4x throughput** over FasterTransformer and Orca at equal latency — a 2023 comparison against 2023 baselines, so read it as a design claim, not a current scoreboard. That is also the answer to "what is vLLM in AI": it is serving infrastructure, not a model. You point it at Hugging Face weights and it gives you a production endpoint. Scale of the project matters here. The **v0.30.0** release alone landed **762 commits from 315 contributors**, per the [vLLM releases page](https://github.com/vllm-project/vllm/releases?ref=edgewisely.com). vLLM's installation docs list NVIDIA CUDA, AMD ROCm, Intel XPU, Apple Silicon, and CPU backends spanning x86, ARM AArch64 and IBM Z — plus out-of-tree hardware plugins including TPU. Nothing else in this category has that footprint. ## What is SGLang? SGLang is a serving framework that started as a **language for programming LLMs**, not just a server. That origin still shapes it. The [SGLang paper](https://arxiv.org/abs/2312.07104?ref=edgewisely.com) describes two runtime optimisations: RadixAttention for KV cache reuse, and compressed finite state machines for faster constrained decoding. It reported **up to 6.4x higher throughput** versus contemporaneous systems on agent control, JSON decoding, RAG and multi-turn chat — again, a 2023/2024 measurement. Note the workload list. Those are all cases where many requests share a long prefix. SGLang is hosted under the non-profit LMSYS organisation. Its [release history](https://github.com/sgl-project/sglang/releases?ref=edgewisely.com) shows the same weekly cadence as vLLM, and its hardware-platforms index covers NVIDIA and AMD GPUs, Ascend NPUs, CPU servers, Jetson Orin, TPU, XPU and Moore Threads GPUs. ## vLLM vs SGLang: the differences that matter The engines converged on most features. They diverge on cache indexing. vLLM hashes each KV block by its tokens plus the hash of its parent block. Per the [vLLM prefix caching design doc](https://docs.vllm.ai/en/stable/design/prefix%5Fcaching/?ref=edgewisely.com), the default algorithm has been **sha256 since v0.11**, with `xxhash` available for speed at some collision risk. Lookups are exact-match on full blocks. SGLang keeps prefixes in a **radix tree** and evicts leaves under LRU. A tree gives you partial-prefix matching and branch-aware eviction for free; a hash table does not. | | vLLM | SGLang | | ------------------------------ | ---------------------------------------------------------- | ------------------------------------------------------------ | | **Core KV mechanism** | PagedAttention + hash-based block cache | RadixAttention (radix tree) | | **Prefix match granularity** | Full blocks, exact hash | Tree path, partial prefixes | | **Cache eviction control** | Block-level, hash algorithm configurable | lru default, plus lfu, slru, priority, tlru | | **Structured output backends** | xgrammar, guidance (auto default) | XGrammar (default), Outlines, Llguidance | | **Hardware breadth** | CUDA, ROCm, XPU, Apple Silicon, x86/ARM/S390X CPU, plugins | CUDA, ROCm, Ascend NPU, TPU, XPU, Jetson, CPU, Moore Threads | | **Licence** | Apache 2.0 | Apache 2.0 | | **Latest release** | v0.30.0, 22 Sep 2026 | v0.5.20, 18 Sep 2026 | | **GitHub stars** | \~92.6k | \~36.4k | Both ship speculative decoding, prefill/decode disaggregation, quantised KV cache and hierarchical cache offload. Feature parity is close enough that feature checklists are a bad way to choose. ![Diagram from the SGLang paper illustrating RadixAttention prefix sharing across requests](https://storage.ghost.io/c/54/5a/545a66b3-60ef-480c-80ae-765bac52f6ec/content/images/2026/09/sglang-radixattention-figure.jpg) Figure: [SGLang paper (arXiv:2312.07104)](https://arxiv.org/abs/2312.07104?ref=edgewisely.com) ## Why published benchmarks won't settle sglang vs vllm for you Every benchmark on page one was run on one GPU configuration, one model, one request mix, one week. All four variables move the result. Three specific reasons published numbers mislead: 1. **Prefix reuse rate dominates.** If 80% of your prompt tokens are a shared system prompt, cache architecture decides throughput. A benchmark with random prompts measures kernels instead. 2. **Both projects ship weekly.** vLLM and SGLang both cut releases within the last week of this writing. A benchmark three months old predates several attention backends. 3. **Publisher incentive.** Most ranking comparisons are GPU-cloud vendor blogs. They benchmark on the hardware they rent. Run your own. Replay a day of real production traffic through both engines on the GPUs you actually have, and measure p99 time-to-first-token alongside throughput. Two days of work beats any blog. Your retrieval design shifts the answer too — [whether you lean on RAG or long context](https://www.edgewisely.com/rag-vs-long-context-research-disagrees/) changes how much prefix your requests actually share. ## vLLM vs TensorRT-LLM: where that fits TensorRT-LLM is NVIDIA's engine, and by its own repo description it targets NVIDIA GPUs only. That is the deciding fact. If your fleet is mixed, or might be, it's out. If you are NVIDIA-only, latency-bound, and willing to accept an ahead-of-time engine-build step per model and per config, it belongs on your shortlist. At roughly **14.7k stars** against vLLM's 92.6k, the community and model-coverage gap is large. Accelerator choice constrains this more than most teams expect, and [the chip you standardise on](https://www.edgewisely.com/top-7-ai-chip-makers-training-inference-2026/) often decides the engine for you. ## What this means for you **If you're serving one model on one GPU:** use vLLM. Installation is simpler, documentation is deeper, and you will find your error message in a GitHub issue. **If you run agents or multi-turn chat:** try SGLang first. Long shared prefixes across turns is exactly the case RadixAttention was designed for, and the `tlru` eviction policy exists specifically for agentic tail latency. **If structured output is the bottleneck:** test both with XGrammar, which is the default in SGLang and available in vLLM. The engine matters less than the grammar backend here. **If you run a multi-tenant inference platform:** vLLM. Cache-salt isolation, the breadth of quantisation formats and the contributor base all favour it at platform scale. **If you're on AMD, Ascend or TPU:** check each project's hardware page for your exact chip before anything else. Support tiers differ sharply below the NVIDIA line. **If you'd rather not run either:** a managed endpoint removes the question entirely. We compared [the leading managed inference providers](https://www.edgewisely.com/top-7-ai-inference-providers-for-production-llm-workloads-in-2026/) separately, and [the gateway layer that sits in front of them](https://www.edgewisely.com/litellm-vs-openrouter/) is a different decision again. ## Frequently Asked Questions ### What is vLLM? vLLM is an open-source LLM inference engine, Apache 2.0 licensed. It serves models over an OpenAI-compatible HTTP API and uses PagedAttention to manage KV cache memory in fixed-size blocks, which cuts fragmentation and raises throughput. It is the de facto default for self-hosted inference today, with the broadest hardware support in the category. ### Is SGLang faster than vLLM? Sometimes, on some workloads, on someone else's hardware. Neither project publishes a neutral head-to-head. SGLang's radix-tree cache tends to win where prompt prefixes repeat heavily — agents, multi-turn chat, RAG. vLLM often closes the gap after tuning. Benchmark both on your own traffic before believing any published number. ### Does vLLM support GGUF? Yes, but treat it as experimental. vLLM's own documentation calls GGUF support highly experimental and under-optimised, and warns it may be incompatible with other features. Support has moved out of tree into a separate `vllm-gguf-plugin` package you install yourself. Use GGUF to cut memory footprint, not to chase throughput. ### Which inference engine should you use in production? Default to vLLM unless you have a specific reason not to. It has the broadest hardware support, the largest contributor base and the most deployment tooling. Move to SGLang when your traffic is prefix-heavy or structure-heavy and you have measured the difference. Consider TensorRT-LLM only if you are NVIDIA-only and latency-bound. --- **Editor's note — sources:** Version numbers, release dates, licences and documented behaviour were verified on 24 September 2026 against the vLLM and SGLang release pages, the vLLM prefix-caching design and [GGUF quantisation](https://docs.vllm.ai/en/stable/features/quantization/gguf/?ref=edgewisely.com) documentation, and the PagedAttention and SGLang arXiv papers — all linked inline above. GitHub star counts are approximate and drift daily. No cross-engine benchmark figures are asserted here: the only performance numbers cited are each paper's own claims against 2023-era baselines, and both are dated.