> ## Content Index
> Fetch the complete content index at: https://www.edgewisely.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# L40S vs H100: The Honest Bandwidth Math
- URL: https://www.edgewisely.com/l40s-vs-h100/
- Published: 2026-10-26T17:09:26.000Z
- Updated: 2026-10-26T17:09:26.000Z
- Description: L40S vs H100 compared on specs, dollars-per-hour and bandwidth-per-dollar. The L40S wins compute-per-dollar; the H100 wins bandwidth-bound LLM inference.
- Author: John Karpentar
- Tags: Chips, Cloud

**TL;DR**

- **L40S vs H100** in short: the **L40S** (48 GB GDDR6, **864 GB/s**, **$1.09/hr** on RunPod) is cheaper per hour than the **H100 SXM** (80 GB HBM3, **3.35 TB/s**, **$3.99/hr**) — but per hour is the wrong unit.
- LLM decode is memory-bandwidth-bound, and on **bandwidth-per-dollar the H100 SXM (840 GB/s per $/hr) edges out the L40S (793)**. The "cheap H100" framing breaks for inference serving.
- The L40S's real win is **compute-per-dollar**: it delivers roughly **1.36x more FP8 TFLOPS per dollar**, so it wins for vision, fine-tuning and batch throughput that fit in 48 GB.
- **No NVLink and no meaningful FP64** on the L40S rule it out for multi-GPU large-model training and HPC.

Use the **L40S** for cost-sensitive inference of models under \~13B that fit in 48 GB, vision workloads, fine-tuning, and media or graphics. Use the **H100** for large-model training, long-context or high-batch LLM serving, multi-GPU jobs needing NVLink, and FP64\. The L40S is cheaper per hour; the H100 is often cheaper per unit of work that matters.

## What's the difference between the L40S and the H100?

They're built for different jobs. The **L40S** is an Ada Lovelace card with 48 GB of GDDR6 — a universal data-center GPU tuned for graphics, media and compute-bound AI. The **H100** is a Hopper data-center accelerator with 80 GB of HBM3, built for the largest training and inference runs.

The gap that decides most AI decisions is memory. The H100 SXM moves data at **3.35 TB/s** ([NVIDIA H100 datasheet](https://bxin.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-c-series-rack-servers/nvidia-h100-80-gpu.pdf?ref=edgewisely.com)); the L40S moves it at **864 GB/s** ([NVIDIA L40S datasheet](https://www.supermicro.com/datasheet/datasheet%5FNVIDIA%5FL40S%5FSystems.pdf?ref=edgewisely.com)). That's a roughly **3.9x** bandwidth advantage for the H100, and bandwidth — not raw TFLOPS — is what governs token generation.

The second gap is interconnect. The H100 has **NVLink** (900 GB/s on SXM) for pooling GPUs into one fast memory domain. The **L40S has no NVLink** — multi-GPU traffic falls back to PCIe. If your model needs more than one GPU to train, that's a hard constraint, not a tuning knob.

## L40S vs H100 specs

Load-bearing numbers in bold. Tensor figures are the **with-sparsity** values NVIDIA headlines; dense throughput is roughly half.

| Spec                      | NVIDIA L40S      | NVIDIA H100 SXM   | H100 PCIe     |
| ------------------------- | ---------------- | ----------------- | ------------- |
| Architecture              | Ada Lovelace     | Hopper            | Hopper        |
| Memory                    | **48 GB GDDR6**  | **80 GB HBM3**    | 80 GB HBM2e   |
| Memory bandwidth          | **864 GB/s**     | **3.35 TB/s**     | 2 TB/s        |
| FP16/BF16 Tensor (sparse) | 733 TFLOPS       | **1,979 TFLOPS**  | 1,513 TFLOPS  |
| FP8 Tensor (sparse)       | **1,466 TFLOPS** | **3,958 TFLOPS**  | 3,026 TFLOPS  |
| FP64 Tensor               | none             | **67 TFLOPS**     | 51 TFLOPS     |
| Transformer Engine / FP8  | Yes              | Yes               | Yes           |
| NVLink                    | **No**           | **Yes, 900 GB/s** | Yes, 600 GB/s |
| TDP                       | **350 W**        | **700 W**         | 300–350 W     |
| Form factor               | 2-slot PCIe      | SXM               | 2-slot PCIe   |
| Typical RunPod $/hr       | **$1.09**        | **$3.99**         | $2.89         |

Specs from the NVIDIA L40S and H100 datasheets linked above; prices from [RunPod's rate card](https://www.runpod.io/gpu-cloud/pricing?ref=edgewisely.com).

For how the L40S stacks up against the previous-generation workhorse, see our [L40S vs A100 comparison](https://www.edgewisely.com/l40s-vs-a100/); for the H100 against its own predecessor and a workstation-class challenger, see [A100 vs H100](https://www.edgewisely.com/a100-vs-h100/) and [RTX Pro 6000 vs H100](https://www.edgewisely.com/rtx-pro-6000-vs-h100/).

## Which is faster for inference?

It depends on whether you're compute-bound or bandwidth-bound — and most LLM serving is bandwidth-bound.

Token generation (decode) reads the entire model and KV cache from memory for every token. Throughput scales with memory bandwidth, so the H100 SXM's 3.35 TB/s gives it close to its **3.9x** raw bandwidth edge in real decode. For interactive LLM serving, the H100 is several times faster, not marginally.

Prefill and compute-heavy workloads tell a different story. For batched vision inference, embeddings, or prompt prefill that saturates the tensor cores, the L40S's **1,466 FP8 TFLOPS** keeps it competitive — especially once you weigh price. And if a model doesn't fit in 48 GB, the L40S isn't slower, it simply can't run it without quantization or offload.

## L40S vs H100 cost: which is actually cheaper?

Per hour, the L40S is **2.7x cheaper** ($1.09 vs $3.99). Per unit of work, it depends entirely on what bounds your workload — and the arithmetic overturns the page-one assumption.

Compute the two ratios that matter from the rate card above:

| Metric              | L40S                      | H100 SXM                   | Winner           |
| ------------------- | ------------------------- | -------------------------- | ---------------- |
| Price               | $1.09/hr                  | $3.99/hr                   | L40S (2.7x)      |
| Bandwidth per $/hr  | 864 ÷ 1.09 = **793 GB/s** | 3350 ÷ 3.99 = **840 GB/s** | **H100 (1.06x)** |
| FP8 TFLOPS per $/hr | 1466 ÷ 1.09 = **1,345**   | 3958 ÷ 3.99 = **992**      | **L40S (1.36x)** |

Read those two rows carefully, because they contain the whole decision. On **bandwidth-per-dollar** — the metric that governs LLM decode — the H100 SXM is actually *slightly cheaper* than the L40S. The hourly discount is an illusion the moment your workload is memory-bound; you pay less per hour and get less bandwidth per dollar, plus you're capped at 48 GB. For token serving, the H100 is the cheaper machine.

On **compute-per-dollar**, the L40S wins by about **36%**. For a compute-bound job that fits in 48 GB — batched vision, embeddings, LoRA fine-tuning of a small model, media transcode — the L40S does more FLOPs per dollar and is genuinely the cheaper choice.

This is the arithmetic a GPU-rental vendor won't hand you, because every page-one result for this query has a commercial stake in the rental. The neutral read: "cheaper per hour" and "cheaper per token" are different claims, and for the LLM inference most people are googling this for, they point opposite directions.

## What this means for you

**Inference of small models (<\~13B, fits in 48 GB):** L40S. You get 1.36x the compute per dollar, FP8 via the Transformer Engine, and the memory headroom is adequate. The clear budget win.

**Large-model training:** H100\. You need the 80 GB, the HBM3 bandwidth, and above all NVLink to shard across GPUs. The L40S can't pool memory over NVLink, so multi-GPU training is bottlenecked on PCIe.

**Long-context / high-batch LLM serving:** H100\. This is bandwidth-bound, where the H100 wins on both raw speed (\~3.9x) and bandwidth-per-dollar. Bigger KV caches also need the 80 GB.

**Budget-constrained, throughput over latency:** L40S. If you can batch work and tolerate lower per-request speed, the compute-per-dollar advantage compounds.

**Media, graphics, transcode:** L40S, decisively. It has display-oriented hardware (NVENC/NVDEC, RT cores) the H100 lacks; the H100 is a headless compute part.

**Anything needing FP64 (HPC, scientific):** H100\. The L40S has no FP64 Tensor path; it's the wrong tool.

## The verdict

The L40S is a real H100 alternative — for the right workload, not as a blanket swap. If your job is compute-bound and fits in 48 GB, it delivers more compute per dollar and is the smart buy. If your job is bandwidth-bound LLM serving or large-model training, the hourly discount evaporates: the H100 SXM is faster *and* cheaper per unit of bandwidth, with 80 GB and NVLink the L40S can't match. Match the GPU to the bottleneck, not to the sticker price.

## Frequently Asked Questions

### Is the L40S a good alternative to the H100?

Yes, for compute-bound workloads that fit in its 48 GB — vision, fine-tuning, embeddings, media. At $1.09/hr versus $3.99/hr it offers about 1.36x more FP8 compute per dollar. It's a poor substitute for large-model training or bandwidth-bound LLM serving, where its 864 GB/s and lack of NVLink are hard limits.

### L40S vs H100: which is better for AI inference?

For bandwidth-bound LLM token generation, the H100 is better and, surprisingly, cheaper per unit of bandwidth (840 vs 793 GB/s per $/hr). For compute-bound inference of smaller models that fit in 48 GB — vision or batched throughput — the L40S wins on cost. Decode-heavy serving favors the H100; compute-heavy throughput favors the L40S.

### How much slower is the L40S than the H100?

For memory-bandwidth-bound LLM decode, expect the L40S to be roughly 3-4x slower, tracking the 864 GB/s versus 3.35 TB/s gap. For compute-bound work that saturates its tensor cores and fits in 48 GB, the gap narrows to roughly 2x on raw FP8 throughput and disappears on a per-dollar basis.

### Does the L40S support NVLink?

No. The L40S has no NVLink; multi-GPU communication falls back to PCIe (128 GB/s on Gen5). The H100 SXM supports NVLink at 900 GB/s. If your workload needs several GPUs pooled into one memory domain — large-model training especially — the L40S is the wrong card.

### Is the L40S cheaper than the H100?

Per hour, yes — about 2.7x cheaper ($1.09 vs $3.99 on RunPod). Per unit of work, only sometimes. It's cheaper for compute-bound jobs that fit in 48 GB, but for bandwidth-bound LLM serving the H100 SXM is actually slightly cheaper once you divide bandwidth by price.

---

**Editor's note — sources:** NVIDIA L40S datasheet (L40S architecture, 48 GB GDDR6, 864 GB/s, 1,466 FP8 TFLOPS sparse, 350 W); NVIDIA H100 datasheet (80 GB, 3.35 TB/s SXM, tensor throughput, NVLink 900 GB/s, 700 W); RunPod pricing (L40S $1.09/hr, H100 PCIe $2.89/hr, H100 SXM $3.99/hr — the basis for all per-dollar arithmetic, fetched 2026-10-26). All linked inline above. Tensor figures are with-sparsity; dense is \~half. Memory types (HBM3/HBM2e/GDDR6) are NVIDIA's established Hopper and Ada specs.