The NVIDIA H200 is a mid-cycle refresh of the Hopper architecture, released in late 2024. The H200 retains the same compute throughput as the H100 but nearly doubles memory capacity (from 80GB to 141GB HBM3e) and substantially increases memory bandwidth (from 3 TB/s to 4.8 TB/s).
The H200 was specifically engineered to address the memory bottleneck in inference for very large models. Modern frontier LLMs have grown to the point where fitting model weights and KV cache in 80GB was constraining batch sizes and context lengths; the 141GB of the H200 gives inference serving substantial breathing room.
For enterprise AI deployment through 2026, the H200 is the preferred choice for new inference capacity. Training workloads continue to use H100s (for existing clusters) or B200s (for new frontier capacity). The distinction between training-optimized and inference-optimized hardware is now a meaningful planning consideration for large-scale AI deployments.