NVIDIA inference software keeps driving down token costs, long after AI infrastructure is deployed. ⚡ In just one month…
By nvidia · AI Infra
NVIDIA inference software keeps driving down token costs, long after AI infrastructure is deployed. ⚡ In just one month on NVIDIA Blackwell, software optimizations improved DeepSeek V4 performance by up to 5×, reducing token costs to roughly one-fifth of previous levels. NVIDIA's integrated inference software stack compounds improvements across runtimes, kernels, networking, and hardware, delivering up to 20× higher throughput on the same GPU. Co-designed with NVIDIA GPUs, CPUs, networking,…