Day 0 support for Mistral Large 4 on vLLM
·27 min read
Day-0 vLLM support for Mistral Large 4 with sparse MLA, cross-layer KV sharing, DSpark speculative decoding, DCP/PCP long context and NIXL P/D on NVIDIA Blackwell.
#model-support
24 posts
Day 0 support for Muse Spark Open on vLLM
·9 min read
Day-0 vLLM support for Muse Spark Open: 816B MoE, 1M context, multimodal inputs, DSpark speculative decoding, NVIDIA and AMD serving, initial TPU support, and LoRA RL.
Announcing Day 0 Support for NVIDIA Nemotron 3.5 Super VL on vLLM
·6 min read
Run NVIDIA Nemotron 3.5 Super VL on vLLM for multimodal agent workflows with image, document, and video understanding, OpenAI-compatible serving, and multi-token prediction.
Day 0 Support for Qwen3.8-2.4T-A95B on vLLM
·4 min read
Day-0 vLLM support for Qwen3.8-2.4T-A95B: a 2.4-trillion-parameter hybrid MoE model served out of the box, with FP8/BF16 checkpoints plus NVFP4 and MXFP4 quantized weights, and co-developed kernels on NVIDIA and AMD hardware.
Announcing Day-0 Support for NVIDIA Nemotron 3.5 Lightning on vLLM
·11 min read
How vLLM serves NVIDIA Nemotron 3.5 Lightning with OpenAI-compatible APIs, speculative decoding, and BF16/NVFP4 checkpoints across NVIDIA GPUs and edge systems.
TML Inkling on vLLM: Day-0 Support with Optimized Performance
·8 min read
vLLM brings day-0 support to TML Inkling, a 1T-parameter multimodal model, with MTP, long-context serving, parallelism, and up to 380 tokens per second per user on NVIDIA GB200 GPUs.
Announcing Day-0 Support for NVIDIA Nemotron 3 Ultra on vLLM
·7 min read
How to serve NVIDIA Nemotron 3 Ultra with vLLM for long-running agentic reasoning, including BF16 and NVFP4 checkpoints, supported GPU configurations, OpenAI-compatible deployment, and NeMo RL integration.
Run Highly Efficient Multimodal Agentic AI with NVIDIA Nemotron 3 Nano Omni Using vLLM
·7 min read
How to serve NVIDIA Nemotron 3 Nano Omni with vLLM for multimodal agentic AI, including BF16, FP8, and NVFP4 checkpoints, vision/audio/video inputs, supported GPUs, OpenAI-compatible APIs, and deployment recipes.
DeepSeek V4 in vLLM: Efficient Long-context Attention
·17 min read
A first-principles walkthrough of DeepSeek V4's long-context attention, and how we implemented it in vLLM.
Announcing Gemma 4 on vLLM: Byte for byte, the most capable open models
·3 min read
How vLLM supports Google's Gemma 4 open models across NVIDIA, AMD, Intel, and TPU backends, with multimodal inputs, agentic workflows, long context, function calling, and deployment recipes.
Run Highly Efficient and Accurate Multi-Agent AI with NVIDIA Nemotron 3 Super Using vLLM
·5 min read
How to serve NVIDIA Nemotron 3 Super with vLLM for multi-agent AI, including BF16, FP8, and NVFP4 checkpoints, 1M-token context, Thinking Budget, MTP, supported GPUs, and OpenAI-compatible deployment.
Run Highly Efficient and Accurate AI Agents with NVIDIA Nemotron 3 Nano on vLLM
·5 min read
How to serve NVIDIA Nemotron 3 Nano with vLLM for efficient agentic AI, including BF16, FP8, and NVFP4 checkpoints, 1M-token context, hybrid MoE architecture, Thinking Budget, supported GPUs, and OpenAI-compatible deployment.
Run Multimodal Reasoning Agents with NVIDIA Nemotron on vLLM
·4 min read
How to serve NVIDIA Nemotron Nano 2 VL with vLLM for multimodal reasoning agents, including video understanding, document intelligence, Efficient Video Sampling, 128K context, and OpenAI-compatible deployment.
Chasing 100% Accuracy: A Deep Dive into Debugging Kimi K2's Tool-Calling on vLLM
·11 min read
How vLLM debugged Kimi K2 tool-calling accuracy, covering chat-template compatibility, add_generation_prompt handling, schema validation failures, benchmark fixes, and tool-use reliability.
Now Serving NVIDIA Nemotron with vLLM
·5 min read
How vLLM serves NVIDIA Nemotron Nano 2 for agentic reasoning, including hybrid Transformer-Mamba architecture, thinking budget control, open weights and data, throughput benefits, and deployment commands.
DeepSeek-V3.2-Exp in vLLM: Fine-Grained Sparse Attention in Action
·8 min read
How vLLM supports DeepSeek-V3.2-Exp with DeepSeek Sparse Attention, lightning indexer caches, separate prefill and decode paths, FlashMLA sparse attention, DeepGEMM kernels, and Blackwell deployment.
vLLM Now Supports Qwen3-Next: Hybrid Architecture with Extreme Efficiency
·3 min read
How vLLM supports Qwen3-Next with hybrid attention, Gated DeltaNet, full attention, high-sparsity MoE, multi-token prediction, hybrid KV cache management, Triton kernels, and CUDA graphs.
GLM-4.5 Meets vLLM: Built for Intelligent Agents
·4 min read
How to run GLM-4.5 and GLM-4.5V with vLLM for intelligent agents, including hybrid reasoning modes, FP8 and BF16 serving, multimodal support, and NVIDIA Blackwell and Hopper deployment.
vLLM Now Supports gpt-oss
·5 min read
How vLLM supports gpt-oss 20B and 120B on NVIDIA Blackwell, Hopper, and AMD GPUs, with MXFP4 MoE kernels, efficient attention, hybrid KV cache allocation, and built-in tool support.
MiniMax-M1 Hybrid Architecture Meets vLLM: Long Context, Fast Inference
·6 min read
How vLLM serves MiniMax-M1's hybrid MoE architecture for long-context inference, covering model deployment, memory management, batched serving, backend optimizations, and Docker-based setup.
Transformers modeling backend integration in vLLM
·6 min read
How vLLM integrates the Hugging Face Transformers modeling backend to serve more model architectures efficiently, including text and vision-language models through model_impl="transformers".
Llama 4 in vLLM
·4 min read
How vLLM serves Meta Llama 4 Scout and Maverick multimodal MoE models with long-context support, tensor parallel deployment, H100 and H200 guidance, FP8 variants, and performance tips.
Introducing vLLM Inference Provider in Llama Stack
·8 min read
How Llama Stack integrates vLLM as an inference provider through remote and inline providers, enabling OpenAI-compatible vLLM serving for local and Kubernetes generative AI application deployments.
Announcing Llama 3.1 Support in vLLM
·6 min read
How vLLM supports Meta Llama 3.1 models, including 128K context, Llama 3.1 405B serving, chunked prefill, FP8 quantization, tensor and pipeline parallelism, CPU offloading, and early performance results.