Install
Easy, fast, and cheap LLM serving for everyone vLLM is a fast and easy-to-use library for LLM inference and serving. Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evolved into a community-driven project with contributions from both academia and industry. vLLM is fast with: Efficient management
- 137articles · 90d
- 2+ day agolatest article
- Jun 16, 2026earliest in window
- 0%with images
- 126avg words
- Science & Technology 137
- Software Dev. 134
- Computers & Electronics 130
- STEM 3
- Jobs & Education 2
- Software 2
- Business & Industrial 1
- Internet & Telecom 1
Please confirm you are human
This browser or connection looks automated. Press and continuously hold the control for 3 seconds to enable Google-hosted web results and, when separately allowed, AI-assisted answers.
A successful check enables 100 search requests. Interactive access does not authorize scraping, systematic collection, or reuse of search output.
News
fused_allreduce_rms_norm
1+ mon, 1+ week ago (104+ words) vLLM Fused all-reduce + residual-add + RMSNorm for eager model paths. This recovers a fusion that vLLM's torch.compile passes would normally do but that doesn't fire for models running eager (or under a breakable CUDA graph). All-reduce + add residual + (standard) RMSNorm,…...
vllm.model_executor.layers.attention.rswa_attention
2+ mon, 1+ week ago (86+ words) vLLM docs Attention layer that reports RSWASpec as its KV cache spec. Drop-in replacement for the standard Attention layer when the model is configured with Reference Sliding Window Attention (R-SWA, rswa_window > 0 ). The actual masking logic lives in the attention backend (FlexAttention…...