vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Open on GitHubDetails
- Maker
- @vllm-project
- Language
- Python
- stars
- 93.4K
- stars today
- 67
- AI relevance
- 0.90
A high-throughput and memory-efficient inference and serving engine for LLMs
Open on GitHub