vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Open on GitHubDetails
- Maker
- @vllm-project
- Language
- Python
- stars
- 89.7K
- stars today
- 71
- AI relevance
- 0.90
A high-throughput and memory-efficient inference and serving engine for LLMs
Open on GitHub