vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Open op GitHubDetails
- Maker
- @vllm-project
- Taal
- Python
- stars
- 89.7K
- stars today
- 71
- AI relevance
- 0.90
A high-throughput and memory-efficient inference and serving engine for LLMs
Open op GitHub